The Rust language provides the async/await syntax, which can turn imperative code into Future objects, but to run those Future objects, you must repeatedly call their poll method. Tokio is the piece of code that calls that method. It does so in a manner such that futures are only polled if able to continue work, and not e.g. waiting for a timer.
Besides that it provides lots of utilities for working with async code.
It's similar. libdispatch is primarily a task queue to simplify and optimize multithreaded workloads in objc/swift. Tokio uses a similar task queue pattern for scheduling cpu-bound or blocking io, but the main feature is an non-blocking/asynchronous IO runtime. Overall, it's more similar to libuv (the async io library that powers node.js), but with built-in support for Rust's async/await syntax
Yes in some regards. Both offer eventloops (in GCD: dispatch queues), which run small chunks of code which belongs to independent tasks on the same thread.
However there are some differences:
- Tokio is focussed on running async/await based code, whereas libdispatch currently mostly targets running callbacks/continuations. This might change once Swift offers async/await support, which for sure could run on top of GCD queues.
- GCD queues provide a lot more fine-grained control. users can exactly specify on which queue to run some code on. And tasks can jump between code. There might also be a dedicated main thread (UI) queue. Tokio just spins up a single queue which runs all code, which might be empowered by a multithreaded executor. This makes it less usable for UI.
Yes, but it's strictly userspace. It's more like the concurrency primitives in the JVM. (Eg. work-stealing threadpool, timers, various abstractions over OS/kernel lower level async stuff.)
Rust used to have a lot of stuff built into the language, including garbage collection![0] I think that a long time ago they chose to move stuff out into libraries so that Rust could be a serious competitor to C++.
Well, I took the phrase 'repeatedly call' to mean pretty much that!
What bugs me about it, if I'm understanding it correctly, is that you have two options once an async call is issued:
- the calling thread effectively waits for completion. This is fine if a fork/join pattern is useful to you (i.e. issue N async calls and then wait for N completions). This isn't proper asynchrony though.
- the future is pushed on to a thread that does nothing but poll for completions. This effectively imposes an O(n) inefficiency into your code.
I can't speak to the same level of depth about the C++ model as the Rust one, but, while you could do those things, it's not the usual way that it works, at least, if I'm understanding your terms correctly. I'll admit that I find your terms in the first bullet pretty confusing, and the second, only slightly. Let's back up slightly. You have:
* A future. You can call poll on a future, and it will return you either "not yet" or "done." This API is provided by the standard library. You can create futures with async/await as well, which is provided by the language. These tend to nest, so you can end up with one big future that's composed out of smaller futures.
* A task. Tasks are futures that are being executed, rather than being constructed. Creating a task out of a future may place the future on the heap.
* An executor. This is provided by Tokio. By handing a future to Tokio's executor, you create a task. The job of the executor is to keep track of all tasks, and decide which one to call poll on next.
* A reactor. This is also provided by Tokio. An executor will often employ a reactor to help decide which task to execute and when. This is sometimes called an "event loop," and coordinates with the operating system (or, if you don't have one of those, the hardware) to know when something is ready.
* A Waker. When you call poll on a future, there's one more bit that happens we couldn't talk about until we talked about everything else. If a future is going to return "not yet," it also constructs a Waker. The Waker is the bridge between the task, the reactor, and the executor.
So. You have a task. That task needs to get something from a file descriptor in a non-blocking way. At some point, there's a future way down in the chain whose job it is to handle the file descriptor. When you ask it to be created, it will return "not ready", and construct a waker that uses epoll (or whatever) via the reactor. At some point, the data will be ready, and the reactor will notice, and tell the executor "hey this task is now ready to execute again," and when some time is free, the executor will eventually call poll on it a second time. But until that point, the executor knows it's not ready, and so won't call poll again.
Whew. Does that make sense? I linked my talks in this thread already, but this is kind of a re-hash of them.
This is an awesome rundown of the whole stack. It's almost like you've explained this stuff before. ;)
It might sound complicated, but for typical applications almost all of this happens "under the hood". Usually you'll just add an attribute to `main` to start your runtime, then you can compose/await futures without ever needing to think about `poll` and friends.
Besides that it provides lots of utilities for working with async code.