For rust, fibers (as a user-space, cooperative concurrency abstraction) would mandate a lot of design choices, such as whether stacks should be implemented using spaghetti stacks or require some sort of process-level memory mapping library, or even if they were just limited to a fixed size stack.
All three of these approaches would cause issues when interacting with code in another language with a different ABI. It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another.
One of the benefits of async/await is the 'await' keyword itself. The explicit wait-points give you the ability to actually reason about the interactions of a concurrent program.
Yielding fibers are a bit like the 'goto' of the concurrency world - whenever you call a method, you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed. The need to be defensive when interfacing with the outside world means fibers tend to be better for tasks which run in isolation and communicate by completion.
Green threads, fibers and coroutines all share the same set of problems here, but really user space cooperative concurrency is just shuffling papers on a desk in terms of solving the hard parts of concurrency. Rust async/await leaves things more explicit, but as a result doesn't hide certain side effects other mechanisms do.
> you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed
This may be true in JS (or Haskell), but not in Rust, where you already have multithreading (and unrestricted side-effects), and so other code may always be interleaved. So this argument is irrelevant in languages that offer both async/await and threads.
Furthermore, the argument is weak to begin with because the difference is merely in the default choice. With threads, the default is that interleaving may happen anywhere unless excluded, while with async/await it's the other way around. The threading approach is more composable, maintainable, and safer, because the important property is that of non -interference, which threads state explicitly. Any subroutine specifies where it does not tolerate contention regardless of other subroutine it calls. In the async/await model, adding a yield point to a previously non-yielding subroutine requires examining the assumptions of its callers. Threads' default -- of requiring an explicit statement of the desired property of interference is the better one.
I never fully understood the FFI issue. When calling an not-known coroutine safe FFI function you would switch form the coroutine stack to the original thread stack and back. This need not be more expensive than a couple of instructions on the way in and out.
Interestingly, reenabling frame pointers was in the news recently, which would add a similar amount of overhead to every function call. That was considered a more than acceptable tradeoff.
The main problem with fibers/goroutines and FFI is that one of the benefits of fibers is that each fiber starts with a very small stack (usually just a few kBs) unlike native threads usually starting with a much larger stack (usually expressed in MB). The problem is that the code must be prepared to grow the stack if necessary, which is not compatible with the C FFI. That's one of the reasons why Go's FFI to C, for example, is slower than Rust.
Sure, if you are using split stacks, goroutine code is inherently slower. But for FFI you would switch to the main thread stack that is contiguous, so you won't pay any split stack cost there.
Go abandoned split stacks years ago due to the "hot split problem", and is now using contiguous stacks that are grown when necessary via stack copying. Go switches to the system stack when calling C code. There is some overhead (a few tens of ns) due to that switch, compared to languages like Rust or Zig which don't need to switch the stack.
The issue is that the FFI might use C thread-local storage and end up assuming 1-1 threading between the language and C, but if you're using green threads then that won't be the case.
In my opinion, by default fibers should use "full" stacks, i.e. a reasonable amount of unpopulated memory pages (e.g. 2 MiB) with guard page. Effectively, the same stack which we use for threads. It should eliminate all issues about interfacing with external code. But it obviously has performance implications, especially for very small tasks.
Further, on top of this we can then develop spawning tasks which would use parent's stack. It would require certain language development to allow computing maximum stack usage bound of functions. Obviously, such computation would mean that programmers have to take additional restrictions on their code (such as disallowing recursion, alloca, and calling external functions without attributing stack usage), but compilers already routinely compute stack usage of functions, so for pure Rust code it should be doable.
>It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another.
It's a weird example. How would a C library know about fiber runtime used in Rust?
>Yielding fibers are a bit like the 'goto' of the concurrency world - whenever you call a method, you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed.
I find this argument funny. Why don't you have the same issue with preemptive multitasking? We live with exactly this "issue" in the threading world just fine. Even worse, we can not even rely on "critical sections", thread's execution can be preempted at ANY moment.
As for `await` keyword, in almost all cases I find it nothing more than a visual noise. It does not provide any practically useful information for programmer. How often did you wonder when writing threading-based code about whether function does any IO or not?
> It’s a weird example. How would a C library know about the fiber runtime used in Rust.
Well, if you green thread were launched on an arbitrary free OS thread (work stealing), then for example your TLS variables would be very wrong when you resume execution. Does it break all FFI? No. But it can cause issues for some FFI in a way that async/await cannot.
> I find this argument funny. Why don't you have the same issue with preemptive multitasking? We live with exactly this "issue" in the threading world just fine. Even worse, we can not even rely on "critical sections", thread's execution can be preempted at ANY moment.
It’s not about critical sections as much. Since the author referenced go to, I think the point is that it gets harder to reason about control flow within your own code. Whether or not that’s true is debatable since there’s not really any implementation of green threads for Rust. It does seem to work well enough for Go but it has a required dedicated keyword to create that green thread to ease readability.
> As for `await` keyword, in almost all cases I find it nothing more than a visual noise. It does not provide any practically useful information for programmer. How often did you wonder when writing threading-based code about whether function does any IO or not?
Agree to disagree. It provides very clear demarcation of which lines are possible suspension points which is important when trying to figure out where “non interruptible” operations need to be written for things to work as intended.
Not sure why you’re starting a totally unrelated debate. If you’re pulling in a library via FFI, you have no control over what that library has done. You’d have to audit the source code to figure out if they’ve done anything that would be incompatible with fibers. And TLS is but one example. You’d have to audit for all kinds of OS thread usage (e.g. if it uses the current thread ID as an index into a hashmap or something). It may not be common, but Go’s experience is that there’s some issue and the external ecosystem isn’t going to bend itself over backwards to support fibers. And that’s assuming that these are solved problems within your own language ecosystem which may not be the case either when you’re supporting multiple paradigms.
> In my opinion, by default fibers should use "full" stacks, i.e. a reasonable amount of unpopulated memory pages (e.g. 2 MiB) with guard page.
That's a disaster. If you're writing a server that needs to serve 10K clients concurrently then that's 20GiB of RAM just for the stacks, plus you'll probably want guard pages, and so MMU games every time you set up or tear down a fiber.
The problem with threads is the stacks, the context switches, the cache pressure, the total memory footprint. A fiber that has all those problems but just doesn't have an OS schedulable entity to back it barely improves the situation relative to threads.
Dealing with slow I/O is a spectrum. On one end you have threads, and on the other end you have continuation passing style. In the middle you have fibers/green threads (closer to threads) and async/await (closer to CPS). If you want to get closer to the middle than threads then you want spaghetti stack green threads.
> by default fibers should use "full" stacks, i.e. a reasonable amount of unpopulated memory pages (e.g. 2 MiB) with guard page
Then wouldn't we loose the main benefit of fibers (small stacks leading to a low memory usage in presence of a very large number of concurrent tasks) compared to native threads (the other main benefit being user-space scheduling)? Or perhaps you're thinking of using fibers configured with a very small stack for highly concurrent tasks (like serving network requests) and delegating tasks requiring C FFI to a pool of fibers with a "full" stack?
For rust, fibers (as a user-space, cooperative concurrency abstraction) would mandate a lot of design choices, such as whether stacks should be implemented using spaghetti stacks or require some sort of process-level memory mapping library, or even if they were just limited to a fixed size stack.
> All three of these approaches would cause issues when interacting with code in another language with a different ABI. It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another.
Java (in OpenJDK 21) is doing it. To be fair, Java had no other choice because there is so much sequential code written in Java, but also the Java language and bytecode compilation makes it easy to implement spaghetti stacks transparently. Given those two things it's obviously a good idea to go with green threads. The same might not apply to other languages.
My personal preference is for async/await, but it's true that its ecosystem bifurcating virality is a bit of a problem.
All three of these approaches would cause issues when interacting with code in another language with a different ABI. It can get really complicated, for example, when C code gets called from one fiber and wants to then resume another.
One of the benefits of async/await is the 'await' keyword itself. The explicit wait-points give you the ability to actually reason about the interactions of a concurrent program.
Yielding fibers are a bit like the 'goto' of the concurrency world - whenever you call a method, you don't know if as a side effect it may cause your processing to pause, and if when it continues the state of the world has changed. The need to be defensive when interfacing with the outside world means fibers tend to be better for tasks which run in isolation and communicate by completion.
Green threads, fibers and coroutines all share the same set of problems here, but really user space cooperative concurrency is just shuffling papers on a desk in terms of solving the hard parts of concurrency. Rust async/await leaves things more explicit, but as a result doesn't hide certain side effects other mechanisms do.