discussion was about specific context: avoiding overhead from spawning millions of threads, in this case you shouldn't have any blocking code at all, all API should utilize epoll underneath or something similar.
there are tons of variations, depending on your logic and API. The closest to virtual threads is ForkJoinPool and RecursiveTask, where you can have code like regular blocking code:
var f = async_api_returns_future();
...
var res = f.join();
but join() won't block OS/JVM thread, but make it to perform other tasks in the queue.
Or you can design API which will receive executorService as params, and run callback there, e.g.:
One way to see how different virtual threads are from our old mechanisms is to ask yourself how many IO operations you can have in flight. There are two options: either the operations are blocking, in which case the number will be equal to the (very limited) number of threads in all of your thread pools combined, or the operations are non-blocking, in which case thread context that is so necessary for troubleshooting and JFR profiles is lost (e.g. JFR can't know on behalf of whom is some IO operation performed because the "owner" of some operation -- in the design of the Java platform -- can only be a thread). Virtual threads allow you to have hundreds of thousands (or even millions) of I/O operations in flight (which you need for high throughput as a result of Little's law) while still preserving observable context.
BTW, as for fork-join's `join`, not only is it designed for pure computation only and cannot help with IO, but every `join` increases the depth of the stack, so it is fundamentally limited in how much it can help. FJ is designed for pure computation workloads, so in practice that's not a huge limitation, but virtual threads are designed for IO workloads.
I apologise for not going into more depth here, but as you can imagine, with a user base numbering in the many millions, we can only afford to put in depth explanations in texts and videos that gain a large audience, but once you've familiarised yourself with the material I'll gladly answer specific questions (and perhaps your questions will help us improve our material, too).
> I apologise for not going into more depth here, but as you can imagine, with a user base numbering in the many millions
my concern is that you somehow can find time to write long comments with lots of handwavings (our framework is designed for that, and their framework is not designed for that), but refuse to provide specific code pointers and example in support of your opinion. For example, in this specific case, can you give example how green threads can be used with current Java IO library, or Java JDBC library?
If there's something unclear in that material, please ask. Also, there is no our framework vs. their framework here. I'm only discussing the JDK's own various thread pools vs. virtual threads. They were all designed and implemented by the JDK team.
BTW, I'm not trying to support any opinion. Pretty much all popular Java server frameworks are adopting virtual threads, and Java's virtual threads are poised to become the most popular lightweight user mode threads. We've already convinced everyone that needed convincing. I'm merely offering pointers in case you're interested to learn how to use virtual threads and understand how they offer high throughput and good observability at the same time (whereas before you could have one or the other). Of course, if you're satisfied with the throughput and observability you can get with our old mechanisms, you don't have to use virtual threads. We've not taken anything away.
try (var in = url.openStream()) {
return new String(in.readAllBytes(), StandardCharsets.UTF_8);
}
which claims that this example will scale well with virtual threads, my understanding is that in.readAllBytes() will call OS blocking socket API underneath, which will block OS thread, so you would need many OS threads to scale. Is this understanding correct?
It is not. Blocking IO (with some exceptions mentioned in the JEP) will automatically be translated by the runtime into non-blocking IO when it occurs on virtual threads, and no OS threads will be blocked. The Java code will look blocking and that's what thread dumps and other Java observability mechanisms will show, but to the OS it will seem as if it's running non-blocking code.
You can't do that with thread pools. You could achieve that scalability with async code, but then observability tools will not be able to track the IO operations and who initiated them, but with virtual threads you'll see exactly what business operation is doing what IO and why.
> will automatically be translated by the runtime into non-blocking IO when it occurs on virtual threads, and no OS threads will be blocked
it looks like it is true for several API you implemented support for. What about other API, for example some JDBC driver which wants to use non-blocking DB driver. How to use virtual threads with that?
JDBC drivers are implemented on top of JDK APIs and so will work the same way: their I/O would automatically be non-blocking when run on a virtual thread (module some quality-of-implementation issues around the use of synchronized that we're working on, which are mentioned in the material I linked to).
JDBC drivers that are implemented on top of their own native code are a different matter, but they are not common these days.
