Project Loom and Structured Concurrency • Hacker News Board

Re: Project Loom and Structured Concurrency

#51

Post by pron » Fri, Dec 04, 2020, 3:53 PM UTC

ragnese wrote: post #47

pron wrote: post #45

Earlier quoted context omitted.

> I agree with your point that tacking all of this onto the existing (flawed) Thread API is a risky move. This is not what Loom does, though. Virtual threads are not using the thread API. They are (Java) threads; no more and no less than today's threads. Just as people don't normally use the java.lang.Thread API directly to use today's threads, there's no reason why they should use it with virtual threads. > One adva…

> This is not what Loom does, though. Virtual threads are not using the thread API. They are (Java) threads; no more and no less than today's threads. Just as people don't normally use the java.lang.Thread API directly to use today's threads, there's no reason why they should use it with virtual threads. Right. That's fair. They are threads, but it's just that now you have two "kinds" of Thread, where before you had…

> So having virtual threads and "raw" threads under the same class has pros and cons, IMO.

That's one way to think about it. Another is that Java never gives you "OS threads" it always gives you Java threads, an abstraction with multiple possible implementations. One implementation is no more real or raw than the other (in fact, you could even theoretically make virtual threads the carriers for other virtual threads -- a thread is a thread, after all -- but we explicitly blocked that because it's not useful). There is no real difference between that and ArrayList and LinkedList both implementing the same List interface. They're both just as real, but they have different footprint and performance (the class hierarchy for threads is slightly different, but for uninteresting technical reasons).

> Or do you still disagree?

I still disagree. That network call might take 1ms, and that bit fiddling might trigger a GC collection that takes 10 times that or more. Moreover, neither Kotlin nor C# mark long subroutines with a different colour, and they don't even mark blocking calls with a different colour, just the flavour of them that's to be used with coroutines. The real reason that colour is necessary is because of the way the feature is implemented.

Originally, that colour meant to signify something else: nondeterminism in mostly-deterministic languages like Haskell, and it is also important in JavaScript. Trying to retroactively find a useful meaning for it in Java is an excuse.

Re: Project Loom and Structured Concurrency

#52

Post by Cojen » Fri, Dec 04, 2020, 4:02 PM UTC

pron wrote: post #41

ryanworl wrote: post #38

Earlier quoted context omitted.

Thanks! Is there an example "Hello World" executor that demonstrates the minimum functionality needed? I would like an executor that runs all tasks on a single thread and schedules them deterministically for testing.

var myThreadFactory = Thread.builder().virtual(Executors.newSingleThreadExecutor()).factory(); All threads created by this thread factory will be scheduled to the same kernel thread.

Does this example fully answer the question? Does this allow the tasks to be scheduled deterministically? If I'm very careful and write the tasks such that they make blocking calls at specific locations, then yes. Otherwise, is there any feedback to inform me that tasks are switching context at places I didn't expect? Is it possible to define a custom scheduler that can simply print debug messages at every context switch?

Re: Project Loom and Structured Concurrency

#53

Post by ragnese » Fri, Dec 04, 2020, 4:04 PM UTC

pron wrote: post #48

ragnese wrote: post #44

Earlier quoted context omitted.

> Syntactic coroutines, on the other hand, were something to avoid. Could you articulate or point me to some of the arguments or thoughts that lead to the conclusion here? I know and understand the term "colored functions", but I'd love to read a real analysis of the pros and cons of colored functions, because I personally go back and forth on whether I think they are bad or good. On the one hand, having a function t…

> Could you articulate or point me to some of the arguments or thoughts that lead to the conclusion here? I gave a talk about exactly that recently at Code Mesh. I expect them to post the video soon. https://codesync.global/speaker/ron-pressler/#745why-user-mo... > Also, this statement makes you sound like Goliath. Maybe, but it's not just a matter of size but also trajectory. And, as you say, our competition is most…

You didn't answer a few of the interspersed questions, so I'll press you on the records one.

What about Java records are different than Kotlin's data classes besides forgoing the auto-generated `copy()` method? I understand there are some implementation details, such as inheriting from a Record base class, and how it handles serialization. But I mean as a user.

Can records implement interfaces? Can records be variants in a sealed class hierarchy? Can records have a private primary constructor? Can I customize the getters (to e.g., make defensive copies)?

For other people reading, the answers for Kotlin data classes are: Yes, Yes, Not Really, No.

As an aside, I understand you are an Oracle employee. When you post under this username are you acting in any kind of official capacity for Oracle? Like, is part of your job, so to speak, to have a social media presence? I'm not suggesting anything negative- it's totally fair to invest in outreach, to answer questions, clarify things, etc. I was just curious if I'm talking to "a guy who loves the project he works on" or "a representative of a company".

Re: Project Loom and Structured Concurrency

#54

Post by throwaway894345 » Fri, Dec 04, 2020, 4:09 PM UTC

jnwatson wrote: post #29

I have a lot of experience using concurrency in Go, and for the last couple years have been at the bleeding edge of Python async. The tradeoffs between the two approaches are immense. With the virtual thread model you have: * No function coloring problem. This also means existing code is easier to port. * possibility of transparent M:N scheduling. * Impedence mismatch with OS primitives. * Much more sophisticated run…

I've had similar experiences, but I don't much care for async Python. In particular, it's way too easy to block the event loop either by accidentally calling some function that, perhaps transitively, does blocking I/O (this could be remedied if there was no sync IO) or simply by calling a function which is unexpectedly CPU-bound. And when this happens, other requests start failing unrelated to the request that is causing the problem, so you go on this wild goose chase to debug. Sync I/O is also a much nicer, more ergonomic interface than async IMO. And then there are the type error problems--it's way too easy to forget to `await` something. Mypy could help with this, but it's still very, very immature. Lastly, last I checked the debugger couldn't cope with async syntax--this is obviously not criticizing the async approach in general, but I wanted to round out my complaining about async Python.

I don't mind working with goroutines personally--I use them sparingly, only when I really need concurrency or parallelism. This takes some discipline (e.g., not to go crazy with goroutines and/or channels) and a bit of experience (in the presence of multiple goroutines, what needs to be locked, when to use channels, etc), so if you're relatively new and very impatient or undisciplined you probably won't have a good time (which isn't to say that if you dislike goroutines you must be a novice or undisciplined!). But for me it's nearly an ideal experience.

Re: Project Loom and Structured Concurrency

#55

Post by pron » Fri, Dec 04, 2020, 4:13 PM UTC

Cojen wrote: post #52

pron wrote: post #41

Earlier quoted context omitted.

var myThreadFactory = Thread.builder().virtual(Executors.newSingleThreadExecutor()).factory(); All threads created by this thread factory will be scheduled to the same kernel thread.

Does this example fully answer the question? Does this allow the tasks to be scheduled deterministically? If I'm very careful and write the tasks such that they make blocking calls at specific locations, then yes. Otherwise, is there any feedback to inform me that tasks are switching context at places I didn't expect? Is it possible to define a custom scheduler that can simply print debug messages at every context sw…

It answers the question with a "hello, world." To do more sophisticated stuff, like what you want, you'll need to replace or wrap the standard single-thread Executor with your own Executor. There is exactly one method you need to implement. For example:

    Executor ste = Executors.newSingleThreadExecutor();
    Executor myExecutor = task -> {
      if (task instanceof Thread.VirtualThreadTask vtt) System.out.println("Scheduling " + vtt.thread() + " on " + Thread.currentThread());
      ste.execute(task);
      if (task instanceof Thread.VirtualThreadTask vtt) System.out.println("Descheduled " + vtt.thread() + " from " + Thread.currentThread());
    }
    var myThreadFactory = Thread.builder().virtual(myExecutor).factory();

Re: Project Loom and Structured Concurrency

#56

Post by earthboundkid » Fri, Dec 04, 2020, 4:18 PM UTC

jerf wrote: post #34

jnwatson wrote: post #29

I have a lot of experience using concurrency in Go, and for the last couple years have been at the bleeding edge of Python async. The tradeoffs between the two approaches are immense. With the virtual thread model you have: * No function coloring problem. This also means existing code is easier to port. * possibility of transparent M:N scheduling. * Impedence mismatch with OS primitives. * Much more sophisticated run…

I don't think "task cancellation" is quite the major difference you think. If you model it as thread A wants to cancel thread B, then while threading means that A runs and cancels B, but B may need some time to catch up, the async world has the problem of thread A running at all to cancel B, if B is having a problem that requires cancellation. It's "obvious" and "safe" until it doesn't happen at all. This is a pervas…

Cancellation is a little tricky. If things can cancel at any point, then it's impossible to write safe/correct code. Async has the advantage that the await calls are natural sync points, so it's (probably) safe to cancel there. But historical experience has also shown that people will forget to yield when they should, and it will lead to cooperation problems.

I think the best approach has to look something like Go's, but perhaps a bit more structured (dynamic scoping[1] might help perhaps with task nurseries[2]). Unless you're writing extremely low level code, you want your language runtime to intercept all syscalls and figure out the async story for you. The language should handle making sure that the M:N mapping works out, no one opens a socket the wrong way etc. Then for you as the program writer, your responsibility is just setting explicit cancellation points as part of the general error handling approach. It's still not perfect, but I think that would be the next evolution from what exists today.

[1] https://blog.merovius.de/2017/08/14/why-context-value-matter...

[2] https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

Re: Project Loom and Structured Concurrency

#57

Post by ragnese » Fri, Dec 04, 2020, 4:19 PM UTC

pron wrote: post #51

ragnese wrote: post #47

Earlier quoted context omitted.

> This is not what Loom does, though. Virtual threads are not using the thread API. They are (Java) threads; no more and no less than today's threads. Just as people don't normally use the java.lang.Thread API directly to use today's threads, there's no reason why they should use it with virtual threads. Right. That's fair. They are threads, but it's just that now you have two "kinds" of Thread, where before you had…

> So having virtual threads and "raw" threads under the same class has pros and cons, IMO. That's one way to think about it. Another is that Java never gives you "OS threads" it always gives you Java threads, an abstraction with multiple possible implementations. One implementation is no more real or raw than the other (in fact, you could even theoretically make virtual threads the carriers for other virtual threads…

Fair point about the abstraction level of a Java thread vs an OS thread.

In light of both things you just wrote, let me ask you this: why does Java give us any choice on thread implementation? Why not have everything be a green thread from now on?

If Java threads are not OS threads, can be paused for any amount of time, and there's no reason that network calls or db calls or file IO should be treated any differently, then I'm not sure why the old Java Threads shouldn't be deprecated.

Can you shine some light on that? After Loom drops, when would I ever want something other than a green thread?

Re: Project Loom and Structured Concurrency

#58

Post by pron » Fri, Dec 04, 2020, 4:22 PM UTC

ragnese wrote: post #53

pron wrote: post #48

Earlier quoted context omitted.

> Could you articulate or point me to some of the arguments or thoughts that lead to the conclusion here? I gave a talk about exactly that recently at Code Mesh. I expect them to post the video soon. https://codesync.global/speaker/ron-pressler/#745why-user-mo... > Also, this statement makes you sound like Goliath. Maybe, but it's not just a matter of size but also trajectory. And, as you say, our competition is most…

You didn't answer a few of the interspersed questions, so I'll press you on the records one. What about Java records are different than Kotlin's data classes besides forgoing the auto-generated `copy()` method? I understand there are some implementation details, such as inheriting from a Record base class, and how it handles serialization. But I mean as a user. Can records implement interfaces? Can records be variant…

> What about Java records are different than Kotlin's data classes

Java records, like enums (another feature that is philosophically very similar to records) aim not to reduce the boilerplate of certain operations, but to designate a subset of classes with particular semantic properties (and make those easy to express). In the case of enums, that subset is classes with a well-known, fixed set of instances; for records that is nominal tuples, i.e. immutable, unencapsulated data aggregates, similar to ML's product types. So users, the compilers, and libraries can make certain assumptions about records. For example, their semantic properties (that they are no more than a product of their component types) allow a much better serialization story for them and, indeed, record classes are serialized differently from non-record classes: Instead of invoking a no-arg constructor, their canonical constructor is invoked on deserialization. Their immutability also makes automatic implementations of equality, deconstruction and pattern matching clear and correct.

Just like Kotlin couldn't do user-mode threads efficiently because they have no control over the platform, there was also little point in doing records, because that language's goal was to make it easier, syntactically, to work with the existing Java ecosystem, and, prior to records, Java programmers worked with JavaBean-like classes, so Kotlin tried to make those operations more syntactically pleasant. Java's designers, however, can have an impact on what that ecosystem does and can change its direction.

> As an aside, I understand you are an Oracle employee.

Yes. I work on OpenJDK.

> When you post under this username are you acting in any kind of official capacity for Oracle?

Absolutely not. I speak only for myself. I'm the technical lead for Project Loom, and I want to see what kind of reactions people have to it on social media (as well as conferences, customer meetings, surveys etc.). I guess I see it indirectly as part of my job, at least as far as Loom goes, as these interactions help inform how we explain the capabilities, what features people want etc.. It's nothing official, though. It's also a harmful personal addiction.

Re: Project Loom and Structured Concurrency

#59

Post by pron » Fri, Dec 04, 2020, 4:31 PM UTC

ragnese wrote: post #57

pron wrote: post #51

Earlier quoted context omitted.

> So having virtual threads and "raw" threads under the same class has pros and cons, IMO. That's one way to think about it. Another is that Java never gives you "OS threads" it always gives you Java threads, an abstraction with multiple possible implementations. One implementation is no more real or raw than the other (in fact, you could even theoretically make virtual threads the carriers for other virtual threads…

Fair point about the abstraction level of a Java thread vs an OS thread. In light of both things you just wrote, let me ask you this: why does Java give us any choice on thread implementation? Why not have everything be a green thread from now on? If Java threads are not OS threads, can be paused for any amount of time, and there's no reason that network calls or db calls or file IO should be treated any differently,…

The first part of the answer is the same as that for a similar question we've been asked about LinkedList: we don't deprecate things that are heavily used, however useless or superseded by something else, unless they are very harmful. Deprecation in Java does not mean "unrecommended" but "absolutely do not use this if you want your program to continue working on future versions." Not only is it a compile-time warning, but JDK tooling even checks for uses of deprecated APIs in binaries and warns about them. Ideally, deprecated usages should break people's builds. In other words, deprecation is a big deal and not taken lightly. We might need another standardised term for "unrecommended" or "superseded".

The second part of the answer is that there are still good uses cases for heavyweight threads, that are backed by one OS thread. For example, as carriers for virtual threads or parallel streams, i.e. as approximations of CPU cores, and also for cases where FFI is used for IO or some other native interaction. This is very rare in Java, but very rare could mean "used only in tens of thousands of programs rather than tens of millions".

Re: Project Loom and Structured Concurrency

#60

Post by lackbeard » Fri, Dec 04, 2020, 4:36 PM UTC

mping wrote: post #30

For me the real advantage is not on performance but on the programming model. I have been tinkering with Loom (and clojure) and the idea of "just" calling some library without worrying about blocking is refreshing. That means that for the most of it, you can write your code without worrying too much about some kind of callbacks or async support from your library and it just works. Of course, for those with extreme pe…

I think the vast majority of JVM users won't even need Loom. OS threads perform well enough for most use cases. You can go a very long way with just a ThreadPoolExecutor.