Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thread Pools on the JVM

gist.github.com

Post reply on HN
Print view View original

91–100 of 120 posts

Re: Thread Pools on the JVM

#91

Post by lmilcin »

cogman10 wrote: post #2

Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…

I have discovered ReactiveX for Java and Reactor in particular.

I am working with Kafka and MongoDB and it is normal for my app to have a million in flight transactions at various stages of completion.

In the past it required a lot of planning (and a lot of code) but Reactor let's me build these processes as pipelines with whatever concurrency or scheduler I desire, at any stage of the processing.

We are even doing tricks like merging unrelated queries to MongoDB so that sometimes thousands of same queries are executed together (one query with huge in() or one bulk write rather than separate ones).

This is improving our throughputs by orders of magnitude while the pipeline pulls millions of documents per second from the database.

I just don't see how Loom helps.

Loom could help if you had blocking APIs to start, but you get much better results if you just resolve to use async, non-blocking wrapped in ReactiveX.

Re: Thread Pools on the JVM

#92

Post by Thaxll »

jeffbee wrote: post #61
dragontamer wrote: post #56

Earlier quoted context omitted.

Hmmm. It seems like you're taking this from a perspective of "Pthreads in C++ vs Coroutines in Go", which is correct in some respects, but different from how I was taking the discussion. I guess I was taking it from a perspective of "pthreads in C++ vs Go-like coroutines reimplemented in C++", which would be pthreads vs C++20 coroutines. (Or really: it seems like this "Loom" discussion is more of a Java thing but pro…

Right, I admit there are better ways to do it, but I don't think it's obviously true that goroutines specifically are either more compact or faster to switch between. The benefits might be imaginary. The Go runtime has a thread scheduler that kinda sucks actually (it scales badly as the number of runnable goroutines increases) and there are also ways of making native threads faster, like SwitchTo https://lkml.org/lkm…

Have you tried to context switch between 100k native threads? Good luck with that, in the mean time Go has no problems doing that with 1m goroutines.

Re: Thread Pools on the JVM

#93

Post by geodel »

lmilcin wrote: post #91
cogman10 wrote: post #2

Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…

I have discovered ReactiveX for Java and Reactor in particular. I am working with Kafka and MongoDB and it is normal for my app to have a million in flight transactions at various stages of completion. In the past it required a lot of planning (and a lot of code) but Reactor let's me build these processes as pipelines with whatever concurrency or scheduler I desire, at any stage of the processing. We are even doing t…

Loom will help folks who prefer writing straightforward Java code instead of some random reactive library with obscure exception handling and poor to impossible debuggability.

Now I get it is hard for many folks to understand that part. Just like at my workplace people think it is impossible to write micro service without SpringBoot.

> Loom could help if you had blocking APIs to start, but you get much better results if you just resolve to use async, non-blocking wrapped in ReactiveX.

There might be billions of lines of legacy code which would adapt to Loom with minimal changes but impossible to turn in ReactiveX etc without enormous investment and risk. Your ideas are rather simplistic for real world.

Re: Thread Pools on the JVM

#94

Post by tadfisher »

WatchDog wrote: post #90

If your app is fully non-blocking, doesn't it make sense to just do everything on the one pool, CPU bound tasks and IO polling. Rather than passing messages between threads.

"Fully non-blocking" means "does no work". Ignoring the process' spawning thread, if your app performs CPU-bound tasks on a bounded thread pool, you will be leaving I/O throughput on the table as the number of tasks increases, since I/O-bound tasks will block on waiting for a thread.

Re: Thread Pools on the JVM

#95

Post by avereveard »

christkv wrote: post #15
cogman10 wrote: post #2

Loom can't land fast enough! The current issue the JVM has is that all threads have a corresponding operating system thread. That, unfortunately, is really heavy memory wise and on the OS context switcher. Loom allows java to have threads as light weight as a goroutine. It's going to change the way everything works. You might still have a dedicated CPU bound thread pool (the common fork join pool exists and probably…

Are we coming full circle going back a variant of the original Java green threads?

More like the many-to-many threading model of the Solaris implementation of the JVM

Re: Thread Pools on the JVM

#96

Post by lmm »

pron wrote: post #47
Spivak wrote: post #34

Earlier quoted context omitted.

So loom uses interesting terminology when talking about this. They say that they’re preemptive and not cooperative because there’s not an explicit await/yield keyword that you call from your code but that isn’t the whole story because threads are only preempted when they perform IO or are synchronized. So you as an author can’t know for sure where the yield points are and aren’t supposed to rely on them but they’re s…

> So loom uses interesting terminology when talking about this. That is a common terminology. Wikipedia says: [1] The term preemptive multitasking is used to distinguish a multitasking operating system, which permits preemption of tasks, from a cooperative multitasking system wherein processes or tasks must be explicitly programmed to yield when they do not need system resources. ... The term "preemptive multitasking…

Isn't the point of preemption to degrade gracefully when the cores are oversubscribed? E.g. the first system I worked on ran potentially CPU-heavy work from various clients, and used per-client threads to isolate them; every so often clients would find ways to get their thread stuck doing a large amount of CPU work (e.g. regex backtracking) and although these were in some sense bugs (and we did fix them), it was very useful that even if one or two clients blocked all their threads (which was often more than our number of physical cores), this wouldn't completely block other clients' threads from running.

Re: Thread Pools on the JVM

#97

Post by jfoutz »

dimitrov wrote: post #87
jeffbee wrote: post #41

Earlier quoted context omitted.

Unbounded thread pools are bad, bounded thread pool executors with unbounded work queues are bad, and bounded thread pools with bounded queues, FIFO policies, and silent drops are also bad. There are many bad ways to do this.

> and bounded thread pools with bounded queues, FIFO policies, and silent drops are also bad. Care to elaborate please? Seems like the author is recommending unbounded thread pools with bounded queues for blocking IO. Isn't that pretty similar?

I can't speak for the parent, some things that stand out to me

1. k8s and bare metal, when you make a bunch of threads things get slower. with the FIFO case, you can have pending requests in the queue that don't get their connection canceled event, and the same user puts another request in the queue.

2. Silently dropping is bad, you want an alert - really you want an alert when you get close, so you can add more capacity

3. bounded queue with unbounded threads is really just an unbounded queue - a short line with a mob pushing to get in line

Then, you know, memory on k8s, pod gets OOM killed. that sucks cause you have to reschedule and restart. all the pending requests are dropped.

It's very easy to make something that works, but is actually quite detrimental when things are on fire. little extra gasoline helps get over the hills, but when things are on fire, gasoline makes a bigger fire.

Re: Thread Pools on the JVM

#98

Post by cbsmith »

paulddraper wrote: post #89
cbsmith wrote: post #52

Earlier quoted context omitted.

> That, unfortunately, is really heavy memory wise and on the OS context switcher. So, there was a time where a broad statement like that was pretty solid. These days, I don't think so. The default stack size (on 64-bit Linux) is 1MB, and you can manipulate that to be smaller if you want. That's also the virtual memory. The actually memory usage depends on your application. There was a time where 1MB was a lot of mem…

Yep. Granted there are scenarios where you want 100,000 "threads of execution." And that clearly is going to be impractical for system threads. But if your worried about the overhead of your pool of 50 threads, stop it.

> Granted there are scenarios where you want 100,000 "threads of execution." And that clearly is going to be impractical for system threads.

100,000 was impractical in the 2000's. Today, even with the default Java stack size of 1MB, 100,000 * 1MB = 100 GB of virtual memory. For IO bound tasks, actual memory usage would typically be a fraction of that, possibly under 2GB. That's definitely practical for a modern server.

> But if your worried about the overhead of your pool of 50 threads, stop it.

Yeah, people seem to misunderstand how thread pools work out these days. They're more limits on concurrency than anything else.

Re: Thread Pools on the JVM

#99

Post by cbsmith »

vbezhenar wrote: post #79
cbsmith wrote: post #52

Earlier quoted context omitted.

> That, unfortunately, is really heavy memory wise and on the OS context switcher. So, there was a time where a broad statement like that was pretty solid. These days, I don't think so. The default stack size (on 64-bit Linux) is 1MB, and you can manipulate that to be smaller if you want. That's also the virtual memory. The actually memory usage depends on your application. There was a time where 1MB was a lot of mem…

Your words might be true, but the world jumped on async wagon long time ago and going all in. Nobody likes threads, everyone wants lightweight threads. Emulating lightweight threads with promises (optionally hidden behind async/await transformations) is very popular. So demand for this feature is here. I don't know why, I, personally, never needed that feature and good old threads were always enough for me. It's weir…

It's madness.

Re: Thread Pools on the JVM

#100

Post by cratermoon »

iamcreasy wrote: post #43
cogman10 wrote: post #21

Earlier quoted context omitted.

Yes and no. The new Loom threads will be much lighter weight than the original Java green threads. Further, the entire IO infrastructure of the JVM is being reworked for Loom to make sure the OS doesn't block the VM's thread. What's more, Loom does M:N threading. Same concept, very different implementation.

So, with Loom now we can tell exactly in which order theses threads were executed as it's not up to OS to decide thread execution order anymore?

> we can tell exactly in which order theses threads were executed

Everything I've ever been taught about multi-threading, parallelism, and concurrency says never to make any assumptions about execution order. What are you doing that your care about it?

Post reply on HN