Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why events are a bad idea for high-concurrency servers (2003) [pdf]

people.eecs.berkeley.edu

Post reply on HN
Print view View original

31–40 of 50 posts

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#31

Post by saman_b »

The results of our to be published paper clearly confirm the claims of this paper and shows that if implemented well, threads can perform and scale as good as events with not much memory overhead.

I worked on this subject during my Phd, and the result is a paper that will be published in sigmetrics2020.

We developed an M:N user-level threading library and exhaustively tested it against event-based alternatives and pthread based solutions.

We used both memcached and webservers to test it on 32 core and 64 core servers.

Even connection/pthread looks promising in terms of performance.

You can find the paper and source files here: https://cs.uwaterloo.ca/~mkarsten/papers/sigmetrics2020.html

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#32

Post by the_unproven »

Events can handle much more connections than a thread based approach. For example nginx is implemented with event-driven architecture: https://www.nginx.com/blog/inside-nginx-how-we-designed-for-...

> NGINX scales very well to support hundreds of thousands of connections per worker process. Each new connection creates another file descriptor and consumes a small amount of additional memory in the worker process. There is very little additional overhead per connection. NGINX processes can remain pinned to CPUs. Context switches are relatively infrequent and occur when there is no work to be done.

> In the blocking, connection‑per‑process approach, each connection requires a large amount of additional resources and overhead, and context switches (swapping from one process to another) are very frequent.

This is their explanation on events vs threading approach. Still, a lot of web servers today use a thread-per-connection which is acceptable since a database(e.g. postgres) performance degrades slowly as more active connections are introduced.[1]

[1] https://brandur.org/postgres-connections

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#34

Post by Roboprog »

nine_k wrote: post #25
afcapel wrote: post #22

Earlier quoted context omitted.

A context switch in a modern CPU takes only a few microseconds. A GB of RAM costs less than $10. So those concerns, although valid in theory, are usually irrelevant for most web applications. On the other hand, simplicity in a code base usually matter. Code written with an evented API, littered of callbacks, is usually harder to read and maintain than that written in a sequential way with a blocking I/O API. You can…

A GB of RAM only costs less than $10 if you are buying for your unpretentious gaming rig. A GB of ECC server RAM costs more. An extra GB of RAM in the cloud can even cost you $10/mo if you have to switch to a beefier instance type.

How much does a MB of L-n cache cost?

I don’t have the answer, but you would want to measure dollars to buy it, and nanoseconds to refill it.

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#35

Post by user5994461 »

afcapel wrote: post #22
cryptonector wrote: post #17

Earlier quoted context omitted.

C10K is another name for evented I/O, and it's from the 90s . By 2003 thread-per-client was already obsolete and known to be very bad. It's really quite simple: threads encourage the use of implicit program state via function calls, with attendant state expansion (stack allocation), whereas evented I/O encourages explicit program state, which means the programmer can make it as small as possible. Smaller server progr…

A context switch in a modern CPU takes only a few microseconds. A GB of RAM costs less than $10. So those concerns, although valid in theory, are usually irrelevant for most web applications. On the other hand, simplicity in a code base usually matter. Code written with an evented API, littered of callbacks, is usually harder to read and maintain than that written in a sequential way with a blocking I/O API. You can…

It's actually a big problem for web servers. If you consider apache for example, that has to do one thread per connection. (yes, apache still doesn't support events for websockets in 2020).

Let's say you configure it for 2000 max connections (really not much) so that's 2000 threads, so 20 GB of memory right away because the thread stack is 10 MB on Linux. It's a lot of memory and it's obliterating all caches.

You can reduce the thread stack to 1 MB (might crash if you go lower) but any caching is still trashed to death.

Next challenge. How do you think concurrency work on the OS with 2000 threads? Short answer is not great.

The software making heavy use of shared memory segments, semaphores, atomics and other synchronization features. That code is genuinely complex, worse than callbacks. Then you're having issues because these primitives are not actually efficient when contended by thousands of threads, they might even be buggy.

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#36

Post by rezonant »

Jweb_Guru wrote: post #23
zzzcpan wrote: post #12

Earlier quoted context omitted.

I don't think there is back and forth anymore. Actual high performance research (e.g. when the cost of a single mutex is more than the whole CPU budget for processing something, like say a packet) has been devoid of threads since they got into the mainstream, so like for almost two decades already. They are still used, because this is what hardware and OS provide to do something on each core, but not for concurrency…

Sure, if you are devoting the whole computer to a single microbenchmark, threads a terrible idea. This is not necessarily the case when you have many heterogeneous applications running on a machine, though.

I'm a little confused by this- if you have multiple independent apps on a machine, would they not already be in separate OS processes?

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#37

Post by zzzcpan »

saman_b wrote: post #31

The results of our to be published paper clearly confirm the claims of this paper and shows that if implemented well, threads can perform and scale as good as events with not much memory overhead. I worked on this subject during my Phd, and the result is a paper that will be published in sigmetrics2020. We developed an M:N user-level threading library and exhaustively tested it against event-based alternatives and pt…

Well, no, you are just hacking it to support your claims. A split-stack work-stealing implementation is ok, nothing special, basically goroutines. But you are not addressing the most important difference between events and shared memory multithreading - synchronizing access to shared memory. Idiomatic event driven applications don't do synchronization and have to be sharded to scale to multiple cores. Choosing memcached and running it multithreaded is particularly bad, as memcached is not a decent event driven applications, it mixes threads and events and suffers from all that synchronization overhead. At least you should run it in one process per core configuration [1]. But it's much worse than that, there is some serious research in this area that addresses those problems, in particular the Anna paper [2], it kills any possibility for shared memory multithreading as a concurrency model to compete with anything, it's just too broken on a fundamental level.

[1] https://github.com/scylladb/seastar/wiki/Memcached-Benchmark

[2] https://dsf.berkeley.edu/jmh/papers/anna_ieee18.pdf

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#38

Post by bullen »

gpderetta wrote: post #11
TheFiend7 wrote: post #7

Earlier quoted context omitted.

Haha I think that's the point of OP's post IMO, and people were even discussing this dichotomy in that thread as well. Total gold, nothing in this world is clean or definitive.

Yes :). I do not actually have a strong opinion on the Event/Threads/Coroutines debate. I think they should all be considered on a case by case basis.

I'll restate the same thing I commented on that post:

https://news.ycombinator.com/item?id=22174201

People only now start to understand that the problem with multi-core is memory speed and the only way to solve that is by using a virtual machine with a concurrent capable memory model = Java.

In-lined memory does not play well with multiple threads. Cache-misses is the way to parallelize code!

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#39

Post by saman_b »

zzzcpan wrote: post #37
saman_b wrote: post #31

The results of our to be published paper clearly confirm the claims of this paper and shows that if implemented well, threads can perform and scale as good as events with not much memory overhead. I worked on this subject during my Phd, and the result is a paper that will be published in sigmetrics2020. We developed an M:N user-level threading library and exhaustively tested it against event-based alternatives and pt…

Well, no, you are just hacking it to support your claims. A split-stack work-stealing implementation is ok, nothing special, basically goroutines. But you are not addressing the most important difference between events and shared memory multithreading - synchronizing access to shared memory. Idiomatic event driven applications don't do synchronization and have to be sharded to scale to multiple cores. Choosing memcac…

I believe one should not confine event-driven only to applications that don't do synchronisation, that's part of the misconception that leads to thinking event-driven has higher performance. This is due to the fact that part of the problem with threads (as you mentioned) is synchronisation, but this is the same problem with event-driven applications. We have a web server experiment that shows comparable performance to an event-driven web server (ulib) with its various hacks to make it faster and is always on the top list on tech-empower.

Regarding memcached, the first reference you posted is from 2015, yes in 2015 memcached was in a very bad shape in terms of synchronisation and things have significantly changed from that version with locks per hash bucket rather than a global lock, avoiding try_lock and .... So those results are too old to rely on. Seastar moves network stack to user-level and if I remember correctly the scheduler consisted of multiple ever looping threads even if there was no work to do. Considering in our experiments 40-60% of the memcached overhead were coming from network I/O, there is no surprise about their results. I would call this a hack for sure.

I have not read the Anna paper to be able to comment, but it seems that they are creating a key-value store using the actor model. I briefly schemed through the paper, and I did not find anything that points to "killing any possibility for shared memory multi-threading as a concurrency model"; this is a very bold claim and if they do claim that, they should have really strong results.

But my guess is anna's whole actor model was implemented on top of threads and shared memory multi-threading? Which will be in contrast of that bold claim. I worked with actor models and implemented one as well, it is a perfect fit for many use cases but threads at least for now are the bread and butter of multicore programming.

Having said that, all these models have their respective place in the software world. What we are trying to show in our paper , through thorough experiments, is that the misconception that event-driven has higher performance than thread programming is not fundamentally true. Therefore, falling into asynchronous programming and create hard to maintain applications [1] only due to better performance has no merit.

[1] https://cacm.acm.org/magazines/2017/4/215032-attack-of-the-k...

Re: Why events are a bad idea for high-concurrency servers (2003) [pdf]

#40

Post by zzzcpan »

saman_b wrote: post #39
zzzcpan wrote: post #37

Earlier quoted context omitted.

Well, no, you are just hacking it to support your claims. A split-stack work-stealing implementation is ok, nothing special, basically goroutines. But you are not addressing the most important difference between events and shared memory multithreading - synchronizing access to shared memory. Idiomatic event driven applications don't do synchronization and have to be sharded to scale to multiple cores. Choosing memcac…

I believe one should not confine event-driven only to applications that don't do synchronisation, that's part of the misconception that leads to thinking event-driven has higher performance. This is due to the fact that part of the problem with threads (as you mentioned) is synchronisation, but this is the same problem with event-driven applications. We have a web server experiment that shows comparable performance t…

> I believe one should not confine event-driven only to applications that don't do synchronisation

This is just silly. If they do synchronization, they are no longer event driven, they are shared memory multithreaded and are bounded by its performance. In such cases event driven model is not actually used for much, only for I/O notification. So of course comparing shared memory multithreaded application with another way of doing shared memory multithreaded application yields comparable performance, you are not comparing them to event driven, which you can't, because they will suck too much in comparison and not support your bullshit claims.

> is that the misconception that event-driven has higher performance than thread programming is not fundamentally true

It is fundamentally true, your paper is just really bad.

Post reply on HN