Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Next Generation Out of Band Garbage Collection

railsatscale.com

Post reply on HN
Print view View original

21–30 of 87 posts

Re: Next Generation Out of Band Garbage Collection

#21

Post by hinkley »

byroot wrote: post #9
hinkley wrote: post #5

> Ideally in a web application, aside from some in-memory caches, no object allocated as part of a request should survive longer than the request itself. This is one of those areas where out of process caching wins. In process caching has a nasty habit of putting freshly created objects into collections that have survived for days or hours, creating writes in the old generation and back references from old to new. Go…

Author here. Agreed. We have some facility for out of process caching (node local memcached), and I frequently have to argue with colleagues that it's generally preferable to in-process caching.

I accept that it is true but I bristle at the fact of it. It shouldn’t be true.

Re: Next Generation Out of Band Garbage Collection

#23

Post by echelon »

hinkley wrote: post #5

> Ideally in a web application, aside from some in-memory caches, no object allocated as part of a request should survive longer than the request itself. This is one of those areas where out of process caching wins. In process caching has a nasty habit of putting freshly created objects into collections that have survived for days or hours, creating writes in the old generation and back references from old to new. Go…

Rust is emerging as a major contender for HTTP/gRPC backend services.

Actix, Axum, sqlx, diesel, and a whole host of other utilities and frameworks make writing Rust for HTTP just as easy and developer efficient as Golang or Java, but the code will never have to deal with GC.

It's easy to pull request scoped objects into durable caches.

Re: Next Generation Out of Band Garbage Collection

#24

Post by samatman »

byroot wrote: post #14
jrockway wrote: post #8

Earlier quoted context omitted.

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

I don't see how it is imperfect. Per request arenas sound super cool on paper, and work very well on system with clear constraints. But if suddenly a request start allocating more than the arena can accommodate you're in a bit of a pickle. They're absolutely not a panacea. Setting aside the challenge of refactoring the Ruby VM to allow this sort of arenas, they'd be a terrible fit for Shopify's monolith. Ultimately,…

> allocating more than the arena can accommodate

In Zig, at least, this isn't how arenas work. They're a wrapper around a backing allocator, so if the arena runs out of memory, then that means the process is out of memory, something no allocation strategy can fix (ignoring the fact that Zig returns a specific error when that happens, and maybe you can trigger some cache eviction or something like that).

It's easy to set them to retain a 'reasonable' allocated capacity when they get reset, for whatever value of reasonable, so big allocation spikes get actually freed, but normal use just moves a pointer back and reuses that memory.

I don't see Shopify harvesting a lot of value from a complete Zig rewrite, no. But arenas are basically ideal for the sort of memory use which web servers typically exhibit.

Re: Next Generation Out of Band Garbage Collection

#25

Post by foobazgt »

byroot wrote: post #9
hinkley wrote: post #5

> Ideally in a web application, aside from some in-memory caches, no object allocated as part of a request should survive longer than the request itself. This is one of those areas where out of process caching wins. In process caching has a nasty habit of putting freshly created objects into collections that have survived for days or hours, creating writes in the old generation and back references from old to new. Go…

Author here. Agreed. We have some facility for out of process caching (node local memcached), and I frequently have to argue with colleagues that it's generally preferable to in-process caching.

A similar strategy is to serialize the object and store it in-process but off heap. This is useful when the values are private to the process, and/or they don't need to survive a crash, and/or you need to avoid the network overhead. Access times are often 100x-1000x faster.

Re: Next Generation Out of Band Garbage Collection

#26

Post by foobazgt »

rty32 wrote: post #18
IshKebab wrote: post #7

Earlier quoted context omitted.

A 10% performance improvement on Python code is laughable. You can get a 5000% performance improvement if you switch to a better language.

Very naive take. 1. 10% performance improvement at Instagram could lead to many millions of revenue "instantly". It is not laughable at any company. 2. It won't be a 5000% performance improvement. Facebook uses its own fork of Python that is heavily optimized. Probably still far from C++, but you should be thinking about languages like Java when talking about performance. "Better" is a very subjective term when discu…

Cinder's benchmarks don't seem "like Java" performance, given they aren't that far off cython.

https://github.com/facebookincubator/cinder/blob/cinder/3.8/...

Re: Next Generation Out of Band Garbage Collection

#29

Post by bob1029 »

spullara wrote: post #20

All the other virtual machines that support GC need to look at the JVM's ZGC and Shenandoah. Sub-millisecond pause times with terabyte heaps.

I think we should be careful when correlating heap size with how long the collection should take.

Also, I really want ZGC in .NET runtime, but I don't think I'll ever get support for it first party. There's some kind of principled ideologue holdout situation going on over at Microsoft. Every time I get into it with one of their engineers I'm sent to some impotent "please may I have a temporary GC exemption" API. All I want is it to do nothing. How hard is it to just not clean up the goddamn garbage? Give me a registry flag + env variable + cli arg all required at the same time if you're so worried someone might trip over it.

Re: Next Generation Out of Band Garbage Collection

#30

Post by spullara »

algorithmsRcool wrote: post #28
spullara wrote: post #20

All the other virtual machines that support GC need to look at the JVM's ZGC and Shenandoah. Sub-millisecond pause times with terabyte heaps.

How much of a throughput penalty do those options incur on the application?

I think it only affects throughput at the limit at the 5% level. All the portfolio companies that implemented it got a net increase in performance as they avoid redlining their servers.
Post reply on HN