Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Next Generation Out of Band Garbage Collection

railsatscale.com

Post reply on HN
Print view View original

31–40 of 87 posts

Re: Next Generation Out of Band Garbage Collection

#31

Post by reubenbond »

bob1029 wrote: post #29
spullara wrote: post #20

All the other virtual machines that support GC need to look at the JVM's ZGC and Shenandoah. Sub-millisecond pause times with terabyte heaps.

I think we should be careful when correlating heap size with how long the collection should take. Also, I really want ZGC in .NET runtime, but I don't think I'll ever get support for it first party. There's some kind of principled ideologue holdout situation going on over at Microsoft. Every time I get into it with one of their engineers I'm sent to some impotent "please may I have a temporary GC exemption" API. All…

Would GC.TryStartNoGCRegion work for you? https://learn.microsoft.com/en-us/dotnet/api/system.gc.tryst...

Re: Next Generation Out of Band Garbage Collection

#32

Post by worik »

ziddoap wrote: post #13
IshKebab wrote: post #7

Earlier quoted context omitted.

A 10% performance improvement on Python code is laughable. You can get a 5000% performance improvement if you switch to a better language.

I bet the engineers at Instagram were unaware of pythons performance profile when they chose it, you should let them know that they should just switch to a different language.

> I bet the engineers at Instagram were unaware of pythons performance profile when they chose it,

Is the Instagram stack Python? I doubt it, but stranger things have happened

I suspect it is actually some derivative of Apache, or Nginx. Something sensible

Re: Next Generation Out of Band Garbage Collection

#33

Post by tsimionescu »

hinkley wrote: post #5

> Ideally in a web application, aside from some in-memory caches, no object allocated as part of a request should survive longer than the request itself. This is one of those areas where out of process caching wins. In process caching has a nasty habit of putting freshly created objects into collections that have survived for days or hours, creating writes in the old generation and back references from old to new. Go…

This could also be a case for non-managed objects in the same process. APIs aren't typically very friendly, but I would expect they could be made so, especially if it was for a dedicated use-case like caching.

Re: Next Generation Out of Band Garbage Collection

#34

Post by simonask »

xerxes901 wrote: post #10
henning wrote: post #6

They built a large codebase on a language that doesn't let you control memory, because that makes you "more productive". So just having Rails allocate a per-request arena that is asynchronously freed which would force the programmer not to have any objects that outlive the request, or just pre-allocating memory for a fixed amount of request handling per server instance, or whatever allocation behavior you want to do…

> Which means your changes have to go through the Ruby team first. Any additional changes would also need to go through them … I do want to pick on this specifically - people can and should be patching open source projects they depend on and deploying them to production (exactly as described in the article). Something being in the language vs in “user” code should be no barrier to improving it.

There's a pretty huge difference between implementing a performance optimization that works in your use case, and upstreaming that optimization to be generally usable.

The latter is often orders of magnitude more work, and the existing solution is probably chosen to be well suited in general.

Re: Next Generation Out of Band Garbage Collection

#35

Post by simonask »

samatman wrote: post #24
byroot wrote: post #14

Earlier quoted context omitted.

I don't see how it is imperfect. Per request arenas sound super cool on paper, and work very well on system with clear constraints. But if suddenly a request start allocating more than the arena can accommodate you're in a bit of a pickle. They're absolutely not a panacea. Setting aside the challenge of refactoring the Ruby VM to allow this sort of arenas, they'd be a terrible fit for Shopify's monolith. Ultimately,…

> allocating more than the arena can accommodate In Zig, at least, this isn't how arenas work. They're a wrapper around a backing allocator, so if the arena runs out of memory, then that means the process is out of memory, something no allocation strategy can fix (ignoring the fact that Zig returns a specific error when that happens, and maybe you can trigger some cache eviction or something like that). It's easy to…

And when the default arena size is often outgrown, you'll known from whatever diagnostics/logging/dashboard solution you are using. Which is incidentally also a great tool when optimizing per-request memory usage.

Being explicit about memory has many advantages, and is a strict requirement when scaling.

Re: Next Generation Out of Band Garbage Collection

#36

Post by simonask »

danmur wrote: post #17
jrockway wrote: post #8

Earlier quoted context omitted.

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

I don't think using Zig over Python is gonna have the biggest impact in making your next big company successful. It's a drop in the ocean compared to the quality of people you have to actually design and build it.

It doesn't matter for a startup trying to get acquired.

It does matter for a company trying to scale its user base while keeping costs down.

Re: Next Generation Out of Band Garbage Collection

#37

Post by httgp »

worik wrote: post #32
ziddoap wrote: post #13

Earlier quoted context omitted.

I bet the engineers at Instagram were unaware of pythons performance profile when they chose it, you should let them know that they should just switch to a different language.

> I bet the engineers at Instagram were unaware of pythons performance profile when they chose it, Is the Instagram stack Python? I doubt it, but stranger things have happened I suspect it is actually some derivative of Apache, or Nginx. Something sensible

Instagram is built with Django.

Re: Next Generation Out of Band Garbage Collection

#38

Post by indulona »

danmur wrote: post #17
jrockway wrote: post #8

Earlier quoted context omitted.

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

I don't think using Zig over Python is gonna have the biggest impact in making your next big company successful. It's a drop in the ocean compared to the quality of people you have to actually design and build it.

That is nonsense. If you can run your code on 10 servers instead of 1k servers, that is an insane time and money saver that could make or break a company.

Re: Next Generation Out of Band Garbage Collection

#39

Post by byroot »

hinkley wrote: post #21
byroot wrote: post #9

Earlier quoted context omitted.

Author here. Agreed. We have some facility for out of process caching (node local memcached), and I frequently have to argue with colleagues that it's generally preferable to in-process caching.

I accept that it is true but I bristle at the fact of it. It shouldn’t be true.

Depends, it's not just about access time and GC pressure, it's also about sharing that cache with other processes on the node.

Re: Next Generation Out of Band Garbage Collection

#40

Post by byroot »

samatman wrote: post #24
byroot wrote: post #14

Earlier quoted context omitted.

I don't see how it is imperfect. Per request arenas sound super cool on paper, and work very well on system with clear constraints. But if suddenly a request start allocating more than the arena can accommodate you're in a bit of a pickle. They're absolutely not a panacea. Setting aside the challenge of refactoring the Ruby VM to allow this sort of arenas, they'd be a terrible fit for Shopify's monolith. Ultimately,…

> allocating more than the arena can accommodate In Zig, at least, this isn't how arenas work. They're a wrapper around a backing allocator, so if the arena runs out of memory, then that means the process is out of memory, something no allocation strategy can fix (ignoring the fact that Zig returns a specific error when that happens, and maybe you can trigger some cache eviction or something like that). It's easy to…

> something no allocation strategy can fix

Well, yes, with a GC when your heap is full, you make space by getting rid of the garbage.

Also, with a good GC, allocating is most of the time just bumping a pointer, exactly like an arena, and the collection time is proportional to the number of live objects, which when triggered out of band is basically 0.

Hence why I think a well tuned GC really isn't that far off.

Post reply on HN