Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Next Generation Out of Band Garbage Collection

railsatscale.com

Post reply on HN
Print view View original

71–80 of 87 posts

Re: Next Generation Out of Band Garbage Collection

#71

Post by samatman »

ngrilly wrote: post #51
samatman wrote: post #24

Earlier quoted context omitted.

> allocating more than the arena can accommodate In Zig, at least, this isn't how arenas work. They're a wrapper around a backing allocator, so if the arena runs out of memory, then that means the process is out of memory, something no allocation strategy can fix (ignoring the fact that Zig returns a specific error when that happens, and maybe you can trigger some cache eviction or something like that). It's easy to…

I really like Zig, and I’ve been thinking about using it for developing servers offering APIs over HTTP, using arenas bound to request lifetime. I think I would be comfortable developing like this myself. But all the devs in my org have only ever used managed-memory languages such as Java, C#, Python or JavaScript, which makes me hesitant, as I’m wondering about the learning curve, and of course the risk of use-after…

I wrote substantial amounts of C, and Pascal/Delphi before that, before learning Zig, so you and I wouldn't see the same learning curve. That said, I found it straightforward to take up. Andrew Kelly places a great emphasis on simplicity in the sense Rich Hickey uses the term, so Zig has a small collection of complete solutions which compose well.

Now is a great time to pick up the language, but I would say that production is not the right place to do that for a programmer learning memory management for the first time. Right now we're late in the release cycle, so I'd download a nightly rather than use 0.13, if you wanted to try it out. Advent of Code is coming up, so that's an option.

Using a memory-managed language means you need to design a memory policy for the code. Zig's GeneralPurposeAllocator will catch use after free and double free in debug mode, but that can only create confidence in memory handling code if and when you can be sure that there aren't latent bugs waiting to trigger in production.

Arenas help with that a lot, because they reduce N allocations and frees to 1, for any given set of allocations. But one still has to make sure that the lifetime of allocations within the arena doesn't outlast the round, and you can only get that by design in Zig, lifetimes and ownership aren't part of the type system like they are in Rust. In practice, or I should say with practice, this is readily achievable.

At current levels of language maturity, small teams of experienced Zig developers can and do put servers into production with good results. But it's probably not time for larger teams to learn as they go and try the same thing.

Re: Next Generation Out of Band Garbage Collection

#72

Post by WJW »

indulona wrote: post #38
danmur wrote: post #17

Earlier quoted context omitted.

I don't think using Zig over Python is gonna have the biggest impact in making your next big company successful. It's a drop in the ocean compared to the quality of people you have to actually design and build it.

That is nonsense. If you can run your code on 10 servers instead of 1k servers, that is an insane time and money saver that could make or break a company.

For most startups the difference between Rust and Ruby is not 10 servers vs 1000 but rather between using 0.1% of a CPU or 1% of a CPU. A single server running Rails will easily scale to hundreds of thousands of daily users. Most companies never get that many users in the first place, and those that do will have the funds to afford rewriting the hottest paths in a more performant language.

Re: Next Generation Out of Band Garbage Collection

#73

Post by BoppreH »

cletus wrote: post #70

> no object allocated as part of a request should survive longer than the request itself So I've spent a lot of time doing Hack (and PHP) as well as Java, Python and other languages. For me, as far as serving HTTP requests goes, Hack/PHP are almost the perfect language. Why? 1. A stateless functional core. There's no loading of large libraries, which is an issue with Python and Java in certain paradigms. The core API…

If the core API is just functions, how do stateful applications handle connections to this persistent storage? Can you still have a connection pool, or does every request pay the extra latency to start a new connection and re-authenticate?

Re: Next Generation Out of Band Garbage Collection

#74

Post by dinosaurdynasty »

the_mitsuhiko wrote: post #55
hinkley wrote: post #5

> Ideally in a web application, aside from some in-memory caches, no object allocated as part of a request should survive longer than the request itself. This is one of those areas where out of process caching wins. In process caching has a nasty habit of putting freshly created objects into collections that have survived for days or hours, creating writes in the old generation and back references from old to new. Go…

One of the challenges I see in general is that languages don't have enough capabilities to express intent of lifetimes / control flow. What I mean by that is that there is a significant difference between spawning a thread with the intention of joining, allocating memory with the intention of only lasting to the end of the request etc. vs spawning a permanent background thread or stashing away an object into a global…

You could do something like that in Rust with a request-scooped arena. But then you'd have to do Rust.

In node you could use worker threads (which create a new V8 instance in a separate OS thread) but that's probably too heavy handed.

Re: Next Generation Out of Band Garbage Collection

#75

Post by [deleted] »

spullara wrote: post #48
oorza wrote: post #27

Earlier quoted context omitted.

C4 still smokes them both, doesn't it?

hard to smoke sub-millisecond pauses but there may be other axes where it is better. it used to be that people thought azul was better because it was generational but now zgc is as well. my guess is that c4 doesn't have enough of an edge at this point but happy to see benchmarks that prove otherwise.

[deleted]

Re: Next Generation Out of Band Garbage Collection

#76

Post by ngrilly »

samatman wrote: post #71
ngrilly wrote: post #51

Earlier quoted context omitted.

I really like Zig, and I’ve been thinking about using it for developing servers offering APIs over HTTP, using arenas bound to request lifetime. I think I would be comfortable developing like this myself. But all the devs in my org have only ever used managed-memory languages such as Java, C#, Python or JavaScript, which makes me hesitant, as I’m wondering about the learning curve, and of course the risk of use-after…

I wrote substantial amounts of C, and Pascal/Delphi before that, before learning Zig, so you and I wouldn't see the same learning curve. That said, I found it straightforward to take up. Andrew Kelly places a great emphasis on simplicity in the sense Rich Hickey uses the term, so Zig has a small collection of complete solutions which compose well. Now is a great time to pick up the language, but I would say that prod…

I started programming in Pascal, C and C++, so personally I’m fine with manual memory management, especially with a language like Zig. I actually find it quite refreshing. I’m just wondering if it’s possible to “scale” this approach to a team of developers who may not have that past experience (having only worked with GCed languages) without ending in a code base littered with use-after-free errors.

Re: Next Generation Out of Band Garbage Collection

#77

Post by spullara »

BoppreH wrote: post #73
cletus wrote: post #70

> no object allocated as part of a request should survive longer than the request itself So I've spent a lot of time doing Hack (and PHP) as well as Java, Python and other languages. For me, as far as serving HTTP requests goes, Hack/PHP are almost the perfect language. Why? 1. A stateless functional core. There's no loading of large libraries, which is an issue with Python and Java in certain paradigms. The core API…

If the core API is just functions, how do stateful applications handle connections to this persistent storage? Can you still have a connection pool, or does every request pay the extra latency to start a new connection and re-authenticate?

i think they do now but originally one of the reasons people use mysql with languages that are connection per request was that mysql connections were very cheap.

Re: Next Generation Out of Band Garbage Collection

#78

Post by gmueckl »

xerxes901 wrote: post #10
henning wrote: post #6

They built a large codebase on a language that doesn't let you control memory, because that makes you "more productive". So just having Rails allocate a per-request arena that is asynchronously freed which would force the programmer not to have any objects that outlive the request, or just pre-allocating memory for a fixed amount of request handling per server instance, or whatever allocation behavior you want to do…

> Which means your changes have to go through the Ruby team first. Any additional changes would also need to go through them … I do want to pick on this specifically - people can and should be patching open source projects they depend on and deploying them to production (exactly as described in the article). Something being in the language vs in “user” code should be no barrier to improving it.

Patching a dependency comes with significant downstream costs. You need to carry the patch forward to new upstream versions . This implies remembering that the dependency was patched, extracting a patch from the existing changed code, and reapplying the patch, fixing comflicts, recompiling the now special version of that dependency and running tests, checking/updating required license notices accordingly.

This is in essence another form of technical dept.

Re: Next Generation Out of Band Garbage Collection

#79

Post by derefr »

spullara wrote: post #77
BoppreH wrote: post #73

Earlier quoted context omitted.

If the core API is just functions, how do stateful applications handle connections to this persistent storage? Can you still have a connection pool, or does every request pay the extra latency to start a new connection and re-authenticate?

i think they do now but originally one of the reasons people use mysql with languages that are connection per request was that mysql connections were very cheap.

And why pgbouncer used to be considered an essential part of a Postgres web-app-backend deployment — if your business layer didn’t pool and reuse connections, then having an external shim component that pools and reuses connections would solve a lot of the impedance mismatch.

Re: Next Generation Out of Band Garbage Collection

#80

Post by Twirrim »

krapp wrote: post #59
eru wrote: post #57

Earlier quoted context omitted.

Well, Facebook also created their own hacked up version of PHP (called Hack) that's presumably easier to migrate PHP to. Hack is actually surprisingly pleasant, basically about the best language they could have made starting from PHP. (I know, that's damning with faint praise. But I actually mean this unironically. It has TypeScript vibes.)

I was excited about Hack when it came out. Unfortunately PHP took just enough from it to kill it. I gave up on it once Composer stopped supporting it, after backwards compatibility with PHP no longer became a goal. IMHO Hack's best feature was native support for XHP... which (also unfortunately) isn't something PHP decided to take.

Hack really gave PHP/Zend the kick up the arse that it seemed to need.
Post reply on HN