Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Next Generation Out of Band Garbage Collection

railsatscale.com

Post reply on HN
Print view View original

51–60 of 87 posts

Re: Next Generation Out of Band Garbage Collection

#51

Post by ngrilly »

samatman wrote: post #24
byroot wrote: post #14

Earlier quoted context omitted.

I don't see how it is imperfect. Per request arenas sound super cool on paper, and work very well on system with clear constraints. But if suddenly a request start allocating more than the arena can accommodate you're in a bit of a pickle. They're absolutely not a panacea. Setting aside the challenge of refactoring the Ruby VM to allow this sort of arenas, they'd be a terrible fit for Shopify's monolith. Ultimately,…

> allocating more than the arena can accommodate In Zig, at least, this isn't how arenas work. They're a wrapper around a backing allocator, so if the arena runs out of memory, then that means the process is out of memory, something no allocation strategy can fix (ignoring the fact that Zig returns a specific error when that happens, and maybe you can trigger some cache eviction or something like that). It's easy to…

I really like Zig, and I’ve been thinking about using it for developing servers offering APIs over HTTP, using arenas bound to request lifetime. I think I would be comfortable developing like this myself. But all the devs in my org have only ever used managed-memory languages such as Java, C#, Python or JavaScript, which makes me hesitant, as I’m wondering about the learning curve, and of course the risk of use-after-free. Not something I would do anyway before Zig reaches 1.0.

Re: Next Generation Out of Band Garbage Collection

#52

Post by lucianbr »

byroot wrote: post #45
immibis wrote: post #43

Earlier quoted context omitted.

The joke is that Facebook literally did, right?

Did what? Rewrite Instagram into another language? Do you have any source on this? Last time I checked they're working on improving Python performance instead (yes I know they forked it into Cinder, but they're trying to upstream their optimizations [0]). Which is very similar to what we're doing at Shopify. Of course 100% of Instagram isn't in Python, I'm certain there's lots of supporting services in C++ etc, but A…

> The joke is that if Meta thought that replacing all the Python code they have with something else was worth it, they'd have done it already.

"Worth it" depends on both how much performance improvement you get, and how hard it is to replace. Did you consider maybe the rewriting effort is so humongous that it is not worth doing despite large performance improvements? Thus making the joke not funny at all...

Re: Next Generation Out of Band Garbage Collection

#53

Post by debugnik »

byroot wrote: post #45
immibis wrote: post #43

Earlier quoted context omitted.

The joke is that Facebook literally did, right?

Did what? Rewrite Instagram into another language? Do you have any source on this? Last time I checked they're working on improving Python performance instead (yes I know they forked it into Cinder, but they're trying to upstream their optimizations [0]). Which is very similar to what we're doing at Shopify. Of course 100% of Instagram isn't in Python, I'm certain there's lots of supporting services in C++ etc, but A…

I think they meant Facebook switched their PHP code to Hack and HHVM, their own PHP-like language and implementation.

Re: Next Generation Out of Band Garbage Collection

#54

Post by lucianbr »

rty32 wrote: post #18
IshKebab wrote: post #7

Earlier quoted context omitted.

A 10% performance improvement on Python code is laughable. You can get a 5000% performance improvement if you switch to a better language.

Very naive take. 1. 10% performance improvement at Instagram could lead to many millions of revenue "instantly". It is not laughable at any company. 2. It won't be a 5000% performance improvement. Facebook uses its own fork of Python that is heavily optimized. Probably still far from C++, but you should be thinking about languages like Java when talking about performance. "Better" is a very subjective term when discu…

> Facebook uses its own fork of Python that is heavily optimized.

So likely the 5000% improvement is no longer possible because they already did multiple 10% improvements? I don't know how this counters the original point.

All clues point to FB going this route because they had too much code already in PHP, and not because the performance improvement would be small.

In any case, "facebook does it" is not a good argument that something is the right thing to do. Might be, might not be. FB isn't above wrong decisions. Else we should buy "real estate" in the metaverse.

Re: Next Generation Out of Band Garbage Collection

#55

Post by the_mitsuhiko »

hinkley wrote: post #5

> Ideally in a web application, aside from some in-memory caches, no object allocated as part of a request should survive longer than the request itself. This is one of those areas where out of process caching wins. In process caching has a nasty habit of putting freshly created objects into collections that have survived for days or hours, creating writes in the old generation and back references from old to new. Go…

One of the challenges I see in general is that languages don't have enough capabilities to express intent of lifetimes / control flow. What I mean by that is that there is a significant difference between spawning a thread with the intention of joining, allocating memory with the intention of only lasting to the end of the request etc. vs spawning a permanent background thread or stashing away an object into a global cache.

This is starting to really become a problem in the observability space and async locals. Node.js for instance currently will keep async locals around for too long of a time because they are propagated everywhere. For instance if you call `console.log` in a promise you will leak an async local forever.

Next.js famously keeps around way too many async locals past the request boundary for caching related reasons.

A solution would be to have a trampoline to call things through that make it explicit that everything happening past that point is supposed to "detach" from the current flow. An allocator or a context local system can then use that information to change behavior.

Re: Next Generation Out of Band Garbage Collection

#56

Post by byroot »

lucianbr wrote: post #52
byroot wrote: post #45

Earlier quoted context omitted.

Did what? Rewrite Instagram into another language? Do you have any source on this? Last time I checked they're working on improving Python performance instead (yes I know they forked it into Cinder, but they're trying to upstream their optimizations [0]). Which is very similar to what we're doing at Shopify. Of course 100% of Instagram isn't in Python, I'm certain there's lots of supporting services in C++ etc, but A…

> The joke is that if Meta thought that replacing all the Python code they have with something else was worth it, they'd have done it already. "Worth it" depends on both how much performance improvement you get, and how hard it is to replace. Did you consider maybe the rewriting effort is so humongous that it is not worth doing despite large performance improvements? Thus making the joke not funny at all...

That's exactly the joke though. Every time Ruby (or Python) is discussed on HN we get the same old tired question of "why don't they just rewrite in Rust".

But that's some silly engineer tunnel vision, squeezing the very last bit of performance out of a system isn't a goal in itself. You just need it to be efficient enough that it cost you significantly less to run that the amount of revenue it brings you.

I can bet you that moving off Python must have been pitched dozens and dozens of time by Meta engineers, but deemed not worth it, because execution speed isn't the only important characteristic.

So yes, I find it hilarious when HN commenters suggests companies should rewrite all their software into whatever is seen as the most performant one.

Re: Next Generation Out of Band Garbage Collection

#57

Post by eru »

IshKebab wrote: post #50
byroot wrote: post #15

Earlier quoted context omitted.

Meta is just a small startup though, they probably don't have enough resources nor the skills to switch to a better language even after they've heard the gospel.

I don't know if you're joking or not but that is exactly true. Meta went as far as creating their own PHP engine and then a new PHO compatible language because they didn't have the resources to switch from PHP. Instagram is presumably in the same position. Switching language is basically impossible once you have a certain amount of code. I'm sure they were aware of the performance issues with Python but they probably…

Well, Facebook also created their own hacked up version of PHP (called Hack) that's presumably easier to migrate PHP to.

Hack is actually surprisingly pleasant, basically about the best language they could have made starting from PHP. (I know, that's damning with faint praise. But I actually mean this unironically. It has TypeScript vibes.)

Re: Next Generation Out of Band Garbage Collection

#58

Post by eru »

foobazgt wrote: post #26
rty32 wrote: post #18

Earlier quoted context omitted.

Very naive take. 1. 10% performance improvement at Instagram could lead to many millions of revenue "instantly". It is not laughable at any company. 2. It won't be a 5000% performance improvement. Facebook uses its own fork of Python that is heavily optimized. Probably still far from C++, but you should be thinking about languages like Java when talking about performance. "Better" is a very subjective term when discu…

Cinder's benchmarks don't seem "like Java" performance, given they aren't that far off cython. https://github.com/facebookincubator/cinder/blob/cinder/3.8/...

CPython itself has seen lots of performance improvements recently. Benchmarks on CPython 3.12 take about half the time they took on CPython 3.9.

Re: Next Generation Out of Band Garbage Collection

#59

Post by krapp »

eru wrote: post #57
IshKebab wrote: post #50

Earlier quoted context omitted.

I don't know if you're joking or not but that is exactly true. Meta went as far as creating their own PHP engine and then a new PHO compatible language because they didn't have the resources to switch from PHP. Instagram is presumably in the same position. Switching language is basically impossible once you have a certain amount of code. I'm sure they were aware of the performance issues with Python but they probably…

Well, Facebook also created their own hacked up version of PHP (called Hack) that's presumably easier to migrate PHP to. Hack is actually surprisingly pleasant, basically about the best language they could have made starting from PHP. (I know, that's damning with faint praise. But I actually mean this unironically. It has TypeScript vibes.)

I was excited about Hack when it came out. Unfortunately PHP took just enough from it to kill it. I gave up on it once Composer stopped supporting it, after backwards compatibility with PHP no longer became a goal.

IMHO Hack's best feature was native support for XHP... which (also unfortunately) isn't something PHP decided to take.

Re: Next Generation Out of Band Garbage Collection

#60

Post by eru »

krapp wrote: post #59
eru wrote: post #57

Earlier quoted context omitted.

Well, Facebook also created their own hacked up version of PHP (called Hack) that's presumably easier to migrate PHP to. Hack is actually surprisingly pleasant, basically about the best language they could have made starting from PHP. (I know, that's damning with faint praise. But I actually mean this unironically. It has TypeScript vibes.)

I was excited about Hack when it came out. Unfortunately PHP took just enough from it to kill it. I gave up on it once Composer stopped supporting it, after backwards compatibility with PHP no longer became a goal. IMHO Hack's best feature was native support for XHP... which (also unfortunately) isn't something PHP decided to take.

I only used Hack when I was very briefly working for Facebook. (And I used PHP once before nearly 20 years ago by now for some web site I had 'inherited', back when PHP was truly an awful language)
Post reply on HN