Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Next Generation Out of Band Garbage Collection

railsatscale.com

Post reply on HN
Print view View original

11–20 of 87 posts

Re: Next Generation Out of Band Garbage Collection

#11

Post by bhaak »

jrockway wrote: post #8
henning wrote: post #6

They built a large codebase on a language that doesn't let you control memory, because that makes you "more productive". So just having Rails allocate a per-request arena that is asynchronously freed which would force the programmer not to have any objects that outlive the request, or just pre-allocating memory for a fixed amount of request handling per server instance, or whatever allocation behavior you want to do…

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

Startups are not large companies in the beginning.

Although I'm not sure what the preferred language for quickly getting a startup up and running would be these days.

Re: Next Generation Out of Band Garbage Collection

#12

Post by floating-io »

bhaak wrote: post #11
jrockway wrote: post #8

Earlier quoted context omitted.

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

Startups are not large companies in the beginning. Although I'm not sure what the preferred language for quickly getting a startup up and running would be these days.

Whatever works and you can find enough developers for.

The language (or the rest of the stack even) is rarely a barrier to success. What matters are a good idea, good motivation, and decent availability of competence.

JMHO.

Re: Next Generation Out of Band Garbage Collection

#13

Post by ziddoap »

IshKebab wrote: post #7
jeeyoungk wrote: post #2

This is from several years ago (2017), but this has very similar vibe as Instagram disabling Python GC - https://instagram-engineering.com/dismissing-python-garbage-...

A 10% performance improvement on Python code is laughable. You can get a 5000% performance improvement if you switch to a better language.

I bet the engineers at Instagram were unaware of pythons performance profile when they chose it, you should let them know that they should just switch to a different language.

Re: Next Generation Out of Band Garbage Collection

#14

Post by byroot »

jrockway wrote: post #8
henning wrote: post #6

They built a large codebase on a language that doesn't let you control memory, because that makes you "more productive". So just having Rails allocate a per-request arena that is asynchronously freed which would force the programmer not to have any objects that outlive the request, or just pre-allocating memory for a fixed amount of request handling per server instance, or whatever allocation behavior you want to do…

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

I don't see how it is imperfect.

Per request arenas sound super cool on paper, and work very well on system with clear constraints. But if suddenly a request start allocating more than the arena can accommodate you're in a bit of a pickle. They're absolutely not a panacea.

Setting aside the challenge of refactoring the Ruby VM to allow this sort of arenas, they'd be a terrible fit for Shopify's monolith.

Ultimately, while it's a bit counter intuitive, GCs can perform extremely well in term of throughput. Ruby's GC isn't quite there yet, but still perform quite well and is improving every versions.

Re: Next Generation Out of Band Garbage Collection

#15

Post by byroot »

ziddoap wrote: post #13
IshKebab wrote: post #7

Earlier quoted context omitted.

A 10% performance improvement on Python code is laughable. You can get a 5000% performance improvement if you switch to a better language.

I bet the engineers at Instagram were unaware of pythons performance profile when they chose it, you should let them know that they should just switch to a different language.

Meta is just a small startup though, they probably don't have enough resources nor the skills to switch to a better language even after they've heard the gospel.

Re: Next Generation Out of Band Garbage Collection

#16

Post by electricwire »

ziddoap wrote: post #13
IshKebab wrote: post #7

Earlier quoted context omitted.

A 10% performance improvement on Python code is laughable. You can get a 5000% performance improvement if you switch to a better language.

I bet the engineers at Instagram were unaware of pythons performance profile when they chose it, you should let them know that they should just switch to a different language.

They may well have been initially, its a pretty puzzling choice

Re: Next Generation Out of Band Garbage Collection

#17

Post by danmur »

jrockway wrote: post #8
henning wrote: post #6

They built a large codebase on a language that doesn't let you control memory, because that makes you "more productive". So just having Rails allocate a per-request arena that is asynchronously freed which would force the programmer not to have any objects that outlive the request, or just pre-allocating memory for a fixed amount of request handling per server instance, or whatever allocation behavior you want to do…

I'm guessing that Zig, Rust, Oden, and "etc." didn't exist when they started the codebase. Now they need to keep moving in their imperfect state. I don't think anyone would start a large company on Ruby today. (They would on Python, though, which is equally unfortunate.)

I don't think using Zig over Python is gonna have the biggest impact in making your next big company successful. It's a drop in the ocean compared to the quality of people you have to actually design and build it.

Re: Next Generation Out of Band Garbage Collection

#18

Post by rty32 »

IshKebab wrote: post #7
jeeyoungk wrote: post #2

This is from several years ago (2017), but this has very similar vibe as Instagram disabling Python GC - https://instagram-engineering.com/dismissing-python-garbage-...

A 10% performance improvement on Python code is laughable. You can get a 5000% performance improvement if you switch to a better language.

Very naive take.

1. 10% performance improvement at Instagram could lead to many millions of revenue "instantly". It is not laughable at any company. 2. It won't be a 5000% performance improvement. Facebook uses its own fork of Python that is heavily optimized. Probably still far from C++, but you should be thinking about languages like Java when talking about performance.

"Better" is a very subjective term when discussing languages, and I hope such discussions can be more productive and meaningful.

Post reply on HN