Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pitchfork: Rack HTTP server for shared-nothing architecture

github.com

Post reply on HN
Print view View original

31–40 of 71 posts

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#31

Post by runlevel1 »

jaxrtech wrote: post #15
skrtskrt wrote: post #11

I have now worked at several significantly-sized companies that ripped Ruby out of everything as they scaled, usually replaced with Go, though none as large as Shopify. I wonder what that financial calculus looks like between these options for a large organization: 1. trying to basically reinvent the workings of an entire programming language and migrate your Ruby apps to these new tools/runtimes/typecheckers/whateve…

A lot of the time, I ask myself, "why don't we seem to have a tool with robust source-to-source transformation?" You could call it "cross-language refactoring". I wouldn't be surprised if someone with the code-base the size of Google has some internal tool. Maybe this is just wishful thinking. The closest thing I've seen is in academic research, e.g. [1]. [1]: Koppel, Solar-Lezama - Incremental parametric syntax for…

Idioms and features often don't match up between any two languages. For example, what do you do with monkeypatched code when porting from Ruby to Go?

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#32

Post by dexwiz »

jaxrtech wrote: post #15
skrtskrt wrote: post #11

I have now worked at several significantly-sized companies that ripped Ruby out of everything as they scaled, usually replaced with Go, though none as large as Shopify. I wonder what that financial calculus looks like between these options for a large organization: 1. trying to basically reinvent the workings of an entire programming language and migrate your Ruby apps to these new tools/runtimes/typecheckers/whateve…

A lot of the time, I ask myself, "why don't we seem to have a tool with robust source-to-source transformation?" You could call it "cross-language refactoring". I wouldn't be surprised if someone with the code-base the size of Google has some internal tool. Maybe this is just wishful thinking. The closest thing I've seen is in academic research, e.g. [1]. [1]: Koppel, Solar-Lezama - Incremental parametric syntax for…

The cost of this challenge is likely bigger than the market. How many large code bases need to be translated from X to Y? Probably not very many. There are only so many legacy COBOL systems.

Big issues off the top of my head.

Legacy support. Legacy code is already difficult to understand. If it was originally written in another language that would only make it harder to grok.

Transpiled output is often ugly. Developers in general seem to dislike modifying generated code for a number of reasons. I imagine transpiled code would end up implicitly frozen, and new code would only be added to new files.

Language idioms are hard to translate. Some structures are unique to languages and their equivalents may be considered bad in other languages.

You would either need equivalent libraries in new languages or also translate dependencies. You could probably find equivalent libraries, but their scope may vary.

Large scale code organization varies between languages. Things like Dependency Injection may not translate at all. Some languages may use config and some use code for the same thing.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#33

Post by a-dub »

pedrocr wrote: post #28
a-dub wrote: post #22

Earlier quoted context omitted.

many serious applications back then built by developers who cared about performance and latency made use of mod_perl. it put the entire perl interpreter (which then loaded perl modules that defined the application) into the preforked apache httpd processes. often times you'd put a proxy in front, serving static assets with a lighter weight http so that slow loads wouldn't tax limited server capacity for big httpd pro…

It's very similar to that setup. I don't think that was particularly common though and probably the reason prefork went away. Particularly as apache and later nginx were used as the user facing webservers proxying to something else inside (first with fastcgi and latter just http). The preforking advantage then moved to that server inside which is what this is. Nginx or something similar will still be running in front…

it went away because it was resource expensive. even with copy-on-write semantics, those backend httpds with an entire perl runtime in each core were pretty huge and inefficient.

the models that replaced it were single process multiple threads and asynchronous (never block) models that didn't create or require one of many application processes per request.

i think the choice of the preforking model was a stability and portability thing. if one of the children crashed it was no big deal, where a crash in a single process application or webserver will cause the everything to go down. also back in those days threading semantics varied widely between operating systems, the apis themselves differed and linuxthreads were just really too new to run in production.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#34

Post by schappim »

tiffanyh wrote: post #4

I’m still surprised that we haven’t seen much larger improvements in Ruby performance over the last decade given the large number of major tech companies using Rails. Yes, I'm aware of 3x3 but for the most common usage of Ruby which if for Rails apps - there hasn't been anything like the gains PHP saw from 5.x to 7.x. Especially from Microsoft, who has deep compiler/language expertise, and who owns GitHub (large Rail…

>> Yes, I'm aware of 3x3 but for the most common usage of Ruby which if for Rails apps - there hasn't been anything like the gains PHP saw from 5.x to 7.x.

This has simply not been my experience. The jump in performance in the 3.n has been awesome, to the point were I even downgraded / removed servers.

Ironically the greatest speed (and memory) improvements have been in the swapping out the supporting JS ecosystem components that Rails uses, for example swapping Webpack for the Go based Esbuild.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#35

Post by pedrocr »

a-dub wrote: post #33
pedrocr wrote: post #28

Earlier quoted context omitted.

It's very similar to that setup. I don't think that was particularly common though and probably the reason prefork went away. Particularly as apache and later nginx were used as the user facing webservers proxying to something else inside (first with fastcgi and latter just http). The preforking advantage then moved to that server inside which is what this is. Nginx or something similar will still be running in front…

it went away because it was resource expensive. even with copy-on-write semantics, those backend httpds with an entire perl runtime in each core were pretty huge and inefficient. the models that replaced it were single process multiple threads and asynchronous (never block) models that didn't create or require one of many application processes per request. i think the choice of the preforking model was a stability an…

But it didn't go away. Event based servers were developed to handle slow clients with good performance as frontends for pre-fork or threaded application servers. This innovation is an optimization of a pre-fork backend server that itself is almost 15 years old.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#36

Post by wgjordan »

I'm the author of the original 'refork' feature introduced to Puma a couple years ago [1], that served as an inspiration for this project.

As I understand it, Pitchfork is an implementation of a reforking feature on top of Unicorn, a single-threaded multi-process Rack HTTP server.

The general idea of 'reforking' is that in addition to pre-forking worker processes at initialization time, you re-fork them on an ongoing basis, in order to optimize copy-on-write efficiency across processes, especially for memory allocations that might happen after initialization. The Pitchfork author mentions sharing YJIT machine code as a particularly important use-case [2], one that has become relevant in recent Ruby releases.

I never saw much interest in the concept when I originally introduced the reforking feature to Puma, and it remained a kind of interesting experiment that never saw much production uptake. I'm thrilled that Shopify is iterating on the concept through this project and I hope the technique will continue to be refined and developed, and see broader adoption as a result.

[1] https://github.com/puma/puma/pull/2099 ; https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork...

[2] https://github.com/Shopify/pitchfork/pull/1#issuecomment-125...

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#37

Post by wgjordan »

jbverschoor wrote: post #2

How does it compare to puma?

Puma is a multi-thread, multi-process Rack HTTP server that implements a request buffer.

By comparison, Unicorn (the project upon which Pitchfork is based) is a single-thread, multi-process Rack HTTP server that does not buffer requests, so it's only designed for fast clients (or it needs to be paired with a proxy like Nginx to handle slow clients).

The narrower design of Unicorn results in a simpler, less-flexible potentially more efficient architecture.

The reforking feature that Pitchfork introduces to Unicorn was originally implemented in Puma [1], though they're controlled differently and the underlying implementations are entirely different.

[1] https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork...

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#38

Post by joshmn »

compumike wrote: post #19

For those of you scaling Ruby on Rails: are you more constrained by memory or CPU consumption? If you could choose a 50% reduction in process memory footprint, versus a 50% reduction in CPU cycles to serve your average request, which would you pick?

Database locks is our issue. We're doing anywhere from 5-10k req/s to our RDBMS. This could be minimized if not completely avoided, and I'm keen to make it so, but I got here a bit late and the underlying foundation of our application isn't something I can safely describe.

Most of our work happens in the background. We're running a Sidekiq process per CPU core with a concurrency of 20 — which, while the default, seemed to play the best during our tests. We set a max RSS of 2000 and use some code to automatically scale Heroku based on queue size. We do a lot of network IO.

It was a headache to get here, and a hole in our wallet, but I'm mostly happy with the application. Previously, we were running Resque in production on a super over-provisioned AWS node that was costing us $xx,xxx a month: Resque forks the app process for concurrency (we weigh about 450 MB); Sidekiq is a threaded model. Heroku, at least in this case, has been exponentially cheaper. We keep our RDS on AWS for at least some savings versus having it on Heroku. I cannot imagine what a xx-TB database would cost us at Heroku.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#39

Post by ksec »

skrtskrt wrote: post #11

I have now worked at several significantly-sized companies that ripped Ruby out of everything as they scaled, usually replaced with Go, though none as large as Shopify. I wonder what that financial calculus looks like between these options for a large organization: 1. trying to basically reinvent the workings of an entire programming language and migrate your Ruby apps to these new tools/runtimes/typecheckers/whateve…

> significantly-sized companies that ripped Ruby out of everything as they scaled

I am assuming Ruby, used here meant Ruby Rails. Or specifically, ripped Rails out of everything as they scaled?

The calculus, in many situation would indeed be in flavour of another framework or language in ~2013. The cost of CPU Core has dropped significantly since then, we are not far from having 128 x86 CPU core in a single socket. Even if could only do 10 request per second per core at a target latency, this is 1280 Request per second on a single server. ( In the context of a non API serving App, or Server rendering App )

Edit: Miscalculation here, it should be 128 Core, 256 vCPU, so 2560 RPS.... but the main point still stand.

But at the Scale of Shopify, ~$33B Market Cap ( ouch.....I remember they passed $100B market cap in 2020 and $200B market cap in 2021... ) They could finally afford to pour some resources into Ruby. The only Top 10 programming languages without large cooperate resource backing. I am pretty sure the performance potential of YJIT and Ruby tooling has a ROI of less than 2 years at their Scale.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#40

Post by nateberkopec »

skrtskrt wrote: post #11

I have now worked at several significantly-sized companies that ripped Ruby out of everything as they scaled, usually replaced with Go, though none as large as Shopify. I wonder what that financial calculus looks like between these options for a large organization: 1. trying to basically reinvent the workings of an entire programming language and migrate your Ruby apps to these new tools/runtimes/typecheckers/whateve…

The fully loaded cost of even three or four software engineers in the USA outstrips 99% of companies cloud computing budgets.
Post reply on HN