Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pitchfork: Rack HTTP server for shared-nothing architecture

github.com

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#61

Post by byroot »

ksec wrote: post #56
milofeynman wrote: post #44

Earlier quoted context omitted.

You are making multiple comments talking about Microsoft and Shopify not putting resources into Ruby/rails but both companies have core contributors on them.

Excuse me? Where did I said Shopify not putting resources when the last part of my comment was exactly about Shopify putting resources into Ruby Rails. Microsoft have contribution into Ruby? Or Do you meant Github has Core Contributor in Rails? Name me a single Microsoft employees actively helping in Ruby Core.

John Hawthorn from GitHub is a ruby core committer.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#62

Post by bullen »

byroot wrote: post #60
bullen wrote: post #59

Earlier quoted context omitted.

Isn't HTTP/1.1 transfer encoding chunked a simpler solution to the same problem?

Sorry, I see absolutely no relation between the two.

When you chunk responses you save memory as the only buffer you need to keep is the chunk?

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#63

Post by nirvdrum »

bullen wrote: post #62
byroot wrote: post #60

Earlier quoted context omitted.

Sorry, I see absolutely no relation between the two.

When you chunk responses you save memory as the only buffer you need to keep is the chunk?

Neither Puma nor Pitchfork are generally used as a static file server since they're not particularly well-suited to that. They're used as Ruby application servers. The memory consumption and savings being discussed is oriented around the memory required by the Ruby VM to process a dynamic request (e.g., a request to a Ruby on Rails application).

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#64

Post by byroot »

bullen wrote: post #62
byroot wrote: post #60

Earlier quoted context omitted.

Sorry, I see absolutely no relation between the two.

When you chunk responses you save memory as the only buffer you need to keep is the chunk?

Ah I see. As Kevin said, the HTTP response is generally nothing compared to the memory required by the VM.

A response might be a couple MB, a medium sized Rails app will need a couple hundred MB to load it’s code etc.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#65

Post by bullen »

nirvdrum wrote: post #63
bullen wrote: post #62

Earlier quoted context omitted.

When you chunk responses you save memory as the only buffer you need to keep is the chunk?

Neither Puma nor Pitchfork are generally used as a static file server since they're not particularly well-suited to that. They're used as Ruby application servers. The memory consumption and savings being discussed is oriented around the memory required by the Ruby VM to process a dynamic request (e.g., a request to a Ruby on Rails application).

Ok, I was confused by "minimize memory usage by maximizing Copy-on-Write performance"...

That said: Static files never should be chunked.

Chunking is ONLY interesting with dynamic responses.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#66

Post by nirvdrum »

bullen wrote: post #65
nirvdrum wrote: post #63

Earlier quoted context omitted.

Neither Puma nor Pitchfork are generally used as a static file server since they're not particularly well-suited to that. They're used as Ruby application servers. The memory consumption and savings being discussed is oriented around the memory required by the Ruby VM to process a dynamic request (e.g., a request to a Ruby on Rails application).

Ok, I was confused by "minimize memory usage by maximizing Copy-on-Write performance"... That said: Static files never should be chunked. Chunking is ONLY interesting with dynamic responses.

That's fair. I was trying to guess where the misunderstanding arose from and I guessed wrong. I'm sorry about that.

Rounding out the previous thought, the idea with many forking servers is to boot up to the point before a request is served and then fork off for each request. You do gain CoW benefits, but if you have any lazy data structures that are reified in the call, now each child is faulting. Pitchfork will take a child that has processed a request and promote it as the parent, replacing the original process. Now, this new parent is the new CoW base with the expectation that forks from that will result in even greater memory sharing. For a framework like Rails, there's a lot that happens after a request is received.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#67

Post by wgjordan »

byroot wrote: post #53
wgjordan wrote: post #36

I'm the author of the original 'refork' feature introduced to Puma a couple years ago [1], that served as an inspiration for this project. As I understand it, Pitchfork is an implementation of a reforking feature on top of Unicorn, a single-threaded multi-process Rack HTTP server. The general idea of 'reforking' is that in addition to pre-forking worker processes at initialization time, you re-fork them on an ongoing…

Hey Will, Pitchfork author here. As mentioned in the Readme, thanks for your Puma PR, it was indeed quite instrumental in Pitchfork inception. As for why it wasn’t used much as a Puma feature I don’t know, but there are a few challenges with it that prevented me from putting it in production. The most important one being that new workers end up being grand children of the original process, so if the middle process di…

I noticed the double-fork with PR_SET_CHILD_SUBREAPER to reparent the new workers, which is a nice reliability improvement for the edge-case where the middle-parent worker possibly crashes. It adds a Linux dependency (as noted), but that probably still covers most production use-cases where the extra reliability that comes with reparenting is most needed. This enhancement could probably be incorporated into Puma.

As for the concern about forking a process that may have live threads currently processing a request, this should already be solved in the Puma implementation. The worker shuts down and finishes serving all pending requests before reforking. There is also an `on_refork` hook for to trigger extra garbage-collection to maximize copy-on-write efficiency, or close any connections to remote servers (database, Redis, ...) that were opened while the server was running.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#68

Post by gmmeyer »

skrtskrt wrote: post #11

I have now worked at several significantly-sized companies that ripped Ruby out of everything as they scaled, usually replaced with Go, though none as large as Shopify. I wonder what that financial calculus looks like between these options for a large organization: 1. trying to basically reinvent the workings of an entire programming language and migrate your Ruby apps to these new tools/runtimes/typecheckers/whateve…

My personal experience is that the bulk of the time these rewrites don't end up delivering the performance increases that people expect because they don't really understand what's making the prior system slow and don't develop good enough requirements

This is not saying you shouldn't, it doesn't really matter to me. But I think a lot of the time this is driven by engineers who feel like they need to do this rather than a financial decision to save money

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#69

Post by skrtskrt »

gmmeyer wrote: post #68
skrtskrt wrote: post #11

I have now worked at several significantly-sized companies that ripped Ruby out of everything as they scaled, usually replaced with Go, though none as large as Shopify. I wonder what that financial calculus looks like between these options for a large organization: 1. trying to basically reinvent the workings of an entire programming language and migrate your Ruby apps to these new tools/runtimes/typecheckers/whateve…

My personal experience is that the bulk of the time these rewrites don't end up delivering the performance increases that people expect because they don't really understand what's making the prior system slow and don't develop good enough requirements This is not saying you shouldn't, it doesn't really matter to me. But I think a lot of the time this is driven by engineers who feel like they need to do this rather th…

My experience is along the following lines (same for Django as well actually):

1. Company started as Rails monolith

2. Some components of the Rails monolith stretch Ruby capabilities beyond where "throw more servers at it" is still reasonable or effective

3. Factor out some backend functionality into dedicated services which can be scaled separately, Rails still serves as the API gateway calling these services. Anything without scaling issues stays in the monolith for now.

4. Eventually Rails is just an API gateway, no one in the org knows Ruby/Rails and its dependency management madness any more, and it gets replaced with a more-performant, purpose-built API gateway, usually something off-the-shelf.

Re: Pitchfork: Rack HTTP server for shared-nothing architecture

#70

Post by brigandish »

Puts wrote: post #29
treeman79 wrote: post #24

Earlier quoted context omitted.

Database locks is usual issue. Avoidable if planned better. Memory is second. In 15 years CPU has never been a real concern.

Second this. After spending several years of full time just troubleshooting others web apps I can say that not only are database locks in 99.99% of the cases the real bottleneck but also that developers in general are quite bad at managing databases and will try to micro optimizing just about anything before looking at their DB queries.

Most don't know how to improve their DB queries, it's like a black box to them, so they look at the parts they do know. Comfort level is king.
Post reply on HN