Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lessons learned from running GraphQL

blog.dream11engineering.com

Post reply on HN
Print view View original

41–50 of 64 posts

Re: Lessons learned from running GraphQL

#41

Post by pdimitar »

mirekrusin wrote: post #22
yxhuvud wrote: post #19

Earlier quoted context omitted.

To be fair, looking at what kind of optimizations they did to improve the situation, it looks like GQL is not to blame but rather a pretty big disconnect between their implementation practices and understanding about what is costly and what is not costly (ie lack of mechanical sympathy). People may get by without that writing client side javascript, but backend code is not as generous.

Yes, their optimisation team did great job, including writing post about it. They had 5 months from zero to some solution deployed on prod. And they shaved numbers. Great work. However number of servers required to serve this traffic has still very poor ratio. And if you read between lines – they maxed out their optimisation effort on it. They cache at multiple layers. To be more precise this statement that they have…

Yep, they seem to have settled at about 2100 servers which is bonkers insane. Go or Rust will likely max at no more than 100. If not 20-30 even.

Re: Lessons learned from running GraphQL

#42

Post by l-lousy »

pdimitar wrote: post #40
nesarkvechnep wrote: post #31

Earlier quoted context omitted.

Seems to me Elixir + Absinthe would’ve been a better tech stack.

I adore Elixir and have made a good career around it. I'll love it until my grave. But with that many servers I can't help but wonder if Rust wouldn't be a better choice. I don't think they'll even need 100 which, as OP pointed out, will drastically reduce their operational complexity. Furthermore, the strong static typing would have avoided the megamorphic functions right from the start.

Yeah I agree with rust, I have some simple database fetching servers that run crypto functions as well (I.e. login) and for simpler operations they can quickly get to 100K+ requests per second on a single host

Re: Lessons learned from running GraphQL

#43

Post by Zababa »

fabian2k wrote: post #18

The big items in that list of performance issues don't seem to have anything to do with GraphQL. They seem to be related to a heavily function style using lots of immutability and the Ramda library. I'd also suspect that these choices are responsible for the GC issues due to lots of allocations for the immutable objects. I know, premature optimization and all that. But I think at the point where you're going for micr…

If you're using microservices, it may be a good oppurtunity to try a language that supports immutability and the heavy functional style without the loss of performance. Something like Elixir, Haskell, OCaml, Rust.

Re: Lessons learned from running GraphQL

#44

Post by Zababa »

jjtheblunt wrote: post #4
ripper1138 wrote: post #3

I wonder if they feel like GraphQL was worth it, vs. normal API servers. Maybe they saved some dev time on the front end, but did that outweigh the dev time spent on building, optimizing, etc? Somehow I doubt it.

I wonder if they feel like Node was worth it. (seriously)

I wonder why we haven't seen yet some more optimisation for JS. The only thing I'm aware of is the Google Closure Compiler. With everyone using TypeScript, there must be a way to use all that type information to make JS run more efficiently, or compile parts of your TypeScript code.

Re: Lessons learned from running GraphQL

#45

Post by orf »

Wouldn’t using a lambda for this be a good choice? You’re just parsing a input document into a set of backend requests and then executing them - there doesn’t have to be anything stateful here that would require an actual running instance.

If you combine this with API-gateway you’ve got caching (and potentially token auth) for free.

Re: Lessons learned from running GraphQL

#46

Post by doctor_eval »

zelphirkalt wrote: post #28
doctor_eval wrote: post #8

Earlier quoted context omitted.

I’ve been using graphql for years. In my experience, it dramatically simplifies microservice API architecture vs “normal” API servers. Graphql is super easy to understand, easy to deploy, easy to scale and easy to grow. It’s not perfect - the lack of namespaces can be a pain, a few more standard types would be good, and mutations feel a bit under baked - but there’s much to love, and very little to dislike.

Downsides afaik are: (1) No way to do queries, which return recursive JSON objects of arbitrary depth. (2) Not using standard JSON as a format for writing your query, instead unnecessarily making up a new querying lang, a design flaw basically. (3) More dependencies in frontend as well as backend. (4) More difficult to determin what exactly is going on in processing 1 query, ergo more difficult to fix performance pro…

Can’t say I agree.. (1) you can easily create a json data type to emit arbitrary json in your response if that floats your boat (2) graphql queries can be far more expressive than straight json (3) graphql is just a REST call that takes a string and some optional JSON and returns JSON, no need for client side libraries unless you have complex use cases that are enabled by Graphql, (4) this has not been my experience, queries are run by the backend services, not by graphql itself, so the complexity of an individual backend query does not change.

Just my experience, we can agree to disagree!

Re: Lessons learned from running GraphQL

#47

Post by FpUser »

lbriner wrote: post #39
FpUser wrote: post #9

>"We provision approximately 7,500 instances for 1 million requests per second." Looking at this numbers makes me think that a single instance of properly written server running on a single dedicated piece of hardware can handle this without breaking a sweat. My servers for example handle thousands of requests per second. It looks to me like one giant waste of human and hardware resources. Not very "green" approach I…

The benefit you are getting from GraphQL is massive flexibility of the query in return for probably sub-par performance. The fact that you can get X 100K requests/per second best-case is not really the point. The point is if I don't want to write hand-cranked code for every kind of possible query, I take a performance hit as a result. Not sure how easy it would be for them to identify poorly performing queries and sp…

>"The point is if I don't want to write hand-cranked code for every kind of possible query, I take a performance hit as a result."

This is how we end up with the architectures consuming orders of magnitude more computing resources and giant management overhead. Just because someone wants to be spared from a bit of thinking.

I can see how GraphQL would work for orgs with the massive scale like FB/Google/Insert your favorite. For the most of rest of the world it is nothing but unneeded overhead on resource both human and computing.

And of course cloudy people like Amazon would love you to use all this tech. The more you slow down your application the more resources you will be leasing from them so they get more money

Re: Lessons learned from running GraphQL

#48

Post by jensneuse »

orf wrote: post #45

Wouldn’t using a lambda for this be a good choice? You’re just parsing a input document into a set of backend requests and then executing them - there doesn’t have to be anything stateful here that would require an actual running instance. If you combine this with API-gateway you’ve got caching (and potentially token auth) for free.

What you describe already exists. I'm the founder of https://wundergraph.com and we're doing exactly what you describe, combining GraphQL with Auth and Caching, plus some more extras...

Re: Lessons learned from running GraphQL

#49

Post by FpUser »

quonn wrote: post #26
FpUser wrote: post #9

>"We provision approximately 7,500 instances for 1 million requests per second." Looking at this numbers makes me think that a single instance of properly written server running on a single dedicated piece of hardware can handle this without breaking a sweat. My servers for example handle thousands of requests per second. It looks to me like one giant waste of human and hardware resources. Not very "green" approach I…

The default implementation of GraphQL has a lot of overhead in query parsing and validation alone. You can try this yourself with complex queries and simulating some load. But it‘s an issue that can be solved.

">You can try this yourself with complex queries and simulating some load."

I do not need to try it. I know what it takes to parse/validate this kind of queries and then manage to get and assemble the results from numerous sources.

>"But it‘s an issue that can be solved."

No. This issue will not be solved as in general it is a problem of mapping one storage / functionality format to end client format. It can be easily solved for particular situations by writing custom servers (this is for example one of the things I do) but doing it generically introduces overhead / costs that are very unhealthy for a normal businesses.

And it is of course bad as it wastes energy.

Re: Lessons learned from running GraphQL

#50

Post by orf »

jensneuse wrote: post #48
orf wrote: post #45

Wouldn’t using a lambda for this be a good choice? You’re just parsing a input document into a set of backend requests and then executing them - there doesn’t have to be anything stateful here that would require an actual running instance. If you combine this with API-gateway you’ve got caching (and potentially token auth) for free.

What you describe already exists. I'm the founder of https://wundergraph.com and we're doing exactly what you describe, combining GraphQL with Auth and Caching, plus some more extras...

I was referring to deploying the graphql routers onto an AWS lambda within your own account, which looks entirely different from your product?
Post reply on HN