Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Efficient GraphQL Queries in Ruby on Rails and Postgres

pganalyze.com

Post reply on HN
Print view View original

21–30 of 67 posts

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#21

Post by aidos »

leighhalliday wrote: post #17
e12e wrote: post #13

Somewhat on the side, but: > By using the includes method we've been able to knock our queries down from six to two: The first to find the events, and the second to find the categories for those events. Am I the only one that thinks rails/activerecord is crazy here for (often) preferring two trips to the db over a join on indexed primary keys? Are there really (sane) use-cases where that is better?

If you knew ahead of time the fields that the user wanted to load, by all means do a join and load the data ahead of time. This would work great for RESTful APIs... but the issue with GraphQL is that you don't know ahead of time which fields the user will request, so lazy-loading the data provides efficiency without upfront knowledge of what the user will request.

You don’t need to know what they will request, you know what they did request. The dataloader thing is a kludge because you never know where you’ll have to go to get data for a particular part of the tree - maybe it’s another endpoint.

Having said that, I don’t mind the second request model myself. It’s actually pretty clean and when you take the dB roundtrips out of it, the dB will generally do the same work anyway.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#22

Post by bdcravens »

aidos wrote: post #20
bdcravens wrote: post #18

I'm leaning towards letting the elements of my app that need to be in Rails be in Rails, but moving the GraphQL layer to Hasura, bypassing the ActiveRecord complexity. Has anyone else taken this approach?

Yes. Well, we’ve built a prototype recently that just uses Hasura for the backend (don’t need anything other than data access at this stage). It’s a really great engine and I’m glad we took a punt on it. Aside from the expected behaviour, you get some nice bonus stuff like nested updates for free. We mostly tried it out because we normally use python and we wanted subscriptions. The graphql story in python (especiall…

How's the performance? As I recall, the way it does relationships is pretty sane as opposed to AR's defaults of N+1.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#23

Post by aidos »

X6S1x6Okd1st wrote: post #16
ben_jones wrote: post #15

Earlier quoted context omitted.

To play devil's advocate you could argue many web apps are small internal tools that provide lots of business value but only have small (even single digit) user numbers and thus shouldn't optimize in the slightest for db trips (and instead optimize for maintenance or simplicity).

Just because there are few users doesn't mean inefficiency is fine.

There are different types of inefficiency and being inefficient in one area might be worth the trade off to be efficient in another. Also, multiple dB queries really aren’t the end of the world.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#24

Post by honopu »

bdcravens wrote: post #18

I'm leaning towards letting the elements of my app that need to be in Rails be in Rails, but moving the GraphQL layer to Hasura, bypassing the ActiveRecord complexity. Has anyone else taken this approach?

I'm currently working on relegating the Rails app to the API, 95% of queries will be handled by elasticsearch, if it makes sense to make a GraphQL client, I will. Most of our searches are pre-defined, we're using elasticsearch mainly for boosting and relevance score, autocomplete, and having a unified search across a bunch of different installations such as a rails app, wordpress and a few others.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#25

Post by cuddlecake »

X6S1x6Okd1st wrote: post #16
ben_jones wrote: post #15

Earlier quoted context omitted.

To play devil's advocate you could argue many web apps are small internal tools that provide lots of business value but only have small (even single digit) user numbers and thus shouldn't optimize in the slightest for db trips (and instead optimize for maintenance or simplicity).

Just because there are few users doesn't mean inefficiency is fine.

On the other hand, it also does not mean that inefficiency to some degree is not acceptable.

And if it is not acceptable, you can still optimize the living excrements out of your GraphQL resolvers.

Also, the burden of doing more than one round trip usually moves to the client. So there's always some trade-off to be made.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#26

Post by aidos »

bdcravens wrote: post #22
aidos wrote: post #20

Earlier quoted context omitted.

Yes. Well, we’ve built a prototype recently that just uses Hasura for the backend (don’t need anything other than data access at this stage). It’s a really great engine and I’m glad we took a punt on it. Aside from the expected behaviour, you get some nice bonus stuff like nested updates for free. We mostly tried it out because we normally use python and we wanted subscriptions. The graphql story in python (especiall…

How's the performance? As I recall, the way it does relationships is pretty sane as opposed to AR's defaults of N+1.

Performance is really good (not that we’re using it at scale - but apparently even there it shines).

They use a really clever technique for loading where not only do they. pull all the child data for multiple parent records as a single hit, they actually run a single query for multiple connected clients.

So if you have 10,000 connected clients subscribed to the same graphql queries, they build a temp table with 10k rows (one per client) and add all the environment variables. Then they execute the queries once to get all the data for everyone at once (joining to the temp table and also joining to all the tables for access rules you set up in Hasura). Then they split it up and send back the relevant chunks to each client. It’s much more performant than Postgres RSL because the rules can be executed in the joins in a way that works for every client at the same time.

Edit mobile typos

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#27

Post by cuddlecake »

bdcravens wrote: post #22
aidos wrote: post #20

Earlier quoted context omitted.

Yes. Well, we’ve built a prototype recently that just uses Hasura for the backend (don’t need anything other than data access at this stage). It’s a really great engine and I’m glad we took a punt on it. Aside from the expected behaviour, you get some nice bonus stuff like nested updates for free. We mostly tried it out because we normally use python and we wanted subscriptions. The graphql story in python (especiall…

How's the performance? As I recall, the way it does relationships is pretty sane as opposed to AR's defaults of N+1.

There are a lot of per-query optimizations that can be done, and I think I have read somewhere that Hasura does them very aggressively, ensuring performance even on complicated queries.

In any case, you won't have to worry about N+1 Query problems with Hasura.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#28

Post by e12e »

blueplanet200 wrote: post #19
e12e wrote: post #13

Somewhat on the side, but: > By using the includes method we've been able to knock our queries down from six to two: The first to find the events, and the second to find the categories for those events. Am I the only one that thinks rails/activerecord is crazy here for (often) preferring two trips to the db over a join on indexed primary keys? Are there really (sane) use-cases where that is better?

It's more composable with multiple includes. For instance, if you have a simple active record type: class User has_many :comments; has_many :likes; end User.includes(:comments, :likes).find(user_id) Generates 3 queries, but importantly the number of records returned is appropriate. User.joins(:comments, :likes).where(users: { id: user_id }) Generates only a single query, but the number of records returned from this j…

So the problem is active record is no good at joins?

(That is, it's hard/impossible via the active record DSL to make the correct nested join?)

By the way (you may know this) when staying within active record convention, you should never need to single out the ".id" and ".foreign_model_id" fields:

User.joins(:comments, :likes).where(user: some_user)

(where some_user is an instance of User).

I'm still not quite comfortable enough with AR, but I think, leaning on this: https://www.learneroo.com/modules/137/nodes/768

I think what you want in your second case is simply?:

User.joins(:comments, :likes).where(user: some_user).uniq

(I'd have to create some models and watch what ".to_sql" creates to be certain, though)

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#29

Post by gav »

ben_jones wrote: post #14
xyzzy_plugh wrote: post #2

> The following query produces the data that we want... we just need to figure out how to write a batch loader that generates the same result. This is where the abstraction breaks down. Should you care what the query ultimately looks like? For anything beyond a toy, you sure should. But how far should you go to ensure the abstraction produces the implementation you want? Why not just skip to the implementation you wa…

I agree but can't ignore this idiom > If you start without an ORM you end up writing a custom ORM eventually and often that custom ORM is inferior to the third party one you're considering Funny enough this engineering problem becomes an HR one: If your recruiting and training processes can consistently hire devs actually fluent in SQL and web application architecture you can completely avoid the ORM and just write g…

I think this is a really great point, as often the HN answer is "hire better developers". The majority of companies outside the HN bubble are full of very average developers. ORMs work well in that sort of environment as it allows them to be productive, and any performance issues can be tackled by the less-average developers on the team.

A while back, I worked with a client who's stack was predominately Java but due to company policies paid 25-50% under the going rate for decent developers. This caused two problems: not being able to hire enough developers, and some of the hires not being great.

We solved that problem with a technical one: the stack moved to Clojure powering backend services with Node.js on the front end. The coolness and newness of those technologies at the time meant they could make hires. Ironically, now the demand for those technologies means they command high salaries and they're back at square one.

Bit of a long story, but my point is that sometime you need to solve people problems with technology and technology with people.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#30

Post by sandstrom »

We've looked at GraphQL but ended up going with Graphiti instead.

Only a few weeks since we implemented, but happy so far. Feels like it's the best of REST and GraphQL.

This page from Graphiti[1] explains the thinking behind it. Don't be discouraged by the loud pink colors ;)

[1] https://www.graphiti.dev/guides/why

Post reply on HN