Live data from Hacker News

Efficient GraphQL Queries in Ruby on Rails and Postgres

pganalyze.com

41–50 of 67 posts

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#41

> The following query produces the data that we want... we just need to figure out how to write a batch loader that generates the same result. This is where the abstraction breaks down. Should you care what the query ultimately looks like? For anything beyond a toy, you sure should. But how far should you go to ensure the abstraction produces the implementation you want? Why not just skip to the implementation you wa…

The problem with "just SQL" is that you arguably want to compose queries.

A foundational observation that GraphQL makes is that every query is different depending on the client's needs. You can do "SELECT * FROM users JOIN companies ON companies.id = users.employer_id" and call it a day, but now you're already probably overfetching, and the next evolutions of your codebase ends up accruing extra logic to add or remove features: Some queries want to fetch just a subset of fields from each of those relations. Some queries don't want to fetch any companies. Some want to paginate, filter, order in different ways.

You can do all that in REST, of course. You can use an SQL builder to build the final SQL query programmatically. You can add WHERE clauses and FULL OUTER JOIN clauses to do your N+1 queries on foreign associations, with some kind of query parameter to define what associations the client wants. Field selection is also doable with query parameter and so on. You can also centralize a lot of the boilerplate to do this (e.g. FULL OUTER JOIN clauses on associations, if you do them, need to be untangled into object graphs after you get the flat results back).

And so on.

What you end up with is pretty much GraphQL. You need code — identical for every relation, modulo client-specifiable constraints and parameters — for filtering, pagination, selection, joining, etc., all of those things. You need some notation for letting the client specify what they want to fetch, both fields and associations. Then, even when you have ended up with something approximates GraphQL, you have to write a custom client for it that only works for this exact API. And then you have to write documentation. And tests, tests which don't just exercise your "business logic", but also exercises all the REST-based query API stuff.

The benefits to GraphQL are that it's standard, so you can piggyback off existing libraries both on the client side (things like Apollo) and on the backend, libraries that are already tested and documented. GraphQL itself defines a schema, which serves as documentation and a way to machine-generate clients. As a case in point, try out Prima's GraphQL playground app [1]. It's basically a web-based IDE for GraphQL. You can point the app at any GraphQL API, inspect the schema, write queries (you can autocompletion of all fields, queries, mutations, etc.) and see live results. It's a really productive debugging tool.

That alone is worth a lot, and that's why I'm bullish on GraphQL.

[1] https://github.com/prisma-labs/graphql-playground

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#42
After https://stackoverflow.com/a/57247614/308851 I am at a complete loss of why I would I want GraphQL. A client uses it , so I obey but I do not see the point. To quote the linked answer to save you a click:

> GraphQL does not provide any built-in mechanisms for filtering, sorting, pagination or other arbitrary transformations of the response

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#43
post #34

Earlier quoted context omitted.

Yes. I'm not sure about "non-trivial amount of data" - especially if we're talking about a single row (single user) here? I suppose there might be a few blolbs of large json documents? And I'm not sure I understand "repeating information in query results" - surely we're talking about receiving a query, transforming it and responding with the data? In a rails+graphql ideal world, that'd maybe mean Parsing the json que…

>I'm not sure about "non-trivial amount of data" - especially if we're talking about a single row (single user) here? I suppose there might be a few blolbs of large json documents? > >And I'm not sure I understand "repeating information in query results" - surely we're talking about receiving a query, transforming it and responding with the data? For a query like SELECT u.* , c.* FROM users u INNER JOIN comments c ON…

No, you're probably right - I suppose the database drivers might naively serialize that in a one-to-one representation (to be honest, I've never really looked into the protocols that postgres or Ms sql use for transferring data).

I didn't consider the kind of expansion you're alluding to - on the other hand i can't ever remember seeing an indication that the amount of data returned is an issue with app performance.

But I've certainly seen the number of round-trips result in real issues. And complicated joins, for that matter.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#44

> The following query produces the data that we want... we just need to figure out how to write a batch loader that generates the same result. This is where the abstraction breaks down. Should you care what the query ultimately looks like? For anything beyond a toy, you sure should. But how far should you go to ensure the abstraction produces the implementation you want? Why not just skip to the implementation you wa…

The problem with "just SQL" is that you arguably want to compose queries. A foundational observation that GraphQL makes is that every query is different depending on the client's needs. You can do "SELECT * FROM users JOIN companies ON companies.id = users.employer_id" and call it a day, but now you're already probably overfetching, and the next evolutions of your codebase ends up accruing extra logic to add or remov…

I think GraphQL will certainly find it's place after the initial hype. There are some good use cases. But in my opinion it should be only used on a need basis.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#45
post #42

After https://stackoverflow.com/a/57247614/308851 I am at a complete loss of why I would I want GraphQL. A client uses it , so I obey but I do not see the point. To quote the linked answer to save you a click: > GraphQL does not provide any built-in mechanisms for filtering, sorting, pagination or other arbitrary transformations of the response

You wouldn't want to let people filter by an unindexed field, sort huge large result sets, or make your server store a huge result set because they stop making requests after getting the first few pages. GraphQL lets you express what is permitted, without allowing more than your underlying data store can support.

For example, from the GitHub GraphQL API you can in a single request get all open pull requests on a repository and the last 10 comments on each one. You can't get the last 10 comments on all of GitHub or the last 10 comments made by a particular user, because they haven't exposed or even stored the data that way.

In a REST API that would be dozens of separate requests, or the endpoint would just not exist because they have never had the need for that exact combination of records to be queryable.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#46
post #20

I'm leaning towards letting the elements of my app that need to be in Rails be in Rails, but moving the GraphQL layer to Hasura, bypassing the ActiveRecord complexity. Has anyone else taken this approach?

Yes. Well, we’ve built a prototype recently that just uses Hasura for the backend (don’t need anything other than data access at this stage). It’s a really great engine and I’m glad we took a punt on it. Aside from the expected behaviour, you get some nice bonus stuff like nested updates for free. We mostly tried it out because we normally use python and we wanted subscriptions. The graphql story in python (especiall…

Have you tried Prisma? How it compares with Hasura?

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#47
post #43

Earlier quoted context omitted.

>I'm not sure about "non-trivial amount of data" - especially if we're talking about a single row (single user) here? I suppose there might be a few blolbs of large json documents? > >And I'm not sure I understand "repeating information in query results" - surely we're talking about receiving a query, transforming it and responding with the data? For a query like SELECT u.* , c.* FROM users u INNER JOIN comments c ON…

No, you're probably right - I suppose the database drivers might naively serialize that in a one-to-one representation (to be honest, I've never really looked into the protocols that postgres or Ms sql use for transferring data). I didn't consider the kind of expansion you're alluding to - on the other hand i can't ever remember seeing an indication that the amount of data returned is an issue with app performance. B…

I've worked on a few Postgres drivers and the protocol is basically sending data row-by-row, column-by-column. Sure, it could be more clever, but that's time and cpu and memory spent being clever instead of moving data.

But you're right, there are always trade-offs, and there are some situations where the current AR approach is not ideal, and a single join would be more efficient. However, I suspect this is the way AR does it because it's right enough often enough that it works well as the default approach.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#48

We've looked at GraphQL but ended up going with Graphiti instead. Only a few weeks since we implemented, but happy so far. Feels like it's the best of REST and GraphQL. This page from Graphiti[1] explains the thinking behind it. Don't be discouraged by the loud pink colors ;) [1] https://www.graphiti.dev/guides/why

It's strange to me that they pointed out Domain Driven Design without any mention of CQRS (which isn't strictly linked to DDD, but is commonly associated) or how Graphiti would help with some of the pitfalls of CRUD-based APIs. In fact, earlier in the post they point out that the built in "verb" links feature only works with CRUD, and all of there examples are CRUD based.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#49
post #37
post #35

Earlier quoted context omitted.

I've seen many mobile developers be frustrated at constantly needing to ask the backend developers to make new rest endpoints for them because the existing ones were not exposing the data in the way they needed it exposed. And similarly I've seen backend devs be frustrated at continually having to make new endpoints for those nagging mobile devs. If you have teams working independently of one another, where the backe…

> here's access to anything you need - go ahead and use it however you need But that's almost never what you want: as the backend developer, it's one's job to care about security and the performance of queries. As far as I know there's no straightforward way of automagically translating every possible permutation of SQL query primitives into GraphQL primitives in a way that is intuitive to a frontend developer, reaso…

I'd say that that you can generalize fulfilling the queries and making them secure. It's not easy to generalize making them performant. But you can be more reactive on performance, when a query becomes a problem you break out of the general system and write something optimized.

Re: Efficient GraphQL Queries in Ruby on Rails and Postgres

#50
post #42

After https://stackoverflow.com/a/57247614/308851 I am at a complete loss of why I would I want GraphQL. A client uses it , so I obey but I do not see the point. To quote the linked answer to save you a click: > GraphQL does not provide any built-in mechanisms for filtering, sorting, pagination or other arbitrary transformations of the response

GraphQL the language does not specify those features in a rigid way, many GraphQL implementations have those features. It's best to look at the spec as the minimum to expect not the maximum you can get from GraphQL.
Post reply on HN