Debugging GraphQL N+1 Issues with Open Source Tracing Tools
1–10 of 14 posts
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#2It's great that we're talking about tracing / OTEL (Open Telemetry) in the context of GraphQL. We've recently added OTEL support to WunderGraph, allowing our users to trace through the Gateway and their services. Some more info on our OSS implementation can be found here: https://docs.wundergraph.com/docs/guides/monitoring-and-obse...
In addition, we've deployed all the required infrastructure like an OTEL Collector and OTEL backend using Clickhouse to offer a managed end-to-end solution including dashboard for traces and a trace overview. A live demo can be found here (sign up for free into the demo org): https://cloud.wundergraph.com/wundergraph-demo/apollo-federa...
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#3When my tech lead decided we were going to move to GraphQL, one of the reasons I was told it was great was because it solves/avoids the n+1 problem.
But from this it sounds like that's something you have to detect and code for directly... which we'd have happily done in the existing code anyway?
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#4When my tech lead decided we were going to move to GraphQL, one of the reasons I was told it was great was because it solves/avoids the n+1 problem. But from this it sounds like that's something you have to detect and code for directly... which we'd have happily done in the existing code anyway?
I made this related comment about N+1 on a netflix graphql post a few days ago... some of the responses might be interesting to you.
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#5When my tech lead decided we were going to move to GraphQL, one of the reasons I was told it was great was because it solves/avoids the n+1 problem. But from this it sounds like that's something you have to detect and code for directly... which we'd have happily done in the existing code anyway?
Sorry to hear you have an ill informed tech lead :(
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#6When my tech lead decided we were going to move to GraphQL, one of the reasons I was told it was great was because it solves/avoids the n+1 problem. But from this it sounds like that's something you have to detect and code for directly... which we'd have happily done in the existing code anyway?
> one of the reasons I was told it was great was because it solves/avoids the n+1 problem
There's nothing inherent in either GraphQL or REST that admits or prevents the N+1 problem. You have endpoints with business logic (or in GQL's case, "resolvers") and there's no telling what's inside of those methods code.
It's true that GraphQL was built to integrate with tools like a "Dataloader" to solve "N+1", but you could just as validly be using a Dataloader in your REST or gRPC API endpoints.
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#7When my tech lead decided we were going to move to GraphQL, one of the reasons I was told it was great was because it solves/avoids the n+1 problem. But from this it sounds like that's something you have to detect and code for directly... which we'd have happily done in the existing code anyway?
I have used Dataloaders in the past in both Go and Typescript and it just about solves the issue automatically. Dataloader is a specific library that is copied for different languages that allows you to write code that looks like N+1, but it will group the N part into batches, so it becomes just 1+1. It’s kind of weird to explain, but it should be easy to understand once you start working with them.
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#8When my tech lead decided we were going to move to GraphQL, one of the reasons I was told it was great was because it solves/avoids the n+1 problem. But from this it sounds like that's something you have to detect and code for directly... which we'd have happily done in the existing code anyway?
I have used Dataloaders in the past in both Go and Typescript and it just about solves the issue automatically. Dataloader is a specific library that is copied for different languages that allows you to write code that looks like N+1, but it will group the N part into batches, so it becomes just 1+1. It’s kind of weird to explain, but it should be easy to understand once you start working with them.
Dataloaders replace N+1 with E queries (where E is the number of separate entity types to fetch) and an in-memory merging of all datasets (huge if you're not careful).
They are a solution, but not necessarily a good one. And GraphQL doesn't lend itself to goos solutions.
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#9When my tech lead decided we were going to move to GraphQL, one of the reasons I was told it was great was because it solves/avoids the n+1 problem. But from this it sounds like that's something you have to detect and code for directly... which we'd have happily done in the existing code anyway?
It doesn’t solve it, but it lets you build a system that solves it for all future queries going forward. Think of it as a platform for taking endpoint specifications (ie GraphQL queries) and satisfying them efficiently. You have to build the part which makes it efficient, but once that’s done, it’s done.
Re: Debugging GraphQL N+1 Issues with Open Source Tracing Tools
#10For the first production-scale GraphQL servers I built, I just had it dump ascii art representations of the downstream request waterfalls during dev. This was enough to visualise and fix any n+1 issues.