How and why GraphQL will influence the Sourcehut alpha • Hacker News Board

Re: How and why GraphQL will influence the Sourcehut alpha

#121

Post by hurricaneSlider » Thu, Jun 11, 2020, 1:11 PM UTC

dmitriid wrote: post #56

WhatIsDukkha wrote: post #31

I don't understand the attraction to Graphql. (I do understand it if maybe you actually want the things that gRPC or Thrift etc gives you) It seems like exactly the ORM solution/problem but even more abstract and less under control since it pushes the orm out to browser clients and the frontend devs. ORM suffer from being at beyond arms length from the query analyzer in the database server. https://en.wikipedia.org/w…

> I don't understand the attraction to Graphql. It's attractive primarily to frontend developers. Instead of juggling various APIs (oftne poorly designed or underdesigned due to conflicting requirements and time constraints) you have a single entry into the system with almost any view of the data you want. Almost no one ever talks about what a nightmare it becomes on the server-side, and how inane the implementations…

> usable only for internal projects where you have full control of who has access to your system, and can't bring it down because you forgot an authorization on a field somewhere or a protection against unlimited nested queries.

As someone who is building a public facing GraphQL API, I would disagree with this. Directives make it easy to add policies to types and fields in the schema itself, making it amenable to easy review.

A restful API also has the problem that if you want fine grained auth, you'll need to remember to add the policy to each controller or endpoint, so not that different.

The typed nature of GraphQL offers a way of extending and enriching behavior of your API in a very neat, cross cutting way.

For example we recently built a filtering system that introspected over collection types at startup to generate filter input types. We then built middleware that converted filter inputs into query plans for evaluation.

I previously worked at another company that offers a public REST API for public transport. Public transport info is quite a rich interconnected data set. Despite efforts to ensure that filtering was fairly generic, there was a lot of adhoc code that needed to be written to handle filtering. The code grew exponentially more complex as more filters were added. Maybe this system could have been architected in a better way, but the nature of REST doesn't make that exactly easy to do.

Bottom line is that I feel for public APIs, that there is a lot of demand for flexibility, and eventually a public facing RESTful API will grow to match or even exceed that of a GraphQL API in complexity.

Re: How and why GraphQL will influence the Sourcehut alpha

#122

Post by fastball » Thu, Jun 11, 2020, 1:14 PM UTC

StavrosK wrote: post #111

fastball wrote: post #110

Earlier quoted context omitted.

I built the backend for my knowledge-base platform[0] using Flask originally, but performance was definitely a struggle so I rewrote the whole thing with FastAPI. Have definitely seen a serious performance bump from that switch, and currently am quite happy with it. Many of our users are actually impressed with how fast everything is on the platform. I still want to rip out SQLAlchemy ORM and replace it with pure SQL…

That sounds like a good solution, and is a good data point to know, thank you. Did you try Sync FastAPI? I'm wondering how its performance compares with async

When you say sync do you just mean having sync endpoints? If so, then yeah, it's required since we're using SQLAlchemy ORM. Otherwise the calls to SQLA ORM would block the main event loop. As it is, FastAPI (well really Starlette) creates threads for sync endpoints to prevent blocking the main thread/event loop.

So yeah, still seeing good speedups in our own benchmarks even though most of our endpoints are sync.

What was arguably more important though was how much switching to ASGI helped with handling WebSockets. We're using SocketIO, and trying to get a fundamentally async protocol working within sync (Flask) land was a massive pain. We had repeated reliability and deployment issues that were very hard to debug. Switching to FastAPI made that much easier.

Re: How and why GraphQL will influence the Sourcehut alpha

#123

Post by yuchi » Thu, Jun 11, 2020, 1:16 PM UTC

DrFell wrote: post #118

WhatIsDukkha wrote: post #31

I don't understand the attraction to Graphql. (I do understand it if maybe you actually want the things that gRPC or Thrift etc gives you) It seems like exactly the ORM solution/problem but even more abstract and less under control since it pushes the orm out to browser clients and the frontend devs. ORM suffer from being at beyond arms length from the query analyzer in the database server. https://en.wikipedia.org/w…

GraphQL is instant API to frontenders. Their justification for needing it is that the API team takes too long to implement changes, and endpoints never give them the data shape they need. The silent reason is that server-side code, databases, and security are a big scary unknown they are too lazy to learn. A big project cannot afford to ask for high standards from frontenders. You need a hoard of cheap labor to crank…

There’s also a different side of the story: UI are very pricey to get right — that’s because user research is not cheap — so a very efficient approach is having disposable UIs since you know you are going to get them wrong at the start.

Carefully designed endpoints require a lot of back and forth between teams with very different skill set so to build them you need to plan in advice, but it does not work here where you literally have to move fast (and fail fast!).

You only have two options left: (1) you either ask frontenders (or UX-wise devs) to do the whole thing or (2) you build an abstraction layer that let frontenders query arbitrarily complex data structures with near-perfect performances (YMMV).

In case (1) you’re looking for REAL full-stacks, and it’s not that easy to find such talented developers. In case (2) well… that’s GraphQL.

Re: How and why GraphQL will influence the Sourcehut alpha

#124

Post by StavrosK » Thu, Jun 11, 2020, 1:19 PM UTC

fastball wrote: post #122

StavrosK wrote: post #111

Earlier quoted context omitted.

That sounds like a good solution, and is a good data point to know, thank you. Did you try Sync FastAPI? I'm wondering how its performance compares with async

When you say sync do you just mean having sync endpoints? If so, then yeah, it's required since we're using SQLAlchemy ORM. Otherwise the calls to SQLA ORM would block the main event loop. As it is, FastAPI (well really Starlette) creates threads for sync endpoints to prevent blocking the main thread/event loop. So yeah, still seeing good speedups in our own benchmarks even though most of our endpoints are sync. What…

Oh, I can imagine, ASGI must be immeasurably easier. Where do the async speedup gains come from, though, if your database is still sync? Wouldn't threadpools provide comparable performance before?

Re: How and why GraphQL will influence the Sourcehut alpha

#125

Post by say_it_as_it_is » Thu, Jun 11, 2020, 1:46 PM UTC

gbear605 wrote: post #102

say_it_as_it_is wrote: post #73

Where are the GraphQL lessons learned? The author hasn't even implemented a solution with it yet, but that hasn't stopped him from declaring it to the world. I don't find an announcement useful. Maybe GraphQL adopters aren't sharing their experiences with it in production because they're realizing its faults? People are quick to announce successes and very reluctant to own, let alone share, costly mistakes. Also, peo…

The reason this post was written was for users of Sourcehut, especially for people writing to its API. This post isn’t particularly relevant or explanatory for other audiences, but I don’t think it’s supposed to be so that’s fine.

I understand what you mean. That makes sense.

Re: How and why GraphQL will influence the Sourcehut alpha

#126

Post by gavinray » Thu, Jun 11, 2020, 1:49 PM UTC

dmitriid wrote: post #95

rhlsthrm wrote: post #79

Earlier quoted context omitted.

As a full-stack dev, I'm going to always reach for things like Hasura for building my backend from now on. It auto generates a full CRUD GraphQL API from my Postgres DB schema. Most of the backend boilerplate is eliminated this way. If I need to add additional business logic, I can use serverless functions that run between the GraphQL query from the front end and the DB operations (actions in Hasura). Most of the hea…

We're exploring a GraphQL serv(er/ice) for an internal back office system. It needs to combine multiple APIs into a single GraphQL interface. And everything just breaks apart :) (we have PoCs in C# and Java for now).

You can use Remote Schemas in Hasura to combine multiple API's, and Remote Joins to join relational data across data sources:

https://hasura.io/docs/1.0/graphql/manual/remote-schemas/ind...

https://hasura.io/blog/remote-joins-a-graphql-api-to-join-da...

If you need to convert your API's into a GraphQL first, you can wrap the endpoints with resolvers yourself, or use automated tooling:

https://github.com/Urigo/graphql-mesh

https://github.com/IBM/openapi-to-graphql

Re: How and why GraphQL will influence the Sourcehut alpha

#127

Post by karatestomp » Thu, Jun 11, 2020, 1:54 PM UTC

kabes wrote: post #45

kevan wrote: post #35

Earlier quoted context omitted.

Seems like you're looking at this through the lens of a single system that could submit a query to a single database and get all the data it needs. From that perspective GraphQL is definitely an extra layer that probably doesn't make sense. But even then there's still some value in letting the client specify the shape of the the data it needs and having client SDKs (there's definitely non-GraphQL ways to achieve thes…

That's indeed one of its selling points. But most products I see that adopt graphql are exactly 1 database.

There are automagic GraphQL layers that sort-of make sense to me, since at least they remove the biggest pain points. But AFAIK they're all single-database.

Actually stitching together multiple services or DBs with it manually seems like it'd be a hellish experience that'd end in a massive data breech or repeated accidental dataloss + restore-from-backup. Or else valid GraphQL going to such a system would be so restricted that the benefit over just using REST (or whatever) is zero.

Re: How and why GraphQL will influence the Sourcehut alpha

#128

Post by xgenecloud » Thu, Jun 11, 2020, 1:54 PM UTC

eveningcoffee wrote: post #99

rhlsthrm wrote: post #79

Earlier quoted context omitted.

As a full-stack dev, I'm going to always reach for things like Hasura for building my backend from now on. It auto generates a full CRUD GraphQL API from my Postgres DB schema. Most of the backend boilerplate is eliminated this way. If I need to add additional business logic, I can use serverless functions that run between the GraphQL query from the front end and the DB operations (actions in Hasura). Most of the hea…

It is not really a backend when it does not involve business logic but just an access layer for the DB. It is pretty much a client server model.

hey, you might want to check xgenecloud where it is seamless to add business logic for generated APIs.

XgeneCloud makes it really simple to add business logic for generated APIs (REST and GraphQL both) over any SQL databases.

We just launched this week [2]

[1] : https://github.com/xgenecloud/xgenecloud

[2] : https://news.ycombinator.com/item?id=23466782

Website : https://xgenecloud.com

(disclaimer: founder here)

Re: How and why GraphQL will influence the Sourcehut alpha

#129

Post by cnorthwood » Thu, Jun 11, 2020, 1:56 PM UTC

WhatIsDukkha wrote: post #31

I don't understand the attraction to Graphql. (I do understand it if maybe you actually want the things that gRPC or Thrift etc gives you) It seems like exactly the ORM solution/problem but even more abstract and less under control since it pushes the orm out to browser clients and the frontend devs. ORM suffer from being at beyond arms length from the query analyzer in the database server. https://en.wikipedia.org/w…

I don't see GraphQL as an ORM type solution, I see it more like a replacement for REST.

Re: How and why GraphQL will influence the Sourcehut alpha

#130

Post by fastball » Thu, Jun 11, 2020, 2:00 PM UTC

StavrosK wrote: post #124

fastball wrote: post #122

Earlier quoted context omitted.

When you say sync do you just mean having sync endpoints? If so, then yeah, it's required since we're using SQLAlchemy ORM. Otherwise the calls to SQLA ORM would block the main event loop. As it is, FastAPI (well really Starlette) creates threads for sync endpoints to prevent blocking the main thread/event loop. So yeah, still seeing good speedups in our own benchmarks even though most of our endpoints are sync. What…

Oh, I can imagine, ASGI must be immeasurably easier. Where do the async speedup gains come from, though, if your database is still sync? Wouldn't threadpools provide comparable performance before?

Well so actually we have both now.

For WebSockets, all of the code is async, so I'm already using `asyncpg` for any database stuff that is happening there.

With regards to why are the sync endpoints faster, I think it is a number of things, some of which are userland changes that could've been made under Flask, but all of which are somewhat related to the switch. With regards to things that FastAPI itself has changed, I think using a (de)serialization lib like Pydantic and serializing to JSON by default (which is what we were doing under Flask anyway, though with Marshmallow) makes a lot of the code paths in the underlying lib a bit faster, because with Flask there was more "magic" going on behind the scenes. For userland stuff, I think partly because there is less magic going in the background (I really like FastAPIs dependency injection system), it's made it easier to identify the bottlenecks and optimize hot code paths.