Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How and why GraphQL will influence the Sourcehut alpha

sourcehut.org

Post reply on HN
Print view View original

21–30 of 232 posts

Re: How and why GraphQL will influence the Sourcehut alpha

#21

Post by tmpz22 »

Big OOF. Even the flow of the article is muddied to the point where it takes significant effort to dig out the main points for OP's migration from REST to GraphQL. What I got from it:

> The system would become more stable with the benefit of static typing, and more scalable with a faster and lighter-weight implementation.

OK static typing I'm with you so far. Faster and lighter weight? I'm not so sure, sounds like you're having troubles with Flask and SQLAlchemy completely unrelated to REST. All production graphQL implementations I've seen are very heavy when they add in authentication and more advanced query capabilities. Is this REALLY so superior to REST?

> Another (potential) advantage of GraphQL is the ability to compose many different APIs into a single, federated GraphQL schema.

I guess the discoverability of GraphQL is better, but 90% of APIs on the internet prove that large REST APIs are very effective and achieve the same thing.

> I also mentioned earlier that I am unsatisfied with the Python/Flask/SQLAlchemy design that underlies most of SourceHut’s implementation. The performance characteristics of this design are rather poor, and I have limited options for improvement. The reliability is also not something I am especially confident in.

This is where you completely lose me. It's fine if you hate ORMs, its fine if you hate the SQLAlchemy API, but you're blaming your hammer for the fact that you think you built a shoddy house. Going out and buying a hammer won't fix the fact that you're lining up your nails all wrong.

> The GraphQL services are completely standalone, and it is possible to deploy them independently of the web application...

> ...it is my intention to build out experimental replacement backends which are routed through GraphQL instead.

I think these two captions go together, are you describing microservices? This can be achieved just fine with REST using simple load balancing strategies based on url routing or similar.

> Almost all of SourceHut today is built with Python, Flask, and SQLAlchemy, which is great for quickly building a working prototype. This has been an effective approach to building a “good” service and understanding the constraints of the problem space. However, it’s become clear to me that this approach isn’t going to cut it in the long term, where the goal is not just “good”, but “excellent”

This is a classic example of using a handful of annoying issues to justify an exciting large re-write that doesn't actually address the main issues you are having. If you are struggling with the SQLAlchemy library you will find alternative (and perhaps larger) struggles in all GraphQL implementations. Best of luck, this is a road I would not follow you on. Seriously though I wish you the best and hope your product succeeds despite this.

Re: How and why GraphQL will influence the Sourcehut alpha

#22

Post by CameronNemo »

philosopher1234 wrote: post #19
CameronNemo wrote: post #13

Earlier quoted context omitted.

Why? I am currently converting a Flask/SQLAlchemy application to Go and gRPC. Go has not been that much of a win from my perspective. Lots of rough edges and missing functionality in libraries available, a lackluster type system that does enough to get in the way but not enough to properly express programmatic intention, and an atypical non-ideal error handling model have left me with little reason to champion Go. No…

Could you name some specific rough edges and missing functionality that have frustrated you? I'm a big fan of Go and have a good experience with it, so I'd be very curious to hear from someone with an opposite experience.

On several occasions I have gotten panics (segfaults), many times without using CGo. This is not something I would expect from a "high level" language that has a garbage collector.

The constant use of generated code is another real pain point, particularly when I am writing business logic that needs to operate on generated types for which there is no interface available that does what I need (and why would there be? how would the library/codegen-tool author know all the permutations of business logic that might be out there?).

The sql library has some fairly annoying documentation, e.g.

> Scan implements the Scanner interface.

https://golang.org/pkg/database/sql/#NullBool.Scan

> Scanner is an interface used by Scan.

https://golang.org/pkg/database/sql/#Scanner

There is only really one concrete example of how to use Scan on the godoc page.

The introspection capabilities (reflect package) are quite obtuse and error prone (basically requiring 100% test coverage because you lose nearly the entire type system), yet absolutely critical to implementing anything notably complex with Go.

Re: How and why GraphQL will influence the Sourcehut alpha

#23

Post by [deleted] »

ianamartin wrote: post #15
ghoshbishakh wrote: post #6

Earlier quoted context omitted.

what do you feel the problem is?

There are lots of production sites that serve 10,000x as much traffic as sourcehut that are built on Python/flask/sqlalchemy serving RESTful APIs. If you can't make that combination work well, there's another place to look for problems besides your tool kit. You might need to ask yourself if you really understand the tools you're trying to use. But like I said, this has always been a very cool project. My "good luck"…

[deleted]

Re: How and why GraphQL will influence the Sourcehut alpha

#24

Post by xrisk »

ianamartin wrote: post #15
ghoshbishakh wrote: post #6

Earlier quoted context omitted.

what do you feel the problem is?

There are lots of production sites that serve 10,000x as much traffic as sourcehut that are built on Python/flask/sqlalchemy serving RESTful APIs. If you can't make that combination work well, there's another place to look for problems besides your tool kit. You might need to ask yourself if you really understand the tools you're trying to use. But like I said, this has always been a very cool project. My "good luck"…

What are some production websites that run Python stacks? I’m curious.

On a tangential note: if anybody has blog posts on scaling Flask/SQLAlchemy or Django stacks I would appreciate it.

Re: How and why GraphQL will influence the Sourcehut alpha

#25

Post by philosopher1234 »

CameronNemo wrote: post #22
philosopher1234 wrote: post #19

Earlier quoted context omitted.

Could you name some specific rough edges and missing functionality that have frustrated you? I'm a big fan of Go and have a good experience with it, so I'd be very curious to hear from someone with an opposite experience.

On several occasions I have gotten panics (segfaults), many times without using CGo. This is not something I would expect from a "high level" language that has a garbage collector. The constant use of generated code is another real pain point, particularly when I am writing business logic that needs to operate on generated types for which there is no interface available that does what I need (and why would there be?…

>The constant use of generated code is another real pain point, particularly when I am writing business logic that needs to operate on generated types for which there is no interface available that does what I need (and why would there be? how would the library/codegen-tool author know all the permutations of business logic that might be out there?).

Not sure what you mean here. Is there a particular codegen tool you found lacking?

>The introspection capabilities (reflect package) are quite obtuse and error prone (basically requiring 100% test coverage because you lose nearly the entire type system), yet absolutely critical to implementing anything notably complex with Go.

Ah, the lack of generics. I haven't really written any particularly large projects, where have you had to use reflection when working with go in your projects?

I hope my questions don't come across as dismissive. I think the typical go response to your complaints are "it wont come up" (a YAGNI variant), so I'm always interested to hear about the cases where that argument fails.

Re: How and why GraphQL will influence the Sourcehut alpha

#26

Post by ianamartin »

CameronNemo wrote: post #8
ianamartin wrote: post #5

Well, that's too bad. I always thought this was a cool project. But if you can't dev your way into decent performance for a small alpha project using python/flask/sql, I don't think your tools are the problem. And I guarantee that a graphql isn't the solution. So, I mean, good luck.

This quote stuck out to me: >Today, the Python backends to the web services communicate directly with PostgreSQL via SQLAlchemy, but it is my intention to build out experimental replacement backends which are routed through GraphQL instead. This way, the much more performant and robust GraphQL backends become the single source of truth for all information in SourceHut. I wonder how adding a layer of indirection can s…

That quote is sort of exactly what's conceptually wrong with what's goin on in my opinion. Yes, I know, armchair quarterback and I'm not the one out there building stuff like this for free, etc., etc.

But claiming some nebulous backend that's more performant and robust than Postgres is like, WTF? Are you using an actual GraphDB like Neo4J? Are you putting a graph frontend on Postgres like PostGraphQL? None of the post really makes any sense because GraphQL is a Query Language, not a data store. What are the CAP theorem tradeoffs in the new backend? What does more robust mean? What does more performant mean? This is a source control app. Those tradeoffs are meaningful.

There seems to be a lot of conflation between API design and data store and core programming tools all mixed into a big post that mostly sounds to me like, "I don't get how to make this (extremely popular and well-known platform that drives many websites 10000x my size) work well, so I'm trying something different that sounds cool."

Which, again, the author has always said this is an experiment, and that's cool. But the conceptual confusion in the post makes me think that moving away from boring tools and trying new tools is not going to end up going well.

But this is a source control app, and it's hopefully backed up somewhere besides sourcehut so it should be fine if he needs to backtrack.

Re: How and why GraphQL will influence the Sourcehut alpha

#27

Post by searchableguy »

xrisk wrote: post #24
ianamartin wrote: post #15

Earlier quoted context omitted.

There are lots of production sites that serve 10,000x as much traffic as sourcehut that are built on Python/flask/sqlalchemy serving RESTful APIs. If you can't make that combination work well, there's another place to look for problems besides your tool kit. You might need to ask yourself if you really understand the tools you're trying to use. But like I said, this has always been a very cool project. My "good luck"…

What are some production websites that run Python stacks? I’m curious. On a tangential note: if anybody has blog posts on scaling Flask/SQLAlchemy or Django stacks I would appreciate it.

Dropbox is a notable one born here.

Re: How and why GraphQL will influence the Sourcehut alpha

#28

Post by alexchamberlain »

z3t4 wrote: post #3

Im a Sourcehut user and all I care about is that I can host my public Mercurial repositories, and that web frontend is somewhat usable (it currently is). I dont care what languages is ised to build it nor do i care about apis and other features. If i where to chose I would like an api that i could use with curl, eg. Dont complicate things.

I think this is simply a technical blog post - it could be about any site. It's simply an insight to the way another engineer is thinking, so you can challenge your own.

Re: How and why GraphQL will influence the Sourcehut alpha

#29

Post by ianamartin »

tmpz22 wrote: post #21

Big OOF. Even the flow of the article is muddied to the point where it takes significant effort to dig out the main points for OP's migration from REST to GraphQL. What I got from it: > The system would become more stable with the benefit of static typing, and more scalable with a faster and lighter-weight implementation. OK static typing I'm with you so far. Faster and lighter weight? I'm not so sure, sounds like yo…

This is a better way of saying what I've tried to say in like 3 comments.

Re: How and why GraphQL will influence the Sourcehut alpha

#30

Post by alexchamberlain »

xrisk wrote: post #24
ianamartin wrote: post #15

Earlier quoted context omitted.

There are lots of production sites that serve 10,000x as much traffic as sourcehut that are built on Python/flask/sqlalchemy serving RESTful APIs. If you can't make that combination work well, there's another place to look for problems besides your tool kit. You might need to ask yourself if you really understand the tools you're trying to use. But like I said, this has always been a very cool project. My "good luck"…

What are some production websites that run Python stacks? I’m curious. On a tangential note: if anybody has blog posts on scaling Flask/SQLAlchemy or Django stacks I would appreciate it.

Don't Instagram and YouTube still have large Python code bases?
Post reply on HN