Live data from Hacker News

How We Went All In on sqlc/pgx for Postgres and Go

brandur.org

131–140 of 160 posts

Re: How We Went All In on sqlc/pgx for Postgres and Go

#131
post #24

Earlier quoted context omitted.

A few years ago I had spent a year working with a Go project that made heavy use of one of the (then) popular Go ORMs. Learned my lesson, never again. Magic=Bad.

What ORM was that?

Likely Gorm, I'm using that at the moment and it's eeehhhh.

Re: How We Went All In on sqlc/pgx for Postgres and Go

#132
post #29
post #12

From the article: > I’ve largely covered sqlc’s objective benefits and features, but more subjectively, it just feels good and fast to work with. Like Go itself, the tool’s working for you instead of against you, and giving you an easy way to get work done without wrestling with the computer all day. I've been meaning to write a blog post about sqlc myself, and when I get to it, I'll probably quote this line. sqlc is…

> Why would someone author an ORM, painstakingly creating Go functions that just map to existing SQL features? The answer to this question lies in the assumption you make in this statement: > the one which every engineer already knows: SQL. Not every engineer knows, or wants to learn, SQL. I've met very competent engineers, SMEs over their particular system, who were flummoxed by SQL. And many more just want to work…

ORMs often introduce their own flavor of Domain-Specific Language, or their own language (Entities instead of e.g. rows). I'd posit that learning an ORM is just as much work as learning SQL, but with an extra layer of indirection.

It's like learning HTML via React.

Re: How We Went All In on sqlc/pgx for Postgres and Go

#133

I think I'm missing something, but I don't get sqlc. Let's say I want to "get a list of authors", so in sqlc I would write: -- name: ListAuthors :many SELECT * FROM authors ORDER BY name; and in Go I can then say `authors, err := queries.ListAuthors(ctx)`. This is cool. Now, if I want to "get a list of American authors" I would write: -- name: ListAuthorsByNationality :many SELECT * FROM authors WHERE nationality = $…

> without having to write manually the N potential sql queries that the above code may represent.

The article provides an alternative by using conditionals inside of the SQL, but honestly it's not an improvement.

Re: How We Went All In on sqlc/pgx for Postgres and Go

#134

>ORMs also have the problem of being an impedance mismatch compared to the raw SQL most people are used to, meaning you’ve got the reference documentation open all day looking up how to do accomplish things when the equivalent SQL would’ve been automatic. Easier queries are pretty straightforward, but imagine if you want to add an upsert or a CTE. How does this make sense? Most ORMs will give yo a way to execute raw…

Well yeah, but one of the motivations of using an ORM is that you don't have to write (database engine specific) SQL. I mean the database agnosticism is generally speaking not an issue, but still.

Re: How We Went All In on sqlc/pgx for Postgres and Go

#135
post #57

Earlier quoted context omitted.

> Working with SQL in X (any language) usually has a poor developer experience that is why ORM or query builders are popular. At least when it comes to Postgres, I don't understand why more developers don't create their own user-defined functions with PL/pgSQL. It's very a robust and powerful procedural language. In my opinion, ORM's like SQLAlchemy add a completely unnecessary layer of abstraction. ORMs might be con…

I did this on a recent project, and it worked really well. I had each function definition in its own .sql file, with a preceding "drop function" call, and a Makefile clause to run them all. Which meant managing versions was easy (coupled with migration .sql files). I also got to find out if any of my SQL was broken right up front, and testing the SQL was simple - call the function and check the return. I also defined…

> I had each function definition in its own .sql file, with a preceding "drop function" call, and a Makefile clause to run them all.

why drop instead of CREATE OR REPLACE ?

Re: How We Went All In on sqlc/pgx for Postgres and Go

#136
post #27

I agree whole-heartedly that writing SQL feels right. Broadly speaking, you can take the following approaches to mapping database queries to Go code: - Write SQL queries, parse the SQL, generate Go from the queries (sqlc, pggen). - Write SQL schema files, parse the SQL schema, generate active records based on the tables (gorm) - Write Go structs, generate SQL schema from the structs, and use a custom query DSL (prote…

- generate code from running database and use a type-safe query builder (https://github.com/bokwoon95/go-structured-query)

Re: How We Went All In on sqlc/pgx for Postgres and Go

#137
post #96

I'm a big fan of the database first code generator approach to talking to an SQL database, so much so that I wrote pggen[1] (not to be confused with pggen[2], as far as I can tell a sqlc fork, which I just recently learned about). I'm a really big partisan of this approach, but I think I'd like to play the devil's advocate here and lay out some of the weaknesses of both a database first approach in general and sqlc i…

Really appreciate the in-depth comment. Also, thanks for open-sourcing pggen. I love have more projects in this space. I'll have to take a look at how it works. I did want to address the last point in your comment. > This works great until you have a query like `SELECT * FROM foos WHERE field = $1` When sqlc sees a query with a *, it rewrites the query in the generated code to have explicit column references. For exa…

Thanks for the correction! When I read the generated code, I just focused on the scan call. Since pggen doesn't parse the SQL, there is no great way to detect `SELECT ` and rewrite the query, so I hadn't considered this as a possibility. Parsing the SQL really opens up a lot of cool possibilities for you.

You already know this but in case anyone else is reading, another super cool thing that sqlc can do is infer good names for query arguments in go code by looking at what they are compared to in the SQL code. Thus for a query like `SELECT FROM foos WHERE created_at > $1`, the generated go wrapper would have a `createdAt` arg instead of having it be named something like `arg1`. Since opendoor/pggen doesn't parse the SQL, you need to explicitly override the argument names if you want to provide better names. Of course the names won't be perfect with sqlc's approach, but they will be better than `arg1` and it's still a very cool detail. It might not be obvious how neat this is if you haven't had to implement it, which is why I mention it.

Re: How We Went All In on sqlc/pgx for Postgres and Go

#138
post #96

I'm a big fan of the database first code generator approach to talking to an SQL database, so much so that I wrote pggen[1] (not to be confused with pggen[2], as far as I can tell a sqlc fork, which I just recently learned about). I'm a really big partisan of this approach, but I think I'd like to play the devil's advocate here and lay out some of the weaknesses of both a database first approach in general and sqlc i…

Really appreciate the in-depth comment. Also, thanks for open-sourcing pggen. I love have more projects in this space. I'll have to take a look at how it works. I did want to address the last point in your comment. > This works great until you have a query like `SELECT * FROM foos WHERE field = $1` When sqlc sees a query with a *, it rewrites the query in the generated code to have explicit column references. For exa…

Oh, another issue worth mentioning specifically when you are using the jackc/pgx driver is that pgx maintains a prepared statement cache under the hood for performance reasons, but the statements in the cache can be invalidated by query migrations. So if you do `SELECT * FROM foos WHERE field = $1` and then run `ALTER TABLE foos DROP COLUMN bar`, the next time you run that query you will get an error. This used to just be broken until you restarted the process to clear the cache, but I've patched pgx to make it invalidate the cached prepared statement when it sees one of these errors (it still returns the error though since jackc thought a retry would be too complicated). I added an automatic retry to opendoor/pggen to make it so users don't need to worry about this kind of low level detail. You may want to consider that for sqlc as well. The other option is to advise users to disable the cache by setting it's size to zero (I forget the exact config option, but jackc said there is one when I first brought up the issue).

Re: How We Went All In on sqlc/pgx for Postgres and Go

#140
i have been working with ORM and plain SQL for the past 10 years in a bunch of languages and libraries (php, java, javascript, Go). The issue i have with ORM and other libraries that supposedly reduce the work for you is that it's black magic. You will encounter yourself one day having to dig into the source code of the library to tackle nasty bugs or add new features. It's exhausting. When I started using Go I mostly used plain SQL queries. I took on the manual endeavour to map and hydrate my objects. Sure, it's more manual work, but abstractions have a cost too. That bill might have to be paid one day. One way or the other. Personally, I am never looking back. But every one of us has has a different use case. Therefore ymmv
Post reply on HN