Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLx – Rust SQL Toolkit

github.com

Post reply on HN
Print view View original

91–100 of 101 posts

Re: SQLx – Rust SQL Toolkit

#91

Post by tracker1 »

banashark wrote: post #52

One thing I don't usually see addressed with the pure-sql approaches is how to handle dynamic query building. The most common example being large configurable forms that display a data grid. Kysely[1] does a good job of starting from this angle, but allowing something like specifying the concrete deserialization type similar to the libraries here. I'm a big fan of sql in general (even if the syntax can be verbose, th…

Not rust, but I've been a pretty big fan of Dapper and Dapper.SqlBuilder in the C# space... have used it with MS-SQL and PostgreSQL very effectively, even with really complex query construction against input options.

https://github.com/DapperLib/Dapper/blob/main/Dapper.SqlBuil...

Re: SQLx – Rust SQL Toolkit

#92

Post by LtdJorge »

germandiago wrote: post #83

I am not much into Rust ATM. I am quite comfortable with C++. So here it goes my question: I use sqlpp11 in C++. I generate code and I can use it with strong typing by including some headers. This Rust crate seems to provide compile-time checking. But it will give me code-completion? It is very nice that by pressing '.' you know what you potentially have.

It depends. On RustRover you do, because the query text can be language-injected as SQL, and it uses your configured schema.

Re: SQLx – Rust SQL Toolkit

#93

Post by LtdJorge »

valenterry wrote: post #72

Just coming here to make a prediction: using raw SQL is not great for anything but very simple cases. You can make it type-safe, but that becomes tricky once things become dynamic. But the real problem is ergonomy. The better solution in almost any language is to leverage the syntax of your language to allow for as much (non-macro) type-safety and auto-completion as possible. For example instead of: SELECT country, C…

I agree and prefer Diesel to SQLx, but I do use both.

Re: SQLx – Rust SQL Toolkit

#94

Post by frollogaston »

bitbasher wrote: post #74
frollogaston wrote: post #27

Earlier quoted context omitted.

How is it more LoC in Go, just cause of the "if err" stuff?

Go's verbose error handling certainly impacted the vertical height of files (lots of early returns), but wasn't a big contributor to overall LoC. The more serious LoC offenders in Go were: 1. Marshalling/Unmarshalling code (for API responses, to/from external services, etc). In general, working with JSON in Go was painful and error prone. Rust's serde made this a complete non-issue. 2. Repetitive sql query code (quer…

Hm. I've used Rust a lot more than Go, so this is secondhand to me. I know that generics are iffy and nullness is annoying. If you're paying for static types in Go and still not getting the guarantees, that really bites.

Re: SQLx – Rust SQL Toolkit

#95

Post by conroy »

jchw wrote: post #12
bitbasher wrote: post #8

I've been using sqlx with postgres for several months now on a production server with decent query volume all day long. It has been rock solid. I find writing sql in rust with sqlx to be far fewer lines of code than the same in Go. This server was ported from Go and the end result was ~40% fewer lines of code, less memory usage and stable cpu/memory usage over time.

Speaking of Go, if you want compile-time type checking like what SQLx offers, the Go ecosystem has an option that is arguably even better at it: https://sqlc.dev/ It has the advantage that it implements the parsing and type checking logic in pure Go, allowing it to import your migrations and infer the schema for type checking. With SQLx you need to have your database engine running at compile time during the proc mac…

Maintainer of sqlc here. Thanks for the kind words! I'm considering switching to the sqlx model of talking to a running database simply because trying to re-implement PostgreSQL internals has been a huge challenge. It works for most queries, but for the long tail of features, it's a losing battle.

Can you tell me why it's a non-starter for you?

Re: SQLx – Rust SQL Toolkit

#96

Post by jchw »

conroy wrote: post #95
jchw wrote: post #12

Earlier quoted context omitted.

Speaking of Go, if you want compile-time type checking like what SQLx offers, the Go ecosystem has an option that is arguably even better at it: https://sqlc.dev/ It has the advantage that it implements the parsing and type checking logic in pure Go, allowing it to import your migrations and infer the schema for type checking. With SQLx you need to have your database engine running at compile time during the proc mac…

Maintainer of sqlc here. Thanks for the kind words! I'm considering switching to the sqlx model of talking to a running database simply because trying to re-implement PostgreSQL internals has been a huge challenge. It works for most queries, but for the long tail of features, it's a losing battle. Can you tell me why it's a non-starter for you?

I think it's only a non-starter for me in SQLx if not using query caching. Caching makes the situation workable.

For sqlc, it isn't really a big problem because you only need to run the code generation when you're actually modifying database things. Still, with that having been said, I think just passing a database URI and having analysis work based on that is unideal. Using an actual database isn't a huge problem, but having to manage the database instance out of band is the part that I think isn't great, because it allows for the schema in the code to trivially desync with the schema used in analysis. If I used SQLx I'd probably be compelled to try to wire up a solution that spawns the database and migrates it up hermetically for the caching part. Likewise if I used this mode of sqlc.

I guess it might be possible for sqlc to add first class support for that sort of concept, but I can see holes in it. For one thing, you have to figure out where to grab binaries from and what version. An approach using Docker/Podman works, and at least partly solves this problem because you could allow specifying any OCI image, but that has caveats too, like requiring Docker or Podman to be installed. The most heroic effort would be to use some kind of solution using WASM builds of database engines: pulling down and running something like PGlite in process seems like it would be an almost ideal solution, but it sticks you to whatever things can actually be made to work in WASM in terms of features, extensions and versions, at least unless/until database servers and extension vendors miraculously decide that supporting WASM as a target is a good idea. Still, if you want some crazy ideas for how to make the UX better, I think either the Docker approach or the WASM approach could be made to work to some degree.

Barring that, though, I'd be most likely to have some kind of Docker setup for running sqlc with an ephemeral database instance. It's not pretty, but it works...

I don't think it would be a non-starter, though. I only really think that connecting to the database from within rustc invocations is a non-starter.

Re: SQLx – Rust SQL Toolkit

#97

Post by devjab »

written-beyond wrote: post #86
bbkane wrote: post #44

Earlier quoted context omitted.

Maybe I'm drinking the sqlc Kool aid, but because I'm already using migration files, setting up the config to point to them and a folder of SQL queries was pretty painless. And of course now that I have it, the incremental cost of adding a new query is really low as well

That's all understandable. But like I said I did spend 2 weeks working with SQLc, however when I compared it to just writing the query in my code, the developer experience was miles apart. You could compare it to people writing CSS, JavaScript and Markup in separate files Vs having just one file in React/Svelte etc. which gives the user the option to combine everything into one. There maybe a lot of drawbacks from th…

We're into Go and SQLC by extension because we write systems with 0 dependencies outside of the standard library. Which is a security and compliance thing.

As far as building something fast, I'm with you. I always reach out for Python with UV, Litestar and Advanced Alchemy when I want to build personal web projects. I don't think SQLC is bad as such, once you've written your SQL you can essentially compile that into a CRUD application which is ready to go. As you've pointed out, however, you'd need to slam something like a GraphQL engine on top of it if you wanted rich quries easily, and you'd still not have the auto-generated OpenAPI that comes with Python web frameworks.

SQLC is for code where you want a low amount (or zero) external depedencies. Which is a very "Go" thing to want. It does scale well, but that requires you to build various CLI tools to help maintain things as well as your own Go modules to add "quality of life" like dynamic routers and get queries for low traffic requests.

I'll try SQLx eventually when I get time to look more into Rust.

Re: SQLx – Rust SQL Toolkit

#98

Post by banashark »

rzmmm wrote: post #68
banashark wrote: post #52

One thing I don't usually see addressed with the pure-sql approaches is how to handle dynamic query building. The most common example being large configurable forms that display a data grid. Kysely[1] does a good job of starting from this angle, but allowing something like specifying the concrete deserialization type similar to the libraries here. I'm a big fan of sql in general (even if the syntax can be verbose, th…

One approach is to create views for the required data and then just select the columns which are needed. The joins will be pruned by the query planner if they are not needed, so there is no need for conditional joins.

Yeah this definitely makes sense, and is good database API design as well.

Re: SQLx – Rust SQL Toolkit

#99

Post by ringeryless »

jchw wrote: post #48
jjice wrote: post #45

Earlier quoted context omitted.

It's absolutely core to SQLx. I'm surprised to hear that that isn't widely known based on the parent. The first time I used SQLx has to be 4 or 5 years ago and they had it back then.

Well, it hurts that it isn't the default. The README still tells you to set the environment variable, it just isn't the "default" way to do things. In my opinion it would be better to entirely remove support for connecting to the database during compilation. Does anyone actually want to use it that way? Comparing and contrasting, sqlc type checking happens via code generation, basically the only option in Go since th…

"default"? they are explicitly different macros with different syntax. use whichever you prefer.

Re: SQLx – Rust SQL Toolkit

#100

Post by moggers123 »

pie_flavor wrote: post #75
moggers123 wrote: post #41

SQLx is great, but I really wish they had a non-async interface. I had to switch a project from sqlx to rusqlite seemingly just due to the overhead of the async machinery. Saw a 20x latency reduction that I narrowed down to "probably async" (sort of hard to tell, I find it very difficult to do perf analysis of async code). I try to avoid discussing async so as to not come off as a frothing-at-the-mouth-chest-thumping…

Async does not incur 20x slowdowns when you're I/O bound. It would be ridiculous for copying a few bytes to be slower than a syscall. This sounds like mutex issues, or WAL config, or something like that.

I just chucked something together to try and demonstrate. I don't see the massive 20x slowdown, only about a 3x slowdown (5x on release build). Still enough to be painful for the use case in question.

https://github.com/Moggers/rusqlitebenchmark

Do you think you could look through it and point out what you think the reason is? I think they've both got the same WAL and mutex settings. Its a very contrived and synthetic example but actually somewhat representative of what the original code wanted to do.

Post reply on HN