Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

trailbase.io

Post reply on HN
Print view View original

11–20 of 37 posts

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#11

Post by trailbase »

trailbase wrote: post #9
randomwebdev wrote: post #7

(Disclaimer: I'm PocketBase author) It is nice to see more backends utilizing SQLite. The benchmarks and the Comparisions section also seem well done. Just a nitpick - list the versions of the tested platforms. Based on your benchmarks repo it looks like that the tests were done against PocketBase [0]: https://github.com/pocketbase/benchmarks [1]: https://pocketbase.io/docs/go-overview/#custom-sqlite-driver

- It is nice to see more backends utilizing SQLite. Hey thanks for chiming it. Huge fan of PocketBase, has been a major inspiration :applause:. For anyone driving by, certainly a more mature product. - Based on your benchmarks repo it looks like that the tests were done against PocketBase You're right. I did run v0.22.21, which simply was current when I ran the benchmarks first. I absolutely will add the information,…

I'm happy to report that v0.25.0 already w/o CGO driver (fighting it right now) and GOOS=linux CGO_ENABLED=0 GOAMD64=v4 improved about 35% from 61.7s per 100k inserts to about 40s :clap:

I still wanna get the mattn/go-sqlite3 driver to work and it's getting a bit late here for writing coherent text... I'll update the benchmarks ASAP

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#12

Post by bigiain »

trailbase wrote: post #10
Guillaume86 wrote: post #8

Looks interesting, might play with it and the C# client soon, do you plan to build a LINQ provider eventually or keep the APIs similar across client languages?

I may plea the fifth, i.e. I'm not sure what this entails but would love to hear more. Also feel free to file a feature request. Naively, I would argue that being idiomatic in the respective ecosystem is more important than perfect consistency. Only few users will likely use 2 or more languages and probably even then there's a balance to be struck.

Pretty sure you'll want to consider usage by teams writing in all of Swift, Java, and Javascript as relatively common.

You're likely to encounter people doing "fringe" stuff like transpiling Go or Rust into Wasm, but those folk are all most likely capable of dealing with idiomatic impedance mismatches themselves.

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#13

Post by Guillaume86 »

trailbase wrote: post #10
Guillaume86 wrote: post #8

Looks interesting, might play with it and the C# client soon, do you plan to build a LINQ provider eventually or keep the APIs similar across client languages?

I may plea the fifth, i.e. I'm not sure what this entails but would love to hear more. Also feel free to file a feature request. Naively, I would argue that being idiomatic in the respective ecosystem is more important than perfect consistency. Only few users will likely use 2 or more languages and probably even then there's a balance to be struck.

I agree I would generally prefer clients to be idiomatic for the target language but in the case of LINQ providers I understand not doing them since they are quite involved to implement.

You can probably leverage something like https://linq2db.github.io/ and replace some parts of their SQLite provider with HTTP calls to avoid coding most complicated parts. This would be for the raw sql API, for the records API I'm not sure LINQ makes sense since from what I saw in the docs it would be a pretty small subset of LINQ anyway. An easy way to generate the model classes from the DB schema would be nice too (a source generator accessing the schema via some endpoint?).

Edit: it's nice seeing C# getting some love early in a project, I'm used to fallback to js/ts to try the new stuff :)

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#14

Post by steve_adams_86 »

I love the roadmap ideas for this. I'm a heavy user of pocketbase and I'm very happy with it, but I'd be more than happy to see this type of solution become more common and address needs like multi-tenancy. I'm also stoked to see the focus on performance (though pocketbase does excellent in this regard as it is)

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#15

Post by laurencerowe »

Given the underlying SQLite calls are sync why is the query API async?

From my own experiments with Node and SQLite I've found synchronous sqlite libraries like https://www.npmjs.com/package/better-sqlite3 substantially faster, especially when making many simple queries (a pattern encouraged by SQLite.)

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#17

Post by oulipo »

So all the backend API is implemented using "ACL rules" in a list, and then directly on the client, like "client-side Firebase" right?

I feel I prefer to have a locked-down database, and implement everything "backend-side" with a kind of "admin API" which has access to everything, and checks user roles in the backend, it feels cleaner to me, is that also possible?

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#18

Post by trailbase »

bmacho wrote: post #16

That "JavaScript Performance" diagram with the vertical "CPU cores" axis is so weird. I am not sure if it is a mistake, it probably is.

You might be thinking JavaScript runs on a single threaded event loop? That's correct, however you can run N event loops in parallel (isolates in v8 lingo). `deno serve` even has a `--parallel` flag (https://docs.deno.com/runtime/reference/cli/serve/).

It would have certainly been simpler to just plot the overall runtime (width of the graph), I did think that it was quite interesting that PB's goja integration takes a while before utilizing the entire machine.

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#19

Post by trailbase »

oulipo wrote: post #17

So all the backend API is implemented using "ACL rules" in a list, and then directly on the client, like "client-side Firebase" right? I feel I prefer to have a locked-down database, and implement everything "backend-side" with a kind of "admin API" which has access to everything, and checks user roles in the backend, it feels cleaner to me, is that also possible?

IIUC, what you're asking for is what it is, i.e. TrailBase is like FireBase, as opposed to a FireBase running entirely on the client. We probably both agree that ACLs enforced by the client are no protection at all.

Did I misunderstand? Was there something that thew you off? - always keen to improve

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#20

Post by trailbase »

laurencerowe wrote: post #15

Given the underlying SQLite calls are sync why is the query API async? From my own experiments with Node and SQLite I've found synchronous sqlite libraries like https://www.npmjs.com/package/better-sqlite3 substantially faster, especially when making many simple queries (a pattern encouraged by SQLite.)

I'm not sure if you're referencing the client libs or the server-side v8 integration. Either way, both are async. The client is async because there's network in between. And the server-side v8 integration is async to schedule execution on a dedicated SQLite event loop.

What you're saying makes a lot of sense. SQLite is sync and if you're program is alone accessing SQLite doing a single task, going sync is the way. If you're doing a lot of parallel work, both your JS event loop interleaving many tasks and several event loops accessing SQLite in parallel you have to make trade-offs. Specifically, `conn.query` may block for a long time w/o doing any work. Depending on your use-case it may or may not be ok to block the event-loop that entire period. TrailBase's setup is optimized to maximize throughput under highly concurrent loads, rather than minimizing latency in single-threaded workloads. That's not to say, TrailBase isn't quick. It's pretty low-latency even under load. However, if that's all you're after you're probably better off with better-sqlite3 or dropping down to C :).

Does that make sense?

Post reply on HN