Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

trailbase.io

Post reply on HN
Print view View original

21–30 of 37 posts

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#21

Post by trailbase »

steve_adams_86 wrote: post #14

I love the roadmap ideas for this. I'm a heavy user of pocketbase and I'm very happy with it, but I'd be more than happy to see this type of solution become more common and address needs like multi-tenancy. I'm also stoked to see the focus on performance (though pocketbase does excellent in this regard as it is)

Thanks! Happy to hear more ideas, if there's something you've always wanted :). PocketBase is great (and as I just learned, it has recently gotten even faster). One of my main goals was to expose more of the raw SQLite goodness.

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#22

Post by trailbase »

bigiain wrote: post #12
trailbase wrote: post #10

Earlier quoted context omitted.

I may plea the fifth, i.e. I'm not sure what this entails but would love to hear more. Also feel free to file a feature request. Naively, I would argue that being idiomatic in the respective ecosystem is more important than perfect consistency. Only few users will likely use 2 or more languages and probably even then there's a balance to be struck.

Pretty sure you'll want to consider usage by teams writing in all of Swift, Java, and Javascript as relatively common. You're likely to encounter people doing "fringe" stuff like transpiling Go or Rust into Wasm, but those folk are all most likely capable of dealing with idiomatic impedance mismatches themselves.

Absolutely. In terms of new clients targeting mobile teams Swift and Java/Kotlin would make the most sense. I mostly meant that even folks who code in Swift, Java and JS, would likely prefer the respective clients to be idiomatic than the APIs exactly matching each other.

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#23

Post by trailbase »

Guillaume86 wrote: post #13
trailbase wrote: post #10

Earlier quoted context omitted.

I may plea the fifth, i.e. I'm not sure what this entails but would love to hear more. Also feel free to file a feature request. Naively, I would argue that being idiomatic in the respective ecosystem is more important than perfect consistency. Only few users will likely use 2 or more languages and probably even then there's a balance to be struck.

I agree I would generally prefer clients to be idiomatic for the target language but in the case of LINQ providers I understand not doing them since they are quite involved to implement. You can probably leverage something like https://linq2db.github.io/ and replace some parts of their SQLite provider with HTTP calls to avoid coding most complicated parts. This would be for the raw sql API, for the records API I'm no…

The raw query API is an admin-only API at least for now, since SQLite doesn't have a built-in concept of row-level security. From a client's perspective it's only RESTful APIs at the moment.

That said, you absolutely can generate code from the DB schema already. There are `/examples` (just none in C# yet).

I'm optimistic that C# will become increasingly important with respect to cross-platform mobile/desktop development and thus receive higher priority treatment in general.

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#24

Post by trailbase »

trailbase wrote: post #11
trailbase wrote: post #9

Earlier quoted context omitted.

- It is nice to see more backends utilizing SQLite. Hey thanks for chiming it. Huge fan of PocketBase, has been a major inspiration :applause:. For anyone driving by, certainly a more mature product. - Based on your benchmarks repo it looks like that the tests were done against PocketBase You're right. I did run v0.22.21, which simply was current when I ran the benchmarks first. I absolutely will add the information,…

I'm happy to report that v0.25.0 already w/o CGO driver (fighting it right now) and GOOS=linux CGO_ENABLED=0 GOAMD64=v4 improved about 35% from 61.7s per 100k inserts to about 40s :clap: I still wanna get the mattn/go-sqlite3 driver to work and it's getting a bit late here for writing coherent text... I'll update the benchmarks ASAP

Back from the dead. I also now vaguely remember that there was some hiatus (probably between v0.22 and v0.23) where you took some time to rewrite the DB layer to enable custom DB drivers.

Anyway, w/o fog I managed to run with mattn/go-sqlite3. I'm not sure this is expected but it didn't seem to make much of a difference with my setup (For transparency, I do recently have some issues with repeatability likely due to btrfs). I'm certainly not seeing 4x but around 30-35%, which is still very impressive!

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#25

Post by oulipo »

trailbase wrote: post #19
oulipo wrote: post #17

So all the backend API is implemented using "ACL rules" in a list, and then directly on the client, like "client-side Firebase" right? I feel I prefer to have a locked-down database, and implement everything "backend-side" with a kind of "admin API" which has access to everything, and checks user roles in the backend, it feels cleaner to me, is that also possible?

IIUC, what you're asking for is what it is, i.e. TrailBase is like FireBase, as opposed to a FireBase running entirely on the client. We probably both agree that ACLs enforced by the client are no protection at all. Did I misunderstand? Was there something that thew you off? - always keen to improve

No I meant, is it "just like Firebase (ACLs on the backend) but simpler"?

I understand ACLs on the backend and API queries on the client, I just don't find it that practical to use

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#26

Post by randomwebdev »

trailbase wrote: post #24
trailbase wrote: post #11

Earlier quoted context omitted.

I'm happy to report that v0.25.0 already w/o CGO driver (fighting it right now) and GOOS=linux CGO_ENABLED=0 GOAMD64=v4 improved about 35% from 61.7s per 100k inserts to about 40s :clap: I still wanna get the mattn/go-sqlite3 driver to work and it's getting a bit late here for writing coherent text... I'll update the benchmarks ASAP

Back from the dead. I also now vaguely remember that there was some hiatus (probably between v0.22 and v0.23) where you took some time to rewrite the DB layer to enable custom DB drivers. Anyway, w/o fog I managed to run with mattn/go-sqlite3. I'm not sure this is expected but it didn't seem to make much of a difference with my setup (For transparency, I do recently have some issues with repeatability likely due to b…

> I'm not sure this is expected but it didn't seem to make much of a difference with my setup

I expect it to be faster but it is also possible that maybe in your specific collection and execution scenario it somehow perform worst, or at least not the same as in my benchmarks, I'm not sure.

The ~4x mentioned speed up is based on the tests for Hetzner CAX41 with the CGO driver when creating 50k records with 500 concurrent clients:

Old (58.409064001s): https://github.com/pocketbase/benchmarks/blob/54140be5fb0102...

New (13.580098262s): https://github.com/pocketbase/benchmarks/blob/7df0466ac9bd62...

---

Edit: Based on your benchmarks repo (https://github.com/trailbaseio/trailbase-benchmark/tree/main...) note that compiling PocketBase with `CGO_ENABLED=1` is no longer enough to kick in the github.com/mattn/go-sqlite3 driver and you need to enable it manually as shown in the db_cgo.go in our benchmarks OR like in https://pocketbase.io/docs/go-overview/#github-commattngo-sq... (the reason for this change was to avoid the "multiple definition" linker errors).

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#27

Post by bmacho »

trailbase wrote: post #18
bmacho wrote: post #16

That "JavaScript Performance" diagram with the vertical "CPU cores" axis is so weird. I am not sure if it is a mistake, it probably is.

You might be thinking JavaScript runs on a single threaded event loop? That's correct, however you can run N event loops in parallel (isolates in v8 lingo). `deno serve` even has a `--parallel` flag ( https://docs.deno.com/runtime/reference/cli/serve/ ). It would have certainly been simpler to just plot the overall runtime (width of the graph), I did think that it was quite interesting that PB's goja integration take…

Oops, I was completely misinterpreting the chart, but I am cool now. I retract the previous comment, it looks okay.

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#28

Post by trailbase »

randomwebdev wrote: post #26
trailbase wrote: post #24

Earlier quoted context omitted.

Back from the dead. I also now vaguely remember that there was some hiatus (probably between v0.22 and v0.23) where you took some time to rewrite the DB layer to enable custom DB drivers. Anyway, w/o fog I managed to run with mattn/go-sqlite3. I'm not sure this is expected but it didn't seem to make much of a difference with my setup (For transparency, I do recently have some issues with repeatability likely due to b…

> I'm not sure this is expected but it didn't seem to make much of a difference with my setup I expect it to be faster but it is also possible that maybe in your specific collection and execution scenario it somehow perform worst, or at least not the same as in my benchmarks, I'm not sure. The ~4x mentioned speed up is based on the tests for Hetzner CAX41 with the CGO driver when creating 50k records with 500 concurr…

> Edit: Based on your benchmarks repo (https://github.com/trailbaseio/trailbase-benchmark/tree/main...) note that compiling PocketBase with `CGO_ENABLED=1` is no longer enough to kick in the github.com/mattn/go-sqlite3 driver and you need to enable it manually as shown in the db_cgo.go in our benchmarks OR like in https://pocketbase.io/docs/go-overview/#github-commattngo-sq... (the reason for this change was to avoid the "multiple definition" linker errors).

FWIW, I did this locally (also happy to fork PB and check it in, certainly aids transparency). I've already updated the numbers in the benchmark doc but will continue to try squeeze more out of it.

As to 4x, my concurrency levels are significantly lower which could certainly explain it. Is PB juggling multiple connections increasing write lock congestion or are you serializing access, e.g. via a worker thread? Also happy to chat more (feel free to send me an email), I certainly want PB to have the best possible representation and maybe we can even speed things up on both sides

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#29

Post by randomwebdev »

trailbase wrote: post #28
randomwebdev wrote: post #26

Earlier quoted context omitted.

> I'm not sure this is expected but it didn't seem to make much of a difference with my setup I expect it to be faster but it is also possible that maybe in your specific collection and execution scenario it somehow perform worst, or at least not the same as in my benchmarks, I'm not sure. The ~4x mentioned speed up is based on the tests for Hetzner CAX41 with the CGO driver when creating 50k records with 500 concurr…

> Edit: Based on your benchmarks repo ( https://github.com/trailbaseio/trailbase-benchmark/tree/main ...) note that compiling PocketBase with `CGO_ENABLED=1` is no longer enough to kick in the github.com/mattn/go-sqlite3 driver and you need to enable it manually as shown in the db_cgo.go in our benchmarks OR like in https://pocketbase.io/docs/go-overview/#github-commattngo-sq ... (the reason for this change was to av…

> As to 4x, my concurrency levels are significantly lower which could certainly explain it.

Yes I noticed that too so that it is very likely the reason.

In all cases my initial comment wasn't intended to "dispute", argue or anything like that, so please don't feel obligated to waste time updating the benchmarks. They are valid as they are!

My initial comment was more of a note/suggestion to simply list the used versions of the tested platforms (not just for PocketBase) because often they change over time and the shown results could be misleading if someone stumble on the article 2 years later for example.

Re: TrailBase: Sub-millisecond open-source application base with Rust, SQLite and V8

#30

Post by trailbase »

oulipo wrote: post #25
trailbase wrote: post #19

Earlier quoted context omitted.

IIUC, what you're asking for is what it is, i.e. TrailBase is like FireBase, as opposed to a FireBase running entirely on the client. We probably both agree that ACLs enforced by the client are no protection at all. Did I misunderstand? Was there something that thew you off? - always keen to improve

No I meant, is it "just like Firebase (ACLs on the backend) but simpler"? I understand ACLs on the backend and API queries on the client, I just don't find it that practical to use

Could you expand a bit on, how

> I feel I prefer to have a locked-down database, and implement everything "backend-side" with a kind of "admin API" which has access to everything, and checks user roles in the backend, it feels cleaner to me, is that also possible?

is different from what FireBase or TrailBase does?

Are you saying that you'd prefer to run your own backend binary (as opposed to running in an integrated runtime), do your own ACL checking, and have more of a free-form SQL-like API with the DB layer?

Post reply on HN