Can someone explain what "ecological niche" this new Turso DB occupies in between SQLite and Postgres?
Beyond the SQLite single-writer limitation with concurrent writes
11–20 of 75 posts
Re: Beyond the SQLite single-writer limitation with concurrent writes
#12Don't forget that the SQLite team is working on their own multi-writer mode that blows BEGIN CONCURRENT' out of the water: https://news.ycombinator.com/item?id=34434025
Though this stuff moves slowly (that announcement was almost 3 years ago!), so I'm glad to see Turso giving us options today.
Re: Beyond the SQLite single-writer limitation with concurrent writes
#13Earlier quoted context omitted.
> Forking his project while using the name The don't use the name. They use Turso. Even the HN title is wrong - the article title doesn't mention SQLite. They refer to SQLite, but how could you not if that's what you forked from, and that's what has the functionality you're changing. That would be a very weird article if we didn't have that context.
They state, plainly on their home page: “The next evolution of SQLite” That is a material misrepresentation and absolutely trading on the SQLite name
“The next evolution of SQLite”
How? The whole point of trademark is to avoid confusing users that an alternate product is the same as the original product.
By explicitly saying "Next evolution of SQLite", or "A fork of SQLite", or a "Better SQLite" .. all of this is saying our product is distinct and different from SQLite.
If the fork were called "nue-sqlite" or "sqlitest" or "fastsqlite", there's an argument to be made.
Re: Beyond the SQLite single-writer limitation with concurrent writes
#14I’m not in a rush to use a reimplementation of SQLite — particularly from startup bros that had a very public, one-sided, and unpleasant fight with SQLite over their contribution model. D. Richard Hipp is a genuinely fantastic human being, and SQLite is a project developed at literally the level of planetary infrastructure given how broadly and everywhere it appears. Forking his project while using the name to garner…
I'd still give them the benefit of the doubt as most (all?) of the contributors are Finns who you can almost guarantee to have a "no bullshit" type of mentality to essentially everything. And what I'd guess, the team most probably has quite an academic background because of the local culture.
Re: Beyond the SQLite single-writer limitation with concurrent writes
#15Can someone explain what "ecological niche" this new Turso DB occupies in between SQLite and Postgres?
TursoDB aims to be fully compatible with sqlite, so files you create with tursodb can be read by sqlite and vice-versa. Like sqlite, turso is in-process, so it runs alongside your application, which is quite different from postgres' client-server architecture.
The only think turso has in common with postgres is mvcc, which is a rather standard concurrency control model across modern databases. Idk if I answered your question :)
Re: Beyond the SQLite single-writer limitation with concurrent writes
#16Don't forget that the SQLite team is working on their own multi-writer mode that blows BEGIN CONCURRENT' out of the water: https://news.ycombinator.com/item?id=34434025 Though this stuff moves slowly (that announcement was almost 3 years ago!), so I'm glad to see Turso giving us options today.
Looks like that branch is still under actively development - seven new commits this month: https://sqlite.org/hctree/timeline
Re: Beyond the SQLite single-writer limitation with concurrent writes
#17Earlier quoted context omitted.
They state, plainly on their home page: “The next evolution of SQLite” That is a material misrepresentation and absolutely trading on the SQLite name
“The next evolution of SQLite” How? The whole point of trademark is to avoid confusing users that an alternate product is the same as the original product. By explicitly saying "Next evolution of SQLite", or "A fork of SQLite", or a "Better SQLite" .. all of this is saying our product is distinct and different from SQLite. If the fork were called "nue-sqlite" or "sqlitest" or "fastsqlite", there's an argument to be m…
The issue isn’t that you’re mentioning SQLite or acknowledging that it’s a fork.
The problem is that the phrase “the next evolution of SQLite” conveys continuity and endorsement.
A reasonable reader could conclude that Turso is an official successor or the next release of SQLite — that it represents the official lineage of the project.
Phrasing like “SQLite-compatible,” or “a fork of SQLite” would be clear and factual.
Calling it “the next evolution of SQLite” isn’t factual; it’s marketing positioning, and it implies ownership of SQLite’s identity and lineage.
This reflects a broader pattern in how the fork has been presented publicly.
The messaging often treats the original project as something to be fixed rather than a foundation to be respected.
Referring to Turso as a product while leveraging the SQLite name reinforces that framing — co-opting a public-domain engineering gift into a commercial asset.
Re: Beyond the SQLite single-writer limitation with concurrent writes
#18Kind of cool to see work on this. I do hope that the final db file result is still binary compatible with SQLite 3 in whatever direction Turso moves towards though... Rust or not. I've been advocating with several projects over recent years to get SQLite3 as an archive/export/interchange format for data. Need to archive 2019 data from the database, dump it into a SQLite db with roughly the same schema... Need to pass…
> As a secondary, I wonder if it's possible to actively use a SQLite interface against a database file on S3, assuming a single server/instance is the actual active connection.
You could achieve this today using one of the many adapters that turn S3 into a file system, without needing to wait for any SQLite buy in.
Re: Beyond the SQLite single-writer limitation with concurrent writes
#19Kind of cool to see work on this. I do hope that the final db file result is still binary compatible with SQLite 3 in whatever direction Turso moves towards though... Rust or not. I've been advocating with several projects over recent years to get SQLite3 as an archive/export/interchange format for data. Need to archive 2019 data from the database, dump it into a SQLite db with roughly the same schema... Need to pass…
SQLite directly against S3 is workable if you mean querying a read-only database.
For example, from Go, you could use my driver, and point it to a database file stored in S3 using this: https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/readerv...
For read-write it's a terrible idea. Object storage assumes objects are immutable. There may be some support for appends, but modifying the middle of an object in place involves copying the entire thing.
What is on the verge of becoming viable is to use Litestream to do asynchronous replication to S3, and have read replicas that stream the data directly from S3. But what's stored in S3 isn't a database file, but a format created for the purpose called LTX.
Re: Beyond the SQLite single-writer limitation with concurrent writes
#20The single-writer limitation in SQLite is per-database, not per-connection. You can shard your SQLite tables into multiple database files and query across all of them from a single connection. I agree that "the single-writer limitation isn't just a theoretical concern", but it's also solvable without forking SQLite. ulimit's the limit! If your goal is resource maximization of a given computer, though, Postgres is lik…
Joins and Transactions are a pretty big part of SQL. I'm no expert, but if my quick search results are right, both are lost in the separate file per table scenario.