Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beyond the SQLite single-writer limitation with concurrent writes

turso.tech

Post reply on HN
Print view View original

31–40 of 75 posts

Re: Beyond the SQLite single-writer limitation with concurrent writes

#31

Post by rednafi »

glommer wrote: post #23
frumplestlatz wrote: post #3

I’m not in a rush to use a reimplementation of SQLite — particularly from startup bros that had a very public, one-sided, and unpleasant fight with SQLite over their contribution model. D. Richard Hipp is a genuinely fantastic human being, and SQLite is a project developed at literally the level of planetary infrastructure given how broadly and everywhere it appears. Forking his project while using the name to garner…

Author of Turso here. Couple of points * You are right not to rush. You should keep using SQLite until Turso matures. Some use cases are more tolerant to new tech than others. It will take time for us to reach the level of trust SQLite has for broad use cases, but we are hoping to add value for some use cases right away. Never rush, tech matters! * I have never met Hipp, but only heard great things about him. * We ne…

Awesome to see you chiming in here.

Just to set some context, I remember talking to Glauber and Pekka last year, right around when the Turso folks started toying with the idea of reimplementing SQLite in Rust.

It’s a moonshot, but if anyone can pull it off, it’s these people. If you don’t know their background, checkout ScyllaDB. It never ceases to amaze me how belligerent some HN folks can be - they won’t even take five minutes to do a Google search before calling someone a nobody or “SV tech bro.” Not that I’m saying you should do that to anyone.

Re: Beyond the SQLite single-writer limitation with concurrent writes

#32

Post by glommer »

frumplestlatz wrote: post #28
glommer wrote: post #23

Earlier quoted context omitted.

Author of Turso here. Couple of points * You are right not to rush. You should keep using SQLite until Turso matures. Some use cases are more tolerant to new tech than others. It will take time for us to reach the level of trust SQLite has for broad use cases, but we are hoping to add value for some use cases right away. Never rush, tech matters! * I have never met Hipp, but only heard great things about him. * We ne…

I wasn’t describing a legal dispute, and I haven’t made any claim about infringement. “Material misrepresentation” refers to the substance of the public messaging, not to a legal violation. The issue I raised is that the phrasing and the way the fork has been presented create the impression of continuity and endorsement that doesn’t exist. That’s a reputational and ethical concern, not a legal debate. Calling it “the…

I understood your criticism. I disagree with it, and I think we are not misrepresenting anything, as we make it very clear that we are not SQLite, we are just a reimplementation that goes beyond it (evolution).

The disagreement about their contribution model of course happened, but the meaning you ascribe to it, perhaps is something you imagined. It boils down to what you understand "criticism" to be.

If I see someone doing something wrong, I will criticize them. That certainly never happened. What happened is that we pointed out pros and cons of an open and closed development model. We believe a piece of technology that plays the role of SQLite would benefit from having an open model. And exactly because they are absolutely not doing nothing wrong with not being open, we created our own thing. Hard to see how that is a "criticism".

I said that a billion times, and here's a billion and one: there's absolutely nothing wrong with a closed model. SQLite is doing nothing wrong. They contributed tremendously to the databases we used every day.

I do think an Open model yields so many benefits that should someone rewrite SQLite with an open model, even starting 20 years later, they would end up ahead.

There is now a very easy way to prove or disprove this particular hypothesis.

Stay tuned!

Re: Beyond the SQLite single-writer limitation with concurrent writes

#33

Post by fragmede »

glommer wrote: post #30
fragmede wrote: post #27

Earlier quoted context omitted.

More specifically, I didn't see any commits about multiple simultaneous writers having landed in main and it's not document as a feature in the release notes https://github.com/tursodatabase/turso/commits neither the prerelease https://github.com/tursodatabase/turso/releases/tag/v0.3.0-p... nor their latest release. https://github.com/tursodatabase/turso/releases/tag/v0.2.2 The core/mvcc directory does have commits t…

You just have to pass the experimental flag. I thought this was explained in the blog ?

I searched https://turso.tech/blog/beyond-the-single-writer-limitation-... with ctrl-f for "download" and "--exp" and got no results for either. There was a link to a live demo webpage at the very end. Am I looking at the right blog? That's the one linked at the top of the page.

Re: Beyond the SQLite single-writer limitation with concurrent writes

#34

Post by glommer »

rednafi wrote: post #31
glommer wrote: post #23

Earlier quoted context omitted.

Author of Turso here. Couple of points * You are right not to rush. You should keep using SQLite until Turso matures. Some use cases are more tolerant to new tech than others. It will take time for us to reach the level of trust SQLite has for broad use cases, but we are hoping to add value for some use cases right away. Never rush, tech matters! * I have never met Hipp, but only heard great things about him. * We ne…

Awesome to see you chiming in here. Just to set some context, I remember talking to Glauber and Pekka last year, right around when the Turso folks started toying with the idea of reimplementing SQLite in Rust. It’s a moonshot, but if anyone can pull it off, it’s these people. If you don’t know their background, checkout ScyllaDB. It never ceases to amaze me how belligerent some HN folks can be - they won’t even take…

Thanks! Appreciate it, and you will see we came a long way with the implementation. It's now right around the corner! We'd love to see you try and report back your impressions.

Re: Beyond the SQLite single-writer limitation with concurrent writes

#35

Post by glommer »

Permik wrote: post #14
frumplestlatz wrote: post #3

I’m not in a rush to use a reimplementation of SQLite — particularly from startup bros that had a very public, one-sided, and unpleasant fight with SQLite over their contribution model. D. Richard Hipp is a genuinely fantastic human being, and SQLite is a project developed at literally the level of planetary infrastructure given how broadly and everywhere it appears. Forking his project while using the name to garner…

I'd still give them the benefit of the doubt as most (all?) of the contributors are Finns who you can almost guarantee to have a "no bullshit" type of mentality to essentially everything. And what I'd guess, the team most probably has quite an academic background because of the local culture.

Pekka is a Finn, and there are other Finns as well, but not all of us are Finns.

Our Finns are also not the standard Finns. I know Pekka closely for 15 years, and he smiled 3 times.

Re: Beyond the SQLite single-writer limitation with concurrent writes

#36

Post by Quarrelsome »

crazygringo wrote: post #22

While I understand that you could do this, I genuinely don't understand why. SQLite and Postgres/MySQL/etc. occupy different niches. If you need massive concurrent writes, surely that's what Postgres/MySQL/etc. is for? Their engines are built around that from the ground up. SQLite is built around a file that stores data for a single application, as opposed to being client-server with many clients. I've used it a ton…

Agreed, Sqlite docs specifically state that its not designed for this, or rather that the solution isn't appropriate, that database as a service is appropriate when you want multiple clients writing.

The entire "problem" is a side-effect of using the wrong tool for the job.

Re: Beyond the SQLite single-writer limitation with concurrent writes

#37

Post by frumplestlatz »

glommer wrote: post #32
frumplestlatz wrote: post #28

Earlier quoted context omitted.

I wasn’t describing a legal dispute, and I haven’t made any claim about infringement. “Material misrepresentation” refers to the substance of the public messaging, not to a legal violation. The issue I raised is that the phrasing and the way the fork has been presented create the impression of continuity and endorsement that doesn’t exist. That’s a reputational and ethical concern, not a legal debate. Calling it “the…

I understood your criticism. I disagree with it, and I think we are not misrepresenting anything, as we make it very clear that we are not SQLite, we are just a reimplementation that goes beyond it (evolution). The disagreement about their contribution model of course happened, but the meaning you ascribe to it, perhaps is something you imagined. It boils down to what you understand "criticism" to be. If I see someon…

You’ve reframed this as a discussion about open versus closed development. That wasn’t the point I raised.

My concern is about presentation, ethics, respect, and about the co-option of a gift to the commons — particularly how your public messaging gives readers the impression of lineage and endorsement that doesn’t exist.

Regardless of your intent, that’s the effect of calling Turso “the next evolution of SQLite.” You’re welcome to disagree; it would be very strange if you didn’t.

Re: Beyond the SQLite single-writer limitation with concurrent writes

#39

Post by glommer »

Quarrelsome wrote: post #36
crazygringo wrote: post #22

While I understand that you could do this, I genuinely don't understand why. SQLite and Postgres/MySQL/etc. occupy different niches. If you need massive concurrent writes, surely that's what Postgres/MySQL/etc. is for? Their engines are built around that from the ground up. SQLite is built around a file that stores data for a single application, as opposed to being client-server with many clients. I've used it a ton…

Agreed, Sqlite docs specifically state that its not designed for this, or rather that the solution isn't appropriate, that database as a service is appropriate when you want multiple clients writing. The entire "problem" is a side-effect of using the wrong tool for the job.

Linux was designed to run in home PCs, and we keep running it in supercomputers. It works just fine. Tools evolve.

Re: Beyond the SQLite single-writer limitation with concurrent writes

#40

Post by glommer »

jauntywundrkind wrote: post #25

Very excellent to see the embedded DB in the browser link at the bottom. https://shell.turso.tech/ I've been thinking Turso in the browser would be some eventual addition, is something the Rust rewrite would eventually unlock. And that would require considerable extra backing to make go. But here it is now!

It's very easy to do given the async model. It is one of the new features that is working the best so far.
Post reply on HN