Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Work on SQLite4 has concluded

sqlite.org

Post reply on HN
Print view View original

121–130 of 161 posts

Re: Work on SQLite4 has concluded

#121

Post by rurban »

greenhouse_gas wrote: post #104
ianamartin wrote: post #65

Earlier quoted context omitted.

Well, obviously they should rewrite it in Rust.

From the canonical source (as in, [Richard Hipp]( https://en.wikipedia.org/wiki/D._Richard_Hipp) ): Rewriting SQLite in Rust, or some other trendy “safe” language, would not help. In fact it might hurt. Prof. Regehr did not find problems with SQLite. He found constructs in the SQLite source code which under a strict reading of the C standards have “undefined behaviour”, which means that the compiler can generate what…

Rewriting in Rust would not make any sense, sure. But rewriting in a faster and safer language, like Pony or ATS might, esp. for lots of CPU's.

Re: Work on SQLite4 has concluded

#122

Post by rurban »

jandrese wrote: post #46
krylon wrote: post #18

Earlier quoted context omitted.

Look at it this way: SQLite3 is (more or less) slowly becoming SQLite4, except for the parts that did not work out. It is not as shiny, but in the long run, you still get all the goodness. Nevermind the name / version number.

It's basically the Perl 5 of the DB world.

Not at all. perl5 is horrible tech, with no development and being actively destroyed by its maintainers.

Whilst SQLite3 is at the very top of its class, with lots of new features, and very well maintained.

Re: Work on SQLite4 has concluded

#123

Post by rootw0rm »

infogulch wrote: post #80
katastic wrote: post #72

Earlier quoted context omitted.

Perhaps I'm misunderstanding you, but sqlite supports ramdisks/"In-memory databases" https://www.sqlite.org/inmemorydb.html Perhaps the key (ha!, pun) is that you're talking about using RAM _and_ disk with the RAM being for caching/fast access that eventually hits the disk. Whereas, I think, in this case sqlite is either on the disk, or in RAM. There is no multiple tiers. Correct me if I'm off here. Thanks.

Actually in WAL mode with `PRAGMA synchronous = NORMAL`, it only flushes to disk periodically, not after every transaction.

Yup, it's pretty flexible and awesome. Just used a SQLite3 memory db recently to quickly import 50GB of data and then backed it up to disk.

Re: Work on SQLite4 has concluded

#124

Post by chj »

smitherfield wrote: post #89
dude01 wrote: post #79

Earlier quoted context omitted.

We literally lost several years for web app advancement because of that. Reading the decision making, it seemed like overly-legalistic engineers, but I'm open to conspiracy theories that this decision enhanced mobile app store adoption.

No, it was Mozilla who killed it[1] over Apple and Google's strong objections. [1] For pretty much complete nonsense NIH and standards-lawyering reasons.

And they think index db is better..

Re: Work on SQLite4 has concluded

#125

Post by mschuster91 »

chj wrote: post #124
smitherfield wrote: post #89

Earlier quoted context omitted.

No, it was Mozilla who killed it[1] over Apple and Google's strong objections. [1] For pretty much complete nonsense NIH and standards-lawyering reasons.

And they think index db is better..

IndexedDB is a hellhole, but it fitted the NoSQL trend at the time - that's why it was picked and SQLite dumped as "fossil".

I'd love it if someone takes the time and creates a SQL engine based on IndexedDB so one can do serious SQL work instead of working around the many limitations in IndexedDB.

Re: Work on SQLite4 has concluded

#126

Post by kamac »

jey wrote: post #29
kamac wrote: post #20

Earlier quoted context omitted.

Same here. Had to switch to dockerized mariadb for local tests, because migrations wouldn't work.

Wouldn't you want your tests to be run against the same DB family (and version) as production anyway?

I suppose it was a long time coming. SQLite was very convenient for me while the product was in a very early development stage, with everything changing rapidly.

Re: Work on SQLite4 has concluded

#127

Post by dspillett »

bambax wrote: post #101
smitherfield wrote: post #89

Earlier quoted context omitted.

No, it was Mozilla who killed it[1] over Apple and Google's strong objections. [1] For pretty much complete nonsense NIH and standards-lawyering reasons.

I too am so disappointed SQLite isn't available in modern browsers (esp. since it was, for a time). But can't it be resurrected? Couldn't we set up a petition somewhere to bring it back?

It is a bit of a "would you really do that in production?" type hack, but there is a pure JS compile of SQLite3 that you could use: https://github.com/kripken/sql.js/

Possible problems:

* Nearly half an MB of library to add to your project which might be a concern on mobile (~2.1MB uncompressed.

* It handles the whole DB in memory rather than trying to use any sort of local storage as a block store, which pumps up memory use (again, mobile user may particularly find this an issue) and to persist data you have to pickle the whole DB as a single array (which could be a significant performance issue if the data changes regularly and is not very small) and reload it upon new visit.

* Concurrency between multiple tabs/windows is going to be an issue for the same reason.

Re: Work on SQLite4 has concluded

#128

Post by HNSucksAss »

Scaevolus wrote: post #3

For context, SQLite4 explored reimplementing SQLite using a key-value store on log-structured merge trees, like RocksDB and Cassandra. I'd be interested to hear why they stopped. Presumably reimplementing SQL on a KV store was seen as not worth it, when applications that are satisfied with an embedded KV store backend (which is much faster and simpler to write!) already have many options.

It's key/value store, not key-value.

Re: Work on SQLite4 has concluded

#129

Post by Daycrawler »

hasenj wrote: post #83
bane wrote: post #73

SQLite is one of those awesome things that's the exact opposite of magic. It's beautiful, jaw dropping, engineering that exercises so many technical muscles. The number of oddball, often critical, places where I've found SQLite being used would defy belief. As far as I can tell, the "expected" place for SQLite to work seems to be almost anything that's not your normal dB driving some web-based CRUD app...all kinds of…

> your normal dB driving some web-based CRUD app That can totally be handled with SQLite.

Unless you want to scale. I'm all for the rule of least power, but as soon as the app is exposed to multiple users I would ask and be sure about the expected number of simultaneous users before going with SQlite instead of going with a Client/Server RDBMS. Still, the bound is pretty high if you keep your transactions short.

Re: Work on SQLite4 has concluded

#130

Post by k__ »

Afton wrote: post #2

Would love to hear some of the lessons learned...

Their main idea was that B-Trees are slow and LSMs are fast.

This was a partially right assumption, but only for writes.

If you write something in a DB you check some constraints and those checks are reads.

So most DB writes come with a bunch of reads.

The reads were slower with the LSMs, so the B-Trees performed better in "real world" writes (which come with reads) and LSMs only performed better in "artificial" writes (without reads).

Post reply on HN