Live data from Hacker News

Work on SQLite4 has concluded

sqlite.org

141–150 of 161 posts

Re: Work on SQLite4 has concluded

#141
post #69

Earlier quoted context omitted.

It's basically the Perl 5 of the DB world.

Perl 6 you mean?

perl5 has been stealing a bunch of stuff from perl6 and is still actively maintained and doing a major release with new features annually - also continues to Just Work with an extreme commitment to backcompat.

So I'm pretty sure he did mean perl5, and as a happy user of both perl5 and sqlite the comparison seems apt.

Re: Work on SQLite4 has concluded

#142
post #122

Earlier quoted context omitted.

It's basically the Perl 5 of the DB world.

Not at all. perl5 is horrible tech, with no development and being actively destroyed by its maintainers. Whilst SQLite3 is at the very top of its class, with lots of new features, and very well maintained.

For context, rurban has had a long-ongoing feud with the perl5 maintainers that eventually resulted in his removal from the mailing list for being unwilling or unable to disagree sufficiently civilly to hold an effective discussion when people felt he was technically incorrect.

He's also, OTOH, a technically brilliant developer whose positive contributions will be missed.

(also if you ever run into him at a conference, I'd recommend grabbing a beer with him, I've always enjoyed doing so in spite of our spirited disagreements over various things)

Re: Work on SQLite4 has concluded

#143
post #55

Earlier quoted context omitted.

Imagine a company like Boeing then, which does have a 35 year shelf-life on their products.

Pretty soon we'll see aircraft running Kafka, microservices, elasticsearch, etc. All running in kubernetes.

"That's not the sort of 'above the clouds' we had in mind"

Re: Work on SQLite4 has concluded

#144
post #15

The biggest thorn I found working with sqlite was the lack of ability to modify columns with ALTER TABLE which was a real pain. Doesn't look like this is fixed in sqlite4 though...

There's an easy workaround: rename the table, create new table, insert into/select from.

The migration tools I work with have this already automated and (having been involved in the design of said automation) it's really not that difficult to add.

Other DBMSes often have to do complete table rewrites for certain types of ALTER anyway - so while, yes, it's faintly annoying to have to do that in userspace rather than having the db engine do it for you, it's not nearly as big a disadvantage as it first appears.

Re: Work on SQLite4 has concluded

#145
post #29
post #20

Earlier quoted context omitted.

Same here. Had to switch to dockerized mariadb for local tests, because migrations wouldn't work.

Wouldn't you want your tests to be run against the same DB family (and version) as production anyway?

Depending on the situation, it can be well worth it to have your test suite run against SQLite while doing active development to be able to iterate faster and then run it again against the target database before pushing the branch.

Where possible I much prefer to spin up a version of my target database in a tempdir but "faster test cycles" is sometimes worth accepting the trade-offs.

Re: Work on SQLite4 has concluded

#146
post #115

Earlier quoted context omitted.

To be fair, Fossil's intended use case is the exact opposite of the Linux kernel. See 3.3 and 3.4 of the fossil vs git page[0]. [0] https://www.fossil-scm.org/xfer/doc/trunk/www/fossil-v-git.w...

I find most of the differences listed there contrived. One big difference is that fossil includes wiki and ticketing. Philosophy differs: Fossil intentionally limits "distributedness". For example, fossil push/pull always transfers all branches with their name. Private branches are considered an anti-feature. Minor differences are the licence (GPL vs BSD) and the data store (files vs sqlite). Under some circumstances…

You're right, philosophy differs. I generally dislike private branches. It goes back to the origin of git -- intended for the linux kernel, one of the most widely used open source projects with tens of thousands of contributors. Linus doesn't want or need to see a million private branches. None of the projects I work on are of that scale. When your team is under a dozen people, being able to see what your coworkers are playing with in real time (autosync) is actually incredibly useful.

Stand-alone executable is pretty significant. Git is available on most servers -- fossil is not. If it's packaged in your OS, it's often outdated. Stand-alone kinda makes up for this as you can easily get the latest version with a wget & chmod on any computer, on all 3 platforms.

As for sqlite, it is an astoundingly solid rdbms that is well battle-tested. I consider that a big difference.

Re: Work on SQLite4 has concluded

#148

Earlier quoted context omitted.

I got lucky! It was a class event, and he's known to speak at databases / data systems classes. He's a very fun (and opinionated!) speaker. > The in-memory tree is an append-only red-black tree structure used to stage user data that has not yet flushed into the database file by the system. Hmm, ok, so this contradicts my assumption. Actually, now that I think about it, other LSMs like rocksdb / leveldb work like this…

What would be performance for SQLite3 in comparable scenarios? I don't see anything comparative on that benchmarks page.

I've gotten it to do around 40-50k inserts / sec but that's a different scenario - nfs drive, different table and indexes, different queries, different configuration, etc etc. Also I didn't know if he meant that the inserts were disappointing or the overall results were (e.g. a suite of tests including reads / writes of all sorts).

Re: Work on SQLite4 has concluded

#149
post #134

Earlier quoted context omitted.

No, it was Mozilla who killed it[1] over Apple and Google's strong objections. [1] For pretty much complete nonsense NIH and standards-lawyering reasons.

> For pretty much complete nonsense NIH and standards-lawyering reasons. You consider the fact that there is no spec other than "how the current implementation works", so no way to know if the code than works on version n will continue to work on version n +1, and no way to produce an alternate implementation, is a "complete nonsense reason"? I also believe that it wasn't just Mozilla, but also Microsoft that scuttle…

> You consider the fact that there is no spec other than "how the current implementation works", so no way to know if the code than works on version n will continue to work on version n+1, and no way to produce an alternate implementation, is a "complete nonsense reason"?

Yes. Sqlite is among the most stable and mature software in existence. I challenge you to name a single browser technology or API that has been anywhere remotely as stable as Sqlite, especially as implemented. As mentioned elsewhere ITT, Hwaci has contractual commitments to support backwards-compatible Sqlite3 for at least the next 35 years.

At that point, if I had to guess, "web browsers" won't be around any more, the web itself will be a dying or dead legacy technology, and companies yet to be founded will lead the tech sector, while Apple, Google, Microsoft and most likely of all Mozilla will be IBM-style dead letters or Digital[1]-style dead.

Sqlite was primarily written by one person and is maintained by three people.[2] If for some reason Mozilla had to fork or take over maintenance of Sqlite, it wouldn't even be worth noticing in their budget. It would, in fact, likely cost considerably less developer time than they spent designing, implementing and lobbying for IndexedDB.

More likely, if Sqlite3 were no longer being suitably maintained, all the browser vendors, plus the hundreds of other major tech companies that use it, would jointly endow a Sqlite Foundation to take over maintainence, which might cost Mozilla 5 grand or so a year, ye gods.

Even if Mozilla felt compelled for some reason (avoiding dependencies?) to rewrite Sqlite from scratch, Sqlite is 125k lines[3] of ANSI C compared to Firefox's 18 million lines[4] of C++, so Sqlite represents roughly one month of Firefox development time.

Firefox, Chrome, IE/Edge and Safari all make very extensive use of Sqlite internally, including to implement IndexedDB, so, obviously, Mozilla does not actually consider dependency on Sqlite and its API to present a meaningful risk.

Further, as things stand (and as they stood in 2010), Sqlite's massive footprint in the Firefox codebase means Mozilla already must rely on Sqlite remaining maintained, just the same as if WebSQL had been standardized.

> I also believe that it wasn't just Mozilla, but also Microsoft that scuttled this. Mozilla was the louder voice, but Microsoft never supported it either.

MS, especially at the time, ordinarily tried to slow or stall the standardization process for web technologies in general, so IE wouldn't fall too far behind. I expect better of Mozilla.

Mozilla was also by far the loudest (really the only) voice against WebSQL; MS barely participated in the conversation and was mostly noncommittal when it did. It's unlikely MS would've objected at all if Mozilla hadn't given them the opening.

[1] https://en.wikipedia.org/wiki/Digital_Equipment_Corporation

[2] https://sqlite.org/crew.html

[3] https://sqlite.org/testing.html

[4] https://www.openhub.net/p/firefox/analyses/latest/languages_...

Re: Work on SQLite4 has concluded

#150
post #79

Earlier quoted context omitted.

We literally lost several years for web app advancement because of that. Reading the decision making, it seemed like overly-legalistic engineers, but I'm open to conspiracy theories that this decision enhanced mobile app store adoption.

No, it was Mozilla who killed it[1] over Apple and Google's strong objections. [1] For pretty much complete nonsense NIH and standards-lawyering reasons.

And the horrible excuse that SQL was too complex to be exposed to web developers, so we now have SQL emulation libraries on top of IndexedDB that is implemented with SQLite. Oh yeah!
Post reply on HN