Live data from Hacker News

Work on SQLite4 has concluded

sqlite.org

101–110 of 161 posts

Re: Work on SQLite4 has concluded

#101
post #79

Earlier quoted context omitted.

We literally lost several years for web app advancement because of that. Reading the decision making, it seemed like overly-legalistic engineers, but I'm open to conspiracy theories that this decision enhanced mobile app store adoption.

No, it was Mozilla who killed it[1] over Apple and Google's strong objections. [1] For pretty much complete nonsense NIH and standards-lawyering reasons.

I too am so disappointed SQLite isn't available in modern browsers (esp. since it was, for a time). But can't it be resurrected? Couldn't we set up a petition somewhere to bring it back?

Re: Work on SQLite4 has concluded

#102

Earlier quoted context omitted.

I'm jealous you got to hear Dr. Hipp, that sounds cool. Would love to hear more about the circumstances :) Regarding the LSM engine, you can find all the relevant implementation details here: https://sqlite.org/src4/doc/trunk/www/lsm.wiki#summary > The in-memory tree is an append-only red-black tree structure used to stage user data that has not yet flushed into the database file by the system. Under normal circumsta…

I got lucky! It was a class event, and he's known to speak at databases / data systems classes. He's a very fun (and opinionated!) speaker. > The in-memory tree is an append-only red-black tree structure used to stage user data that has not yet flushed into the database file by the system. Hmm, ok, so this contradicts my assumption. Actually, now that I think about it, other LSMs like rocksdb / leveldb work like this…

What would be performance for SQLite3 in comparable scenarios? I don't see anything comparative on that benchmarks page.

Re: Work on SQLite4 has concluded

#103

Earlier quoted context omitted.

Pretty soon we'll see aircraft running Kafka, microservices, elasticsearch, etc. All running in kubernetes.

At least that way if the plane crashes you can just restart it.

Restart just the microservice that crashed!

  $ sudo service port-flaps restart

Re: Work on SQLite4 has concluded

#104

Earlier quoted context omitted.

Everything I hear about SQLite3 always suggests that, essentially, it is considered "done". It does what it is supposed to do with great performance. There is nothing major left to do. If it doesn't meet your needs, pick a different SQL database. Which, while a totally alien concept in the modern software world, is actually a pretty cool thought. (I'm sure under the hood bugs are getting fixed and all)

Well, obviously they should rewrite it in Rust.

From the canonical source (as in, [Richard Hipp](https://en.wikipedia.org/wiki/D._Richard_Hipp)):

Rewriting SQLite in Rust, or some other trendy “safe” language, would not help. In fact it might hurt.

Prof. Regehr did not find problems with SQLite. He found constructs in the SQLite source code which under a strict reading of the C standards have “undefined behaviour”, which means that the compiler can generate whatever machine code it wants without it being called a compiler bug. That’s an important finding. But as it happens, no modern compilers that we know of actually interpret any of the SQLite source code in an unexpected or harmful way. We know this, because we have tested the SQLite machine code – every single instruction – using many different compilers, on many different CPU architectures and operating systems and with many different compile-time options. So there is nothing wrong with the sqlite3.so or sqlite3.dylib or winsqlite3.dll library that is happily running on your computer. Those files contain no source code, and hence no UB.

The point of Prof. Regehr’s post (as I understand it) is the the C programming language as evolved to contain such byzantine rules that even experts find it difficult to write complex programs that do not contain UB.

The rules of rust are less byzantine (so far – give it time :-)) and so in theory it should be easier to write programs in rust that do not contain UB. That’s all well and good. But it does not relieve the programmer of the responsibility of testing the machine code to make sure it really does work as intended. The rust compiler contains bugs. (I don’t know what they are but I feel sure there must be some.) Some well-formed rust programs will generate machine code that behaves differently from what the programmer expected. In the case of rust we get to call these “compiler bugs” whereas in the C-language world such occurrences are more often labeled “undefined behavior”. But whatever you call it, the outcome is the same: the program does not work. And the only way to find these problems is to thoroughly test the actual machine code.

And that is where rust falls down. Because it is a newer language, it does not have (afaik) tools like gcov that are so helpful for doing machine-code testing. Nor are there multiple independently-developed rust compilers for diversity testing. Perhaps that situation will change as rust becomes more popular, but that is the situation for now.

https://blog.regehr.org/archives/1292#comment-18452

Re: Work on SQLite4 has concluded

#105

Earlier quoted context omitted.

Official SQLite response http://sqlite.1065341.n5.nabble.com/Window-functions-td77470...

So in essence it's a lot of work for a small functionality gain? There are some things that are inconvenient to do without window functions, but my primary use for them is simple optimization. Most of the cases where I've used window function in postgres, the query would be trivial to rewrite to use a subquery instead. Window functions always led to a much faster query that scaled much better over large query sets.

Exactly why I miss them so much.

Re: Work on SQLite4 has concluded

#106

Earlier quoted context omitted.

lack of window functions kill it's usefulness for me.

Official SQLite response http://sqlite.1065341.n5.nabble.com/Window-functions-td77470...

Reading through the responses just screams: not worth the effort. I know it’s an open source volunteer product and even with a commercial arm the lack of window functions (aren’t they part of the SQL standard?) is laughable.

Re: Work on SQLite4 has concluded

#107

Earlier quoted context omitted.

Everything I hear about SQLite3 always suggests that, essentially, it is considered "done". It does what it is supposed to do with great performance. There is nothing major left to do. If it doesn't meet your needs, pick a different SQL database. Which, while a totally alien concept in the modern software world, is actually a pretty cool thought. (I'm sure under the hood bugs are getting fixed and all)

Well, obviously they should rewrite it in Rust.

Because we’ve missed the Rust evangelism strike force so very much.

I honestly hope against hope that you’re being sarcastic.

Re: Work on SQLite4 has concluded

#108
post #12

Earlier quoted context omitted.

Would have been a neat way to experiment around with putting a SQL front end on various KV interfaces. Redis or etcd, for example.

CockroachDB has a good blog post[0] that describes how they implemented SQL. (CockroachDB is a key-value store.) [0] https://www.cockroachlabs.com/blog/sql-in-cockroachdb-mappin...

>(CockroachDB is a key-value store.)

Correction: CockroachDB is based on a KV store, it's a full SQL RDBMS on top of this KV store.

Re: Work on SQLite4 has concluded

#109
post #53
post #13

Richard Hipp has said that they have signed contracts to support SQLite3 for 35 years. SQLite4 is never going to happen.

I love this section from their support page. Nothing like stability through 2050. https://www.hwaci.com/sw/sqlite/prosupport.html > Paid support options and products are provided by Hipp, Wyrick & Company, Inc., (Hwaci), a Georgia corporation with headquarters in Charlotte, North Carolina and has been in business since 1992. Hwaci has an international team of employees and associates representing the best available t…

Dr. Hipp will be 89 years old in 2050 so he simply made a lifetime commitment.

Re: Work on SQLite4 has concluded

#110

Earlier quoted context omitted.

Well, obviously they should rewrite it in Rust.

From the canonical source (as in, [Richard Hipp]( https://en.wikipedia.org/wiki/D._Richard_Hipp) ): Rewriting SQLite in Rust, or some other trendy “safe” language, would not help. In fact it might hurt. Prof. Regehr did not find problems with SQLite. He found constructs in the SQLite source code which under a strict reading of the C standards have “undefined behaviour”, which means that the compiler can generate what…

> it does not have (afaik) tools like gcov

You can just use C tooling with Rust, generally; I know kcov has worked for a long time, but it looks like easy gcov support does exist: https://github.com/kennytm/cov

Post reply on HN