Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite is likely used more than all other database engines combined

sqlite.org

Post reply on HN
Print view View original

41–50 of 82 posts

Re: SQLite is likely used more than all other database engines combined

#41

Post by novariation »

simonw wrote: post #33
TZubiri wrote: post #7

In terms of installations, sure. In terms of total data, probably not. In terms of important data, 100% not. Important data needs guarantees, and its privacy is second to its longevity. Server side and centralized beats client side decentralized for all but sensitive consumer data.

You may be shocked to learn how much of the world's "important data" lives in Excel files.

Fun fact: my (french) company audited the french navy and found out they share important nuclear submarine maintenance data as excel files through email.

Re: SQLite is likely used more than all other database engines combined

#42

Post by tibbar »

hehdhdjehehegwv wrote: post #39
TheRealPomax wrote: post #12

Earlier quoted context omitted.

I'm not sure I buy the "important data" one: my Lightroom catalog, to me , constitutes incredibly important data. Losing it and just having the base files would destroy over a decade of work. It also happens to be a SQLlite3 database. And the same is true for a slew of other applications that (quite rightly) use SQLite databases as their file format. You might be thinking of things like financial transactions, or med…

The number of societal-wide problems that occur if your Lightroom database gets corrupted is zero. The amount of hell that would be unleashed if the financial systems layers upon layers of database transactions got broken is impossible to comprehend. So if you mean “important” as “necessary for society to function”, then no, your browser bookmark files, contact list, or the other two dozen things your laptop and phon…

This is a false equivalence, as one user’s particular usage of SQLite is not comparable to all financial instructions’ databases.

Better to compare to the prospect of all smartphones being irreversibly corrupted at once.

Re: SQLite is likely used more than all other database engines combined

#43

Post by hehdhdjehehegwv »

darby_nine wrote: post #30
talkingtab wrote: post #20

For niche uses of database I am sure this is true. But if you are not in that niche, then Postgresql is usually a much better choice. For example Firefox uses SQLite, because it wants to store lots of application specific data. It is good for that. Personally, I find the tooling, documentation, familiarity and quality of Postgresql makes it my choice even in situations where SQLite might work.

I'd guess most programs ever written are single-process programs with no need for the benefits postgres provides over sqlite. Postgres doesn't even have analogous functionality for eg an in-memory sql instance!

It most definitely does and I’m serving several read-only production databases from memory. It is insanely fast if you do it right.

https://www.postgresql.org/docs/current/pgprewarm.html

Re: SQLite is likely used more than all other database engines combined

#44

Post by ARandomerDude »

novariation wrote: post #41
simonw wrote: post #33

Earlier quoted context omitted.

You may be shocked to learn how much of the world's "important data" lives in Excel files.

Fun fact: my (french) company audited the french navy and found out they share important nuclear submarine maintenance data as excel files through email.

Former US military officer here. This does not surprise me at all. I wasn’t in subs but we absolutely did this for maintenance of other multimillion dollar assets.

Re: SQLite is likely used more than all other database engines combined

#45

Post by wavemode »

hehdhdjehehegwv wrote: post #39
TheRealPomax wrote: post #12

Earlier quoted context omitted.

I'm not sure I buy the "important data" one: my Lightroom catalog, to me , constitutes incredibly important data. Losing it and just having the base files would destroy over a decade of work. It also happens to be a SQLlite3 database. And the same is true for a slew of other applications that (quite rightly) use SQLite databases as their file format. You might be thinking of things like financial transactions, or med…

The number of societal-wide problems that occur if your Lightroom database gets corrupted is zero. The amount of hell that would be unleashed if the financial systems layers upon layers of database transactions got broken is impossible to comprehend. So if you mean “important” as “necessary for society to function”, then no, your browser bookmark files, contact list, or the other two dozen things your laptop and phon…

The goalposts keep moving here. You can just take any arbitrary definition of important and use it to exclude SQLite deployments.

It's all semantics, and also irrelevant to the original article anyway, since nowhere does it argue that SQLite holds the most data (important or otherwise).

Re: SQLite is likely used more than all other database engines combined

#46

Post by Twirrim »

oneplane wrote: post #36
talkingtab wrote: post #20

For niche uses of database I am sure this is true. But if you are not in that niche, then Postgresql is usually a much better choice. For example Firefox uses SQLite, because it wants to store lots of application specific data. It is good for that. Personally, I find the tooling, documentation, familiarity and quality of Postgresql makes it my choice even in situations where SQLite might work.

If I'm not mistaken, Apple's CoreStorage is based on SQLite, so pretty much every Apple device for the last years (or decade, probably) is constantly using tens (or more) SQLite databases, per OS. That includes Macs, Phones, Watches, AppleTV etc. That has got to be more than 3 billion devices (since there are at least 2 billion of the phones in active use). SQLite is also quite often a CI/CD default when building and…

It's also heavily used on Android, so that's some 3bn devices (based on active users numbers from earlier this year). Plus it's in Firefox, Chrome, and so on.

Re: SQLite is likely used more than all other database engines combined

#47

Post by achillean »

qazxcvbnmlp wrote: post #11

SQLite solves a lot of problems well enough that you can focus on other things. Have an mvp you’re not sure you’ll ever have more that 2 users - yep. Storing a little data for an application on the disk and don’t want to write your own schema. Want to teach someone how databases work without setting up a sever - sure.

A few others:

- Want to distribute data to users that don't want to manage a server? A lot of people don't want to manage a server and don't need the best possible performance.

- Want to take data with you on a thumb drive and work with it offline? It's extremely convenient to be able to use SQLite for an app that has to work offline.

- Does the app mostly just read from the database and fit in memory? It's undervalued to just put the entire database into memory so you don't hit the disk and don't introduce network latency. For example, the following website does all enrichment with in-memory SQLite databases: https://shdn.io/analyze?target=ycombinator.com

At Shodan, we distribute versions of our datasets as SQLite and they're a popular way to consume the data without having to manage infrastructure.

Re: SQLite is likely used more than all other database engines combined

#48

Post by hehdhdjehehegwv »

wavemode wrote: post #45
hehdhdjehehegwv wrote: post #39

Earlier quoted context omitted.

The number of societal-wide problems that occur if your Lightroom database gets corrupted is zero. The amount of hell that would be unleashed if the financial systems layers upon layers of database transactions got broken is impossible to comprehend. So if you mean “important” as “necessary for society to function”, then no, your browser bookmark files, contact list, or the other two dozen things your laptop and phon…

The goalposts keep moving here. You can just take any arbitrary definition of important and use it to exclude SQLite deployments. It's all semantics, and also irrelevant to the original article anyway, since nowhere does it argue that SQLite holds the most data (important or otherwise).

I didn’t comment earlier so I can’t move a goalpost I never set down. I agree with the root comment that there may be more instances of SQLite, the most important data is not in SQLite.

For what it’s worth, I have used any number of databases over the years and SQLite is very good for a number of things.

None of those things are the core infrastructure that stores your emails, money, and other must have, shared, high availability data.

There are different tools for different jobs, that’s fine.

Re: SQLite is likely used more than all other database engines combined

#49

Post by bob1029 »

TZubiri wrote: post #7

In terms of installations, sure. In terms of total data, probably not. In terms of important data, 100% not. Important data needs guarantees, and its privacy is second to its longevity. Server side and centralized beats client side decentralized for all but sensitive consumer data.

> Server side and centralized beats client side decentralized

SQLite works on the server too. Important data can live in a SQLite database.

Re: SQLite is likely used more than all other database engines combined

#50

Post by strogonoff »

simonw wrote: post #33
TZubiri wrote: post #7

In terms of installations, sure. In terms of total data, probably not. In terms of important data, 100% not. Important data needs guarantees, and its privacy is second to its longevity. Server side and centralized beats client side decentralized for all but sensitive consumer data.

You may be shocked to learn how much of the world's "important data" lives in Excel files.

A lot of world’s important data is on paper.

As far as important data served from a relational database is concerned, MySQL most likely beats the hell out of everything in terms of popularity. The bulk of SQLite’s usage has to be various runtime caches, file data, and so on.

Post reply on HN