Live data from Hacker News

We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

beets.io

21–30 of 125 posts

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#21
post #14

Unless for example, your run a background process which periodically imports data from an external source and inserts into sqlite database, while your web application continues to render pages for visitors based on existing old data. Your web page will fail, as it will time out on database lock because of that background process.

Yes, that's what the article says.

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#22
post #14

Unless for example, your run a background process which periodically imports data from an external source and inserts into sqlite database, while your web application continues to render pages for visitors based on existing old data. Your web page will fail, as it will time out on database lock because of that background process.

You'll want to turn on write-ahead logging for that scenario[1]

[1]: https://www.sqlite.org/wal.html

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#23
post #16

Abstract the persistence layer, document the contract, build a conformance suite, and write your one impl that uses sqlite. I know sometimes there seems to be an impedance mismatch between callers and databases, but often it's not that bad. And no, don't use an ORM or existing DB abstraction to do this. Sure it takes work, but pluggable storage is nice on some of these types of things.

But what is the value add in this specific situation?

I am admittedly unfamiliar with this specific situation, but if this is a common request it can alleviate the requests by allowing others to write their own impls without decoupling what exists. Again though, maybe that's already present here and I am ignorant of the details.

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#24

Earlier quoted context omitted.

Sorry, put in my no-ORM edit as you may have been writing this. No, I mean higher level than that. ORMs suck as solving your problems specifically. Think more like an API than an ORM. I just started a side project and here's the beginning of an example of the iface I mean [0] and here's the beginning of an example of the sqlite impl I mean [1] (all untested and really early stage of course, but the idea is there). 0…

But they abstract the underlying db very well and have an uniform api that you can rely on. It's also tested, documented and battletested. It has drawbacks but if accepting several db is an important goal, you don't want to handcraft this unless you can measure inaceptable perfs.

> they abstract the underlying db very well

I am admittedly unfamiliar with this specific case, but even a cursory glance showed me some sqlite-specific code embedded in the main library portion [0].

0 - https://github.com/beetbox/beets/blob/4f7c1c9beda6c2a5b0705e...

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#25
post #19

> The idea is that a more complicated DBMS should be faster, especially for huge music libraries. How "huge" are they talking about? I built a tool that imports Apache logs into sqlite for quick analysis and that easily handles several million records.

Just a ballpark: my beets DB is 15 MB for ~1000 albums (well, folders). This post[0] claims 10M albums ever released. So maybe 150 GB upper bound? Assuming my db has been vacuumed recently (~~probably not true~~ Edit: nope, still 15M after vacuum) and ignoring that indexes will scale slightly non-linearly.

[0]: https://www.quora.com/How-many-music-albums-are-available-in...

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#26
post #10

SQLite is a fantastic database for desktop applications. https://www.sqlite.org/appfileformat.html is a great essay on its benefits for this kind of use-case.

Not only desktop, but most mobile platforms as well. Beeing coded in a single C file makes it portable as hell. Their documentation is also a prime example, with great insight.

It's not coded that way, it's combined as part of a script.fd

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#28
post #10

SQLite is a fantastic database for desktop applications. https://www.sqlite.org/appfileformat.html is a great essay on its benefits for this kind of use-case.

Not only desktop, but most mobile platforms as well. Beeing coded in a single C file makes it portable as hell. Their documentation is also a prime example, with great insight.

Not only desktop and mobile, but it can be used as the storage format for larger scale distribued systems. For example it's the on disk format for FoundationDB.

>Beeing coded in a single C file makes it portable as hell.

It's not coded in a single C file...

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#29
post #8

Kudos to this guy. As the maintainer of an open source project, the hardest job is to tell people "your idea is stupid and I won't do it"

That seems pretty easy. It's much harder to try to understand where someone is coming from and engage with them in a positive way, even if you're rejecting their idea in the end.

Re: We’re happy with SQLite and not urgently interested in a fancier DBMS (2016)

#30
We use SQLite in an app with about 1,000 active users. It took us:

- 0h to manage backups ("cp"),

- 0h to manage seeds and tests fixtures ("cp"),

- 0h to configure and secure (void),

- 0h to write the deployment scripts (void),

- 0h monitoring/watchdog jobs (void),

- 1h to rsync for failover ("rsync")

My last projects always spent at least a good 100h to do all of this the right way. Then, if it is not good enough, we'll move to RDS or equivalent.

Post reply on HN