Live data from Hacker News

Consider SQLite

blog.wesleyac.com

11–20 of 274 posts

Re: Consider SQLite

#11

I use SQLite exclusively on a high performance crypto sniper project - https://bsctrader.app and I could not be happier with it. Performs much better then postgres in terms of query latency which is ultra important for the domain we operate in. I take machine level backups every 2 hours, so in the event of an outage, just boot the disk image on a new vm and it's off. I would never do this on my professional job due t…

The stigma of using the most popular database in existence?

Re: Consider SQLite

#12

I have tried adopting SQLite in my side projects. The problem I encountered is that using managed PostgreSQL/MySQL is still more convenient and more reliable than using SQLite on a bare metal VPS. I like to use Heroku or Digital Ocean App platform because I want to spend time creating and not managing the infrastructure (ci/cd, ssl certs, reverse proxy, db backup, scaling, container management and what not). I tried…

What would a managed sqlite even look like? I can't tell if this is a real response or not...

I clearly did not express myself correctly. Being all these managed solutions (heroku, do app platform) running on an ephemeral fs it is not possible to use disk as long term storage as sqlite requires. So I’m basically forced to manage an attached disk to my container which needs to be backed up, monitored and so on

Re: Consider SQLite

#13
I love to see that more projects are using SQLite as their main database.

One thing that I always wondered though: does anyone knows a big project/service that uses Golang and is backed by SQLite? This because SQLite would require CGO and CGO generally adds extra complexities and performance costs. I wonder how big Golang applications fare with this.

Re: Consider SQLite

#14
post #4

There are some important things that SQLite does not do. It is not client/server; a process must be able to fopen() the database file. NFS and SMB are options that can convey access to remote systems, but performance will not likely be good. Only a single process can write to the database at any time; it does not support concurrent writers. The backup tools do not support point-in-time recovery to a specific past tim…

Note -- the single process write at any one time is a killer for most web apps, where for example within SaaS you have many users doing things at the same time.

Re: Consider SQLite

#15

I have tried adopting SQLite in my side projects. The problem I encountered is that using managed PostgreSQL/MySQL is still more convenient and more reliable than using SQLite on a bare metal VPS. I like to use Heroku or Digital Ocean App platform because I want to spend time creating and not managing the infrastructure (ci/cd, ssl certs, reverse proxy, db backup, scaling, container management and what not). I tried…

[deleted]

Re: Consider SQLite

#16

SQLite database can be stored in git which seems like a great benefit. But I wonder would it also be possible to have different "branches" of the database and then merge them at some point?

It's not integrated with git the way you're perhaps imagining, but SQLite sessions[0] is adjacent to what you're imagining.

[0] https://www.sqlite.org/sessionintro.html

Re: Consider SQLite

#17

Just a note that there are significant features of SQLAlchemy that don’t work with SQLite such as ARRAY columns, UUID primary keys and certain types of foreign key constraints.

well no major database other than PostgreSQL has native support for UUID or ARRAY, you can certainly use string-based types for these things for other databases. the DB agnostic UUID is at https://docs.sqlalchemy.org/en/14/core/custom_types.html?hig... and for ARRAY it's likely most convenient to use the SQLite JSON datatype which is also supported directly.

Re: Consider SQLite

#18
post #4

There are some important things that SQLite does not do. It is not client/server; a process must be able to fopen() the database file. NFS and SMB are options that can convey access to remote systems, but performance will not likely be good. Only a single process can write to the database at any time; it does not support concurrent writers. The backup tools do not support point-in-time recovery to a specific past tim…

I don't know what backup tools you have in mind... But since a SQLite database is a single file (modulo write ahead journal and whatnot), making whatever you need is trivial.

Re: Consider SQLite

#19

Earlier quoted context omitted.

What would a managed sqlite even look like? I can't tell if this is a real response or not...

When hankchinaski says 'managed' I think they really mean that there's some capital-A App dashboard somewhere, on Digital Ocean or wherever, and they log in and click 'new database' and that's it. No ssh-ing to a VPS, and choosing the file location where the sqlite file will sit, figuring out backups and so on. But as you say, while you can wrap postgres or redis in that sort of 'just take care of it for me' approach…

Exactly. I was expressing the limitation of that single tier paradimg viewed from an angle of someone who doesn’t want to ssh into machines to configure and deploy code... :)

Re: Consider SQLite

#20
post #4

There are some important things that SQLite does not do. It is not client/server; a process must be able to fopen() the database file. NFS and SMB are options that can convey access to remote systems, but performance will not likely be good. Only a single process can write to the database at any time; it does not support concurrent writers. The backup tools do not support point-in-time recovery to a specific past tim…

> Only a single process can write to the database at any time; it does not support concurrent writers.

I think you can do concurrent writing now with Write-Ahead Logging (WAL): https://www.sqlite.org/wal.html

I've never tried it though, so I don't know how suitable it is for web apps that might potentially have multiple processes trying to write to the DB at the same time.

Post reply on HN