Live data from Hacker News

SQLite Is Serverless

sqlite.org

91–100 of 453 posts

Re: SQLite Is Serverless

#91
It does allow multiple applications to access the same database at the same time, but when you do so it really hurts performance. I noticed this when i wrote a web crawler in go and used sqlite as the backend.

As soon as i connect using the command line interface, it slows down significantly.

Just something to bear in mind if you want to use it with multiple processes!

Re: SQLite Is Serverless

#92
post #91

It does allow multiple applications to access the same database at the same time, but when you do so it really hurts performance. I noticed this when i wrote a web crawler in go and used sqlite as the backend. As soon as i connect using the command line interface, it slows down significantly. Just something to bear in mind if you want to use it with multiple processes!

Even with WAL enabled?

Re: SQLite Is Serverless

#93
post #91

It does allow multiple applications to access the same database at the same time, but when you do so it really hurts performance. I noticed this when i wrote a web crawler in go and used sqlite as the backend. As soon as i connect using the command line interface, it slows down significantly. Just something to bear in mind if you want to use it with multiple processes!

I use it for real time data (around 22G/month, compressed) and I can do all sorts of filthy stuff with the database while the real time processes are running.

Increasing the retry timeout can help, using WAL can help. I'm sure you've tried all this though.

Re: SQLite Is Serverless

#94
post #49

My understanding of serverless = easily scalable, managed service. But somehow the word annoys people. Maybe we should find a better word?

My understanding was that cloud = easily scalable, managed service.

That was mine too. It’s possible that this previously applied to primitives (cpu, memory, network, storage) and now refers to applications (keyvalue store, SQL database, message queues, etc.)

Re: SQLite Is Serverless

#95

> Of those that are serverless, SQLite is the only one known to this author that allows multiple applications to access the same database at the same time. IIRC, MS Access allowed that, which explained a lot of its popularity.

MS Access “allowed” it but depending on the version it could be quite problematic. We had a use case where Tableau connected to an Access file read only (but which another program used as a data store and wrote to it often) on an Windows file share, and once in a while the lock files would get screwy and we would have to manually delete the lock files to get things working again. Deleting lock files could be a huge chore because you have to figure out stuck processes and kill them. Task Manager wasn’t up to the task, so had to use SysInternal tools to help with that.

Access is really meant for single-user scenarios I feel. Maybe the locking mechanism has gotten better but for multiuser access I tell people to use a real SQL database.

Re: SQLite Is Serverless

#96

> Of those that are serverless, SQLite is the only one known to this author that allows multiple applications to access the same database at the same time. IIRC, MS Access allowed that, which explained a lot of its popularity.

Not open source, but Raima supports multiple processes on a single db file as well. https://raima.com/

Re: SQLite Is Serverless

#97
post #85
post #31

Earlier quoted context omitted.

Waaaaait wouldn’t that mean the file system is the server, with some binary API and responsible for handling concurrent access and locks for the entire file? LOL.

That just circle jerk. There is an agreed upon def of serverless. If we reduce things to the absurd we stop being able to reason about things. https://en.m.wikipedia.org/wiki/Serverless_computing

>There is an agreed upon def of serverless.

Sorta but not really. The fact people have worked backwards from marketing names to try and constructively define inherently self-contradictory branding (rather than create a descriptive category into which we place questionable names and ignore them) is an embarrassment for everyone except the marketing departments.

Re: SQLite Is Serverless

#98
post #86
post #81

Earlier quoted context omitted.

Access is actually designed to work in a multiuser situation over a LAN for concurrent read and write. SQLite isn't afaik. As long as the LAN was cabled I never saw any issues. The only reason it was necessary to move to a server type database was because people were insisting on wifi networking.

I'm not sure I understand what the physical transport has to do with this. Can you please expand a bit?

Race conditions not triggering when the transport is fast enough?

Re: SQLite Is Serverless

#99

> Of those that are serverless, SQLite is the only one known to this author that allows multiple applications to access the same database at the same time. IIRC, MS Access allowed that, which explained a lot of its popularity.

This was a standard feature of flat file databases in the early 90s. There were many products. They often had an ODBC driver, which provided a SQL front end. dBase .dbf files were often used for storage. The arcane file locking in Windows is intended for exactly this kind of application.

Apart from quality (!), SQLite's main advantage over these products is broad platform support. And continued existence.

Re: SQLite Is Serverless

#100

> Of those that are serverless, SQLite is the only one known to this author that allows multiple applications to access the same database at the same time. IIRC, MS Access allowed that, which explained a lot of its popularity.

In the old days of DOS I could do that with Clipper 5.2. Two or more PCs using Netbios shared directories could work on the same database, provided that they wouldn't write to the same record. That wasn't a problem because the environment (can't recall for sure if it was Clipper or an external library) allowed single record locking, so I enclosed the lock attempt into a timed spinlock-like block which attempted once per second for 10 or 15 seconds before obtaining the lock or failing with a record busy error. No SQL involved however, and indexes had to be rebuilt by hand like twice a day to be safe. But those were the days when a 486 could crunch a 100.000 records db with multiple indexes very easily.
Post reply on HN