Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Is Serverless

sqlite.org

Post reply on HN
Print view View original

211–220 of 453 posts

Re: SQLite Is Serverless

#211

Post by bane »

hartator wrote: post #208
bane wrote: post #164

Earlier quoted context omitted.

> and I think I've always approached it "from a classic RDBMS angle" Have you tried to figure out where to install the server or asked what the system requirements were for it? Have you grown concerned that once the system moves into production the O&M team won't know how to operate "yet another database"? Do you spend agonizing hours trying to figure out if it supports multithreaded connection pools for multi-user w…

The original document is from SQLite documentation. I think it’s fair for them to make a case why sometimes a file database > a server database. People asking these questions are not stupid. We all have to start somewhere.

Please check it again, I wrote "not stupid".

The SQLite creators have a great deal of documentation that's targeting database administrators and users and trying to explain what this thing is, when I think they really should have just targeted people who need a nice file format and a clean API.

But hell, there's like a trillion SQLite files in use so what do I know?

Re: SQLite Is Serverless

#212

Post by hartator »

bane wrote: post #194
bshipp wrote: post #180

Earlier quoted context omitted.

I disagree. CSVs don't have write-ahead logging and are not able to selectively lock portions file for writes. And I'd say that not only is sqlite in the db marketplace (albeit for a specific subset of database application types) it's one of the largest players.

Why not? Just roll your own write-ahead log when you are writing to CSVs! That's all the SQLite code is doing. SQLite does not selectively lock portions of the file for writes. It locks the entire file using the Operating System's own file handling services. SQLite is a file format. CSVs, XML and JSON are also huge in the dB marketplace. That doesn't make them RDBMSs.

A collection of CSVs can be used as a database.

Re: SQLite Is Serverless

#213

Post by lucideer »

bane wrote: post #164
lucideer wrote: post #156

Earlier quoted context omitted.

I'm not sure I understand your comment. What's the signifigance of it being "just a file format"? What are the aspects of "approaching it from a classic RDBMS angle" that are incompatible with it being "just a file format"? I've always seen it as "just a file format" myself, and I think I've always approached it "from a classic RDBMS angle", but I've never felt confused, and I don't see what I'm overcomplicating.

> and I think I've always approached it "from a classic RDBMS angle" Have you tried to figure out where to install the server or asked what the system requirements were for it? Have you grown concerned that once the system moves into production the O&M team won't know how to operate "yet another database"? Do you spend agonizing hours trying to figure out if it supports multithreaded connection pools for multi-user w…

> Have you tried to figure out where to install the server or asked what the system requirements were for it?

Not exactly, but asking "where to install the server & client and what are the system requirements for each" is not that different to asking "where to install the client and what are the system requirements for it", even when there is no server.

> Have you grown concerned that once the system moves into production the O&M team won't know how to operate "yet another database"?

Yes. Because the production concerns of SQLite are not nil.

> Do you spend agonizing hours trying to figure out if it supports multithreaded connection pools for multi-user writes?

Not agonizing hours, but it is just a slightly rephrased version of a valid question about SQLite w.r.t. concurrent file access (as with locks on file access for any file). Other commenters have brought this up in terms of multi-user access slowing down applications, and setting up intermediary DB access processes using IPC to facilitate this.

> The people asking these questions are stupid

I disagree

> no different than "CSV is serverless" or "JSON is server-less"

CSV and JSON lack any protocol or queryable interface: unless you're using some ancillary tool like `jq` as a comparison, CSV and JSON as filetypes are both "serverless" and "clientless" so not particularly comparable. An article on those would be quite different.

Re: SQLite Is Serverless

#214

Post by cma »

bane wrote: post #202
catalogia wrote: post #195

Earlier quoted context omitted.

> No! SQLite is not an embedded RDBMS. It's a file format. You keep saying it's a file format, but it's quite possible to use sqlite without persisting anything to a file at all. rc = sqlite3_open(":memory:", &db);

You can read/write CSV files, JSON, JPEG, WAV, MP3, MP4, etc. into memory as well. That doesn't make any of them an RDBMS. SQLite is a file format. It has a nice API and uses SQL as the domain language for read/write logic. If it didn't use SQL for the logic, would you still be confused?

SQLite is a whole library with complicated locking protocol options etc. and works with multiple files of different formats (e.g. separate transaction log).

Re: SQLite Is Serverless

#215

Post by bane »

catalogia wrote: post #205
bane wrote: post #202

Earlier quoted context omitted.

You can read/write CSV files, JSON, JPEG, WAV, MP3, MP4, etc. into memory as well. That doesn't make any of them an RDBMS. SQLite is a file format. It has a nice API and uses SQL as the domain language for read/write logic. If it didn't use SQL for the logic, would you still be confused?

Being a file format is only one aspect of what SQLite is. SQLite describes itself as "SQLite is a C-language library that implements a small, fast, self-contained, high-reliability, full-featured, SQL database engine." I think that is a better, more encompassing, description than "it's a file format" Edit: If SQLite lacked any ability to persist data to disk, it would still be very useful as an in-process SQL engine…

Sure, I can get behind that. The logic that handles the read/writing and SQL parsing is all part of the library for sure.

But that library is absolutely not required nor is SQL. One could build their own library that read/write to SQLite files entirely without any of that if they wished. Some people have done ground up rewrites in other languages other than C, but have more or less stuck with the same internal logic and the use of SQL as the read/write logic.

Re: SQLite Is Serverless

#216

Post by JasonFruit »

m0xte wrote: post #135
wenc wrote: post #95

Earlier quoted context omitted.

MS Access “allowed” it but depending on the version it could be quite problematic. We had a use case where Tableau connected to an Access file read only (but which another program used as a data store and wrote to it often) on an Windows file share, and once in a while the lock files would get screwy and we would have to manually delete the lock files to get things working again. Deleting lock files could be a huge c…

It was ok pre-windows 2000. I had access 97 it on 9 workstations on NT with never a problem! Moment office 2000 and windows 2000 came out it went to hell. Moved to SQL Server then. I am not sure what changed. It was quite frankly the most productive custom business software package I have ever used. Literally would take 10 people a week to do something custom you could do in an afternoon with 1 person in Access. I su…

We had a much heavier multiuser setup on Access 97, and it worked fine. We had the same breakdown as you did, and had to do a registry edit on all users' machines to keep it limping along until we could move to SQL Server, which was the right thing to do anyway.

Re: SQLite Is Serverless

#217

Post by cma »

bane wrote: post #200
lazyant wrote: post #183

Earlier quoted context omitted.

Sure. Or a light RDBMS where the main not supported case is concurrent writes.

No, SQLite provides no RDBMS functionality. It is not an RDBMS. It is a file format. Saying that a program that opens and reads/writes a file through a file format API is a light RDBMS turns almost every program in history into an RDBMS. If SQLite didn't force you to use SQL as the read/write logic, absolutely nobody would confuse it for an RDBMS. That's because it's a file format.

It can open multiple files (the transaction log).

Re: SQLite Is Serverless

#218

Post by Frost1x »

jtms wrote: post #159
Frost1x wrote: post #151

Earlier quoted context omitted.

"Serverless" is a garbage marketing term in general. It sounds sexy to nontechnical management folks who are used to hearing a bunch of expensive costs and the word "server" associated with them in some fashion. From that view, anything that gets rid of those pesky "servers" sounds like a win.

This is overly reductive. “Serverless” might be overhyped, but amongst devs and ops it is a perfectly good way to succinctly describe a very specific architecture with real merit. Paying for exactly the compute time i use is pretty sexy to me when I want to deploy and scale a side project.

What was wrong with "managed"? It's pretty generic and transfers across many domains and also, more fairly descriptive of what's going on.

Re: SQLite Is Serverless

#219

Post by bane »

hartator wrote: post #212
bane wrote: post #194

Earlier quoted context omitted.

Why not? Just roll your own write-ahead log when you are writing to CSVs! That's all the SQLite code is doing. SQLite does not selectively lock portions of the file for writes. It locks the entire file using the Operating System's own file handling services. SQLite is a file format. CSVs, XML and JSON are also huge in the dB marketplace. That doesn't make them RDBMSs.

A collection of CSVs can be used as a database.

Yes! If a database is defined as a place where I can read/write data, then any byte stream that I can read/write to is probably a database of a kind.

Re: SQLite Is Serverless

#220

Post by happymellon »

bane wrote: post #188
derefr wrote: post #172

Earlier quoted context omitted.

Consider: it’s totally possible to strip down Postgres until all you have left is an embedded RDBMS of the style of SQLite. (I’m not sure why nobody has done this yet, actually.) Would you call the result “just a file format”? Such an instance of “embedded Postgres” would still have a huge sprawling catalog (PG_DATA) directory attached to each use of it, so it wouldn’t be contained to a single file. But neither is SQ…

SQLite is a file format with a familiar API and uses SQL as the logic for searching/adding data to the file. Approach it exactly the same way you'd approach using a CSV file and all the confusion and overthinking about it goes away. Approach it as a stripped down RDBMS and you end up with all kinds of questions about support for this or that RDBMS familiar service. You can write your own SQLite file reader/writer. He…

File formats don't have an API.

> SQLite is a file format with a familiar API and uses SQL as the logic for searching/adding data to the file.

That description is for a library, not a file format. SQLite is a library, that saves to a convenient format and allows you to query the file using the SQL syntax. File formats dont have "logic".

You are arguing the equivalent that Word is a file format. While there is a Word file format, Word itself is an application.

Post reply on HN