Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Is Serverless

sqlite.org

Post reply on HN
Print view View original

331–340 of 453 posts

Re: SQLite Is Serverless

#331

Post by filleduchaos »

idclip wrote: post #301
filleduchaos wrote: post #178

Earlier quoted context omitted.

That is not the definition of serverless in question.

If you examine it closely, youll see that they have alot incommon, and that the author defined the term neo-serverless to attempt to adress this - both definitions share the fact that multiple applications (in clientless form) can access the database. He even gives amazon S3 as an example. For neo serverless. I agree with you that it take a bit to marry both, but the stretch isnt far. Im also avoiding criticising the…

> both definitions share the fact that multiple applications (in clientless form) can access the database

That is like the least relevant thing on that entire page, to be frank. "SQLite is Serverless" is specifically referring to SQLite being an embeddable library that runs in the same process (and same thread, even) as your application vs. the client-server architecture (database in another process, with communication via a port) that DBMSes like MySQL and co have.

> Im also avoiding criticising the Author for not sticking to the main def

The "main definition" (i.e. the web dev buzzword) came into being years after this post was written.

Re: SQLite Is Serverless

#332

Post by DelightOne »

sly010 wrote: post #300
nicholaides wrote: post #185

I think a good under-appreciated use case for SQLite is as a build artifact of ETL processes/build processes/data pipelines. Seems like lot of people's default, understandably, is to use JSON as the output and intermediate results, but if you use SQLite, you'd have all the benefits of SQL (indexes, joins, grouping, ordering, querying logic, and random access) and many of the benefits of JSON files (SQLite DBs are jus…

I have been using SQLite as a format to move data between steps in a complicated batch processing pipeline. With the right pragmas it is both faster and more compact than JSON. It is also much more "human readable" than gigabytes of JSON. I only wish there was a way to open an http-fetched SQLite database from memory so I don't have to write it to disk first.

What pragmas do you use? It sounds amazing!

Re: SQLite Is Serverless

#333

Post by catalogia »

swiley wrote: post #289
nicholaides wrote: post #185

I think a good under-appreciated use case for SQLite is as a build artifact of ETL processes/build processes/data pipelines. Seems like lot of people's default, understandably, is to use JSON as the output and intermediate results, but if you use SQLite, you'd have all the benefits of SQL (indexes, joins, grouping, ordering, querying logic, and random access) and many of the benefits of JSON files (SQLite DBs are jus…

I was under the impression SQLite files were not supposed to be moved across architectures.

Thankfully that's not the case:

> The SQLite file format is cross-platform. A database file written on one machine can be copied to and used on a different machine with a different architecture. Big-endian or little-endian, 32-bit or 64-bit does not matter. All machines use the same file format. Furthermore, the developers have pledged to keep the file format stable and backwards compatible, so newer versions of SQLite can read and write older database files.

https://www.sqlite.org/different.html

Re: SQLite Is Serverless

#334

Post by filleduchaos »

hn_throwaway_99 wrote: post #294
cle wrote: post #196

> It is important to understand these two different definitions for "serverless". When a database claims to be "serverless", be sure to discern whether they mean "classic serverless" or "neo-serverless". It's really not important to understand that distinction, because this author seems to be the only one making it. Everyone knows what "serverless" means at this point, and it's not an embedded DB.

Exactly. If anything, it seems like the author wants to take the word "embedded", which everyone understands, and somehow redefine it to "serverless" which everyone also understands, and which this is not.

Yes, of course, the author took a time machine to 2007 to try and co-opt your buzzword.

Re: SQLite Is Serverless

#335

Post by filleduchaos »

manigandham wrote: post #312
loukrazy wrote: post #139

Isn’t it easier to say “embedded” and “severless”

Not in this thread apparently. I guarantee nobody here as ever used the term "serverless" over "embedded" or "in-process" in their entire career but apparently the purity and nostalgia of SQLite overrides it.

This article was written in 2007, so yes, people have been using the word "serverless" before managed service providers decided to take it up as a buzzword.

Re: SQLite Is Serverless

#336

Post by cle »

coldtea wrote: post #328
cle wrote: post #196

> It is important to understand these two different definitions for "serverless". When a database claims to be "serverless", be sure to discern whether they mean "classic serverless" or "neo-serverless". It's really not important to understand that distinction, because this author seems to be the only one making it. Everyone knows what "serverless" means at this point, and it's not an embedded DB.

> Everyone knows what "serverless" means at this point, and it's not an embedded DB. Serverless is a marketing term at this point (as it was when it started). This post brings some welcome definitions and expands it to something that has many of the same attributes but wasn't appreciated as such.

Serverless refers to platforms with specific and well-understood properties, such as managed servers, autoscaling, usage-based pricing, etc.

What parts of SQLite have these properties?

Re: SQLite Is Serverless

#337

Post by bane »

travbrack wrote: post #305
bane wrote: post #201

Earlier quoted context omitted.

They're using SQLite as the file format for persisting data. It's a great file format for this. You can even build an RDBMS over top of uncompressed WAV files if you want. It doesn't make WAV files RDBMs.

But SQLite has rdbms features. I remember being able to show tables and do SQL queries in SQLite DBs.

Those are relational database features, not RDBMS features. The SQLite file format specifies a way to organize, store and retrieve fairly arbitrary data using a relational database model.

The library knows how to handle SQL to describe the work being done. The SQL is optional, one can, with the specification, read/write SQLite files in many other ways.

There are almost no RDBMS features in SQLite. There are many many other file formats that store data that offer features that are very similar to SQLite files: indexes, journaling, write logging, etc.

Thought question, you can ask a .tar file to give you a listing of what files and directories are stored in the file. Are .tar files RDBMSs? Consider:

1 - If you consider each file in the .tar file a "table" you can get list of tables.

2 - If each file follows a regular format, say JSON, you can search the "tables" by extracting the file and grepping it or using jq or whatnot.

3 - You can store a special file that is an index of some kind that lets you know in which file some data is, or even where in the file it is.

4 - You can build logic such that when you want to do other CRUD operations you can record a journal and a write ahead log to help build in ACID compliance.

5 - You can build buffer logic to support Write-ahead-logging, transactions and what not to improve performance.

Are .tar files RDBMSs? Trivially no.

But maybe, if you do all these things, you've invented a terrible database and database engine.

However, you need to build a server, user access controls, connection pooling, import/export tools, partitioning, clustering, etc. before you start to arrive at an RDBMS that uses this engine.

Re: SQLite Is Serverless

#338

Post by cle »

mikestew wrote: post #317
cle wrote: post #196

> It is important to understand these two different definitions for "serverless". When a database claims to be "serverless", be sure to discern whether they mean "classic serverless" or "neo-serverless". It's really not important to understand that distinction, because this author seems to be the only one making it. Everyone knows what "serverless" means at this point, and it's not an embedded DB.

Everyone know that “serverless” anything doesn’t run on a server , be it AWS or Azure instances or what have you. That would be both ironic and silly, like someone used the wrong word or something. Serverless databases and what have you have been around longer than the current batch of folks trying to redefine things (or more charitably, ran out names to call things). Like or not, there is a distinction even if the o…

Nobody called them serverless DBs, and nobody does.

There is a vocal minority of reductionists here on HN who dismiss the accepted definition of serverless because the literal meaning doesn’t make sense. It’s just noise though. “Serverless” does have a specific meaning and it’s not “there are no servers anywhere”.

I was around when there was no serverless. Things change, new words arise. Time for us to get with the times.

Re: SQLite Is Serverless

#339

Post by mcpeepants »

yegle wrote: post #276
skrebbel wrote: post #268

Earlier quoted context omitted.

I agree! Also the cryptography people should stop calling their hobby "crypto", everybody knows crypto means bitcoin and stuff. Snobs. (here's SQLite's "Serverless" page the way it was in 2007: https://web.archive.org/web/20071115173112/https://www.sqlit... )

I'm not sure if this comment missed a /s somewhere...

it wouldn't be wrong to include one, but it didn't necessarily need it. the link in parentheses serves the same purpose.

Re: SQLite Is Serverless

#340

Post by bane »

wvenable wrote: post #260
bane wrote: post #202

Earlier quoted context omitted.

You can read/write CSV files, JSON, JPEG, WAV, MP3, MP4, etc. into memory as well. That doesn't make any of them an RDBMS. SQLite is a file format. It has a nice API and uses SQL as the domain language for read/write logic. If it didn't use SQL for the logic, would you still be confused?

All other RDBMS products also store their information inside files. I can take a mdb file from SQL Server and copy it to another server and attach it there. SQLite is obviously a library that stores it's data to a file like millions of other libraries. SQLite is to it's data file the way LAME is to MP3. LAME is not the file format, it's the library.

> All other RDBMS products also store their information inside files.

No they don't. There are many RDBMS (and other DBMSs) that do not store their information inside files.

> I can take a mdb file from SQL Server and copy it to another server and attach it there.

Yes, SQL Server is an RDBMS.

MDB files are files defined by a file format. This format defines something called a database. There are many libraries and other pieces of software than can read/write mdb files (https://jackcess.sourceforge.io/)

SQLite files are files defined by a file format (https://www.sqlite.org/fileformat.html). You can write code that reads and writes to a SQLite database file without using the library. So long as you follow the format specification, you will produce or be able to read from an aribtrary sqlite file that is produced by any code that implements the specification.

The SQLite library is a reference implementation of the specification as well as some sample logic for reading/writing to the files and using SQL to define the interactions with the file. The SQLite library is not required for this, nor is SQL, nor is the logic for ACID compliance.

SQLite is not an RDBMS and offers almost nothing that an RDBMS (or DBMS) might offer. If you wish to use SQLite files with an RDBMS you either have to build the RDBMS yourself, or find somebody else who's done so.

The more you consider the SQLite a file format, the easier it becomes to work with and understand. The more you try to consider it an RDBMS the less it makes sense.

Just because SQL is involved, doesn't mean it's an RDBMS.

Post reply on HN