Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Is Serverless

sqlite.org

Post reply on HN
Print view View original

351–360 of 453 posts

Re: SQLite Is Serverless

#351

Post by bane »

kgwgk wrote: post #263
bane wrote: post #149

I love SQLite, but people approach it from a classic RDBMS angle which confuses them. Here's the deal: SQLite is a file format with a nice API that uses SQL as the paradigm for reading/writing to the file. That's it. Stop overthinking it. Can you write a microservice that stores its data in a big JSON file that you've built some code around to read/write to? Yes. It's just a file, but you have to build all the read/w…

> SQLite is a file format with a nice API that uses SQL as the paradigm for reading/writing to the file. is a collection of bits with a nice interface that uses a query language as the paradigm for reading/writing data.

No, an RDBMS is a piece of software for managing databases in the relational model, access to the databases (such as users and permissions) and provides services such as a server, connection pooling, and so on.

SQLite provides almost no RDBMS features.

This isn't just semantics.

A car is not an engine. A fork is not a kitchen. A SQLite file is not a DBMS.

Re: SQLite Is Serverless

#352

Post by mbreese »

cle wrote: post #329
jiofih wrote: post #322

Earlier quoted context omitted.

This such an air-headed comment to make. You must realize that page describing how SQLite is “serverless” has been up probably longer than your entire adult life. It is important in its context, they are not trying to “redefine” (lol) the term.

It doesn’t matter. It is not the generally accepted definition of serverless. The meaning of a word is based on how it is generally used, and almost nobody means this when they use the word “serverless”. At this point trying to use the word in this way just creates a bunch of unnecessary confusion. Call it something else so we can move on to more important (and clear) discussions. (Also there are a lot of assumptions…

Their usage of serverless predates all of these cloud application offerings. Just because a lot of people in this particular echo chamber like to think of serverless as meaning one thing, there are plenty of others who have been using the same word to mean something different for quite a bit longer.

Moreover, the cloud as a service definition isn’t even accurate. There still is a server — it’s just not one the developer has to worry about.

Re: SQLite Is Serverless

#353

Post by bane »

cma wrote: post #236
bane wrote: post #224

Earlier quoted context omitted.

Almost any modern programming language can open multiple files.

At a minimum it is multiple file formats. But really it is multiple file formats + a fairly intricate library for dealing with them along with locks etc. According to their home page the SQLite database file format is a file format, and SQLite is a library.

No but close. A SQlite file is a single format. The journal file and the WAL file are different formats used for bookkeeping by the library to attempt to be ACID compliant. The library implements some complex logic to ensure this, but reading/writing a SQLite file does not require any of this.

You could write your own code tomorrow that reads/writes SQLite files but does not produce, read, or write WAL or Journal files. So long as the resultant SQLite file follows the specification, it can be read by any other piece of software that implements the specification, such as the SQLite library or SQLJet (https://sqljet.com/).

Re: SQLite Is Serverless

#354

Post by gwd »

bane wrote: post #330
gwd wrote: post #240

Earlier quoted context omitted.

> No, SQLite provides no RDBMS functionality. I think your comment would be more comprehensible if you gave some examples of the kind of functionality you think is missing. I don't know what you mean by "RDBMS", but JSON and XML don't do joins, don't do views, don't do efficient query plans, and so on. It's either ignorance or obstinacy to say SQLite is just a file format.

> ...but JSON and XML don't do joins, don't do views, don't do efficient query plans, and so on. It's either ignorance or obstinacy to say SQLite is just a file format. Sure they do. If you write the logic to do so, and put it behind a nice API, you can make all of this come true. In fact, millions of people every day do joins with JSON and XML in their code every day. You can probably just use Apache Drill as the "l…

First of all, you still haven't answered the question: What is it that an RDBMS has that SQLite doesn't have?

> It's no more an RDBMS than a .docx file is.

Thanks for the idea. Your argument is like saying this:

Microsoft Word is not a word processor -- it's a file format.

I mean, yes, Word has a file format; but it's far more than just a format specification.

> Sure they do. If you write the logic to do so,

Right, but you don't have to write the logic if you're using SQLite. That's the point. SQLite is a library, which provides a way to do SQL operations on data. Like Word, SQLite has a file format, but it is far more.

I just don't get where you're coming from. Do you not know that the SQLite libray can actually do complex SQL queries on data? Or do you think that people shouldn't do that for some reason? Or do you just value SQL queries so little that you don't see any difference between being able to do complex queries and doing `file.Write(json.Marshal(data))`? What is it you're trying to accomplish with this line of argument?

Re: SQLite Is Serverless

#355

Post by mbreese »

cle wrote: post #336
coldtea wrote: post #328

Earlier quoted context omitted.

> Everyone knows what "serverless" means at this point, and it's not an embedded DB. Serverless is a marketing term at this point (as it was when it started). This post brings some welcome definitions and expands it to something that has many of the same attributes but wasn't appreciated as such.

Serverless refers to platforms with specific and well-understood properties, such as managed servers, autoscaling, usage-based pricing, etc. What parts of SQLite have these properties?

And for others the word “serverless” literally means without a server. As in... a library. It’s been a well used definition for many years.

Re: SQLite Is Serverless

#356

Post by FridgeSeal »

sly010 wrote: post #300
nicholaides wrote: post #185

I think a good under-appreciated use case for SQLite is as a build artifact of ETL processes/build processes/data pipelines. Seems like lot of people's default, understandably, is to use JSON as the output and intermediate results, but if you use SQLite, you'd have all the benefits of SQL (indexes, joins, grouping, ordering, querying logic, and random access) and many of the benefits of JSON files (SQLite DBs are jus…

I have been using SQLite as a format to move data between steps in a complicated batch processing pipeline. With the right pragmas it is both faster and more compact than JSON. It is also much more "human readable" than gigabytes of JSON. I only wish there was a way to open an http-fetched SQLite database from memory so I don't have to write it to disk first.

This is a amazing, I think you may have just solved and headed-off a huge number odd problems for me.

Could you talk more about what Pragmas you’ve been using and why?

Re: SQLite Is Serverless

#357

Post by elgenie »

zzo38computer wrote: post #350
nicholaides wrote: post #185

I think a good under-appreciated use case for SQLite is as a build artifact of ETL processes/build processes/data pipelines. Seems like lot of people's default, understandably, is to use JSON as the output and intermediate results, but if you use SQLite, you'd have all the benefits of SQL (indexes, joins, grouping, ordering, querying logic, and random access) and many of the benefits of JSON files (SQLite DBs are jus…

I don't know what is "ETL" meaning here, although SQLite does include a JSON extension to read/write JSON data too, so you can use SQL and JSON together if necessary.

It's an acronym for munging some data

ETL = extract, transform, load

Re: SQLite Is Serverless

#358

Post by manigandham »

filleduchaos wrote: post #335
manigandham wrote: post #312

Earlier quoted context omitted.

Not in this thread apparently. I guarantee nobody here as ever used the term "serverless" over "embedded" or "in-process" in their entire career but apparently the purity and nostalgia of SQLite overrides it.

This article was written in 2007, so yes, people have been using the word "serverless" before managed service providers decided to take it up as a buzzword.

Nobody used it before. Even the about page says: "SQLite is an embedded SQL database engine."

"serverless" is a marketing term, then and now. Not sure why there's such a big defense of it. If you have to argue this much over the provenance of a term that clearly isn't used the same way today then it's a good sign that it's not very useful.

Re: SQLite Is Serverless

#359

Post by zzo38computer »

elgenie wrote: post #357
zzo38computer wrote: post #350

Earlier quoted context omitted.

I don't know what is "ETL" meaning here, although SQLite does include a JSON extension to read/write JSON data too, so you can use SQL and JSON together if necessary.

It's an acronym for munging some data ETL = extract, transform, load

Yes, if that is what you are trying to do, I think SQLite is good. SQLite command shell also has a .import command to read data from a file, and you can also import into a view and use triggers to process the data (this is something I have done). And there is also functions and virtual tables for JSON, and you can load extensions (written in C) to add additional functions, virtual tables, collations, etc. So for many cases, SQLite is useful.

Re: SQLite Is Serverless

#360

Post by crtlaltdel »

nicholaides wrote: post #185

I think a good under-appreciated use case for SQLite is as a build artifact of ETL processes/build processes/data pipelines. Seems like lot of people's default, understandably, is to use JSON as the output and intermediate results, but if you use SQLite, you'd have all the benefits of SQL (indexes, joins, grouping, ordering, querying logic, and random access) and many of the benefits of JSON files (SQLite DBs are jus…

this is inspiring, i cannot believe i had not considered this before!
Post reply on HN