Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Is Serverless

sqlite.org

Post reply on HN
Print view View original

61–70 of 453 posts

Re: SQLite Is Serverless

#61

Post by SQLite »

XelNika wrote: post #9
manigandham wrote: post #6

Earlier quoted context omitted.

Where does it say 2007? Regardless it's vague and irrelevant material. Classic serverless has always been known as "in-process". The attempt at differences seems like adding marketing fluff rather than just removing the page entirely.

It doesn't say 2007 anywhere, but the page is clearly at least that old. https://web.archive.org/web/20071115173112/https://www.sqlit... I also found material from 2004 with the term. https://www.tcl.tk/community/tcl2004/Presentations/D.Richard... > Regardless it's vague and irrelevant material It's not vague at all, the term makes sense to distinguish "in-process" from client/server models. The page also includes an…

Complete history of the document in question is here: https://www.sqlite.org/docsrc/finfo?name=pages/serverless.in...

It was, indeed, written in 2007, but based on ideas that predate that.

Re: SQLite Is Serverless

#62

Post by SQLite »

k__ wrote: post #56
masklinn wrote: post #45

Earlier quoted context omitted.

The page is more that 10 years old. And it uses "serverless" in the sence of server-less aka not having a server. Which is pretty damn sensible terminology, as opposed to the newer "serverless" meaning" runs on some utility server outside of your care or control".

I know, but have you anyone calling "embedded databases" "serverless databases" in the last 10 years? It's like someone found this page and felt very smart about it, because they stick it to the serverless crowd.

"embedded database" does not imply "serverless database". Other embedded RDBMSes run servers; they just run the server as a separate thread rather than a separate process. SQLite is different in that there is no separate thread of control. SQLite runs in the same thread as the application that calls it. There is no separate thread hanging around to clean up or handle background tasks after an SQLite function call returns.

Re: SQLite Is Serverless

#63

Post by oldgradstudent »

lxgr wrote: post #36
oldgradstudent wrote: post #23

> Of those that are serverless, SQLite is the only one known to this author that allows multiple applications to access the same database at the same time. IIRC, MS Access allowed that, which explained a lot of its popularity.

Good point. Berkeley DB also supports multiple processes accessing a database concurrently, as far as I know. I was wondering if the authors were referring to SQL-like databases, but MS Access seems to be one?

They were indeed referring to SQL servers. The paragraph starts with

> Most SQL database engines are client/server based.

I skipped it for brevity.

Re: SQLite Is Serverless

#64

Post by Gys »

The main problem that I see for using sqlite is exactly for it being 'Classic Serverless'. Because how does one keep an up te date backup? Deploying to Heroku, Dokku, AWS Lambda and such means the sqlite file will be lost on a crash or new deploy. Even a VM can crash. Export to S3 on every write? Maybe if changes do not happen often, so only for specific use cases (and actually I think you should just generate static html in that case).

I use sqlite for local tests for cases where the live app has a 'neo-serverless' database. It is very, very fast so the tests run almost instantly.

Re: SQLite Is Serverless

#65

Post by Gys »

threatripper wrote: post #55

In case SQLite is not enough and you need redundant servers or clustering, there's also database servers that use SQLite as storage engine: http://rqlite.com/ https://dqlite.io/

So you run an app with sqlite on one server and sync to sqlite on another server? What would be the benefit to using a separate db server like in the 'neo-serveless' setup?

Re: SQLite Is Serverless

#66

Post by closeparen »

geokon wrote: post #24

a bit tangential, but when do you move form using in-application data structures (maps, trees, vector/arrays) to using a database? Is it basically when the data doesn't fit in memory? I've been programming for almost a decade and I've never come across needing a database... (for context, it's ten years without anything web related) I'm interested in them and I'd love to learn SQL but I can't even think of a use case…

SQL queries tend to be much smaller than their equivalent data-structure traversal procedures. This can be beneficial even when you still want the data in your process's address space, hence embedded database engines like sqlite. Libraries like Linq can also provide the same expressive power over a programming language's native objects and collections.

As to why you'd want a separate DB server process: long lived mutable structures tend to drift into unexpected states, which is why the standard computer troubleshooting procedure since forever is to restart. Database management systems specialize in not having that problem. You deploy one that's widely used and hardened over many years, change it infrequently, and operate it with great care. Or pay a cloud provider to do that.

Then everything else gets to outsource the burden of persistence to it, and the vast majority of the workloads you develop and operate are stateless. These are drastically more convenient and more resilient to mistakes. They're effectively "restarted" with each API request, and you can stand them up, tear them down, migrate them to new machines, etc. as much as you want without fear of data loss.

Re: SQLite Is Serverless

#67

Post by threatripper »

Gys wrote: post #65
threatripper wrote: post #55

In case SQLite is not enough and you need redundant servers or clustering, there's also database servers that use SQLite as storage engine: http://rqlite.com/ https://dqlite.io/

So you run an app with sqlite on one server and sync to sqlite on another server? What would be the benefit to using a separate db server like in the 'neo-serveless' setup?

No, in this case you always have to use rqlite/dqlite because they manage the network synchronization. They use SQLite as storage engine (one SQLite database per server instance).

Re: SQLite Is Serverless

#68

Post by Gys »

threatripper wrote: post #67
Gys wrote: post #65

Earlier quoted context omitted.

So you run an app with sqlite on one server and sync to sqlite on another server? What would be the benefit to using a separate db server like in the 'neo-serveless' setup?

No, in this case you always have to use rqlite/dqlite because they manage the network synchronization. They use SQLite as storage engine (one SQLite database per server instance).

I understand that in those cases rqlite/dqlite is used. But that it just a technical detail. My point is that I am running two servers: one with the app and Xqlite and another one with Xqlite.

In case of a neo-serveless setup, I also have two servers: one with the app, the other one with the db server.

So what are the benefits of the Xqlite setup? I looked into that before and for one thing, Xqlite is slower (obviously) then just sqlite. So speed is not a key benefit. I also will have to manage both servers myself.

At least for a seperate db server I have the benefit that I can buy that as a service, incl management, backups and such.

Re: SQLite Is Serverless

#69

Post by bob1029 »

geokon wrote: post #57
bob1029 wrote: post #39

Earlier quoted context omitted.

I have found that you can go for a disturbingly-long period of time using primitive schemes like LINQ Objects JSON to persist your business data before things start to get hairy. Personally, I would say 10 megs of persisted data is about the upper limit before I am going to start reaching for SQLite. If you start to get clever with schemes like one file per serialized entity, you could potentially avoid using a 3rd p…

will you store large objects like images or audio file directly in SQL?

In most cases I will. Pulling a blob out of a row is a lot faster than opening an additional file handle. There aren't really any downsides to this either unless you are running table scans. SQLite does support indexes (even full-text) and they do work miracles so do use them when necessary.

I would go so far as to argue that SQLite could be used to store all of the assets for any large piece of software (I.e. a AAA game). This would probably wind up faster and more reliable than most alternative solutions out there today.

Re: SQLite Is Serverless

#70

Post by coleifer »

Gys wrote: post #64

The main problem that I see for using sqlite is exactly for it being 'Classic Serverless'. Because how does one keep an up te date backup? Deploying to Heroku, Dokku, AWS Lambda and such means the sqlite file will be lost on a crash or new deploy. Even a VM can crash. Export to S3 on every write? Maybe if changes do not happen often, so only for specific use cases (and actually I think you should just generate static…

>how do you back up a file
Post reply on HN