Live data from Hacker News

SQLite is all you need for durable workflows

obeli.sk

191–200 of 413 posts

Re: SQLite is all you need for durable workflows

#191
post #157

Earlier quoted context omitted.

This could enforce dates are strings. They wanted to enforce dates are dates I thought.

create table events ( id integer primary key, name text not null, event_date text not null check ( -- YYYY-MM-DD event_date glob '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]' and date(event_date) is not null and date(event_date) = event_date ) ); In Python that raises this error if the date is invalid: sqlite3.IntegrityError: CHECK constraint failed: event_date glob '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'

[deleted]

Re: SQLite is all you need for durable workflows

#193
post #112

Earlier quoted context omitted.

I use postgres for very simple apps. I have a Dockerfile I use in my boilerplate repo. It takes a single make cmd for me to build, start and run migrations. Its as simple as using sqlite.

But now you have another process to babysit. How do you keep it healthy? And you have to ensure the client-server communication won't break. For me the main benefit of sqlite is that it's a library rather than an app.

> But now you have another process to babysit. How do you keep it healthy?

I've been assured by many HN users that running apps/sites on a single VPS requires near-zero maintenance or monitoring to achieve acceptable uptime 24/7/365 for years on end, sooooo...just pretend it will never fail like your main server process?

Re: SQLite is all you need for durable workflows

#194

Earlier quoted context omitted.

Not sure why there’s a debate at all. The discussion is on using SQLite instead of jq and markdown files. People got lost on a tangent! :)

No it's not.. the context of other threads on this post (which mention jq) do not apply here. How poorly coded are you?

Like how poorly did Jesus code my DNA? Ask him. By him I mean ChatGPT Jesus mode.

Re: SQLite is all you need for durable workflows

#195
post #9

Earlier quoted context omitted.

Could you give an example of a case where you'd use SQLite instead of jq or grep through Markdown?

My favorite lens on SQLite is that it is actually two things: 1. A robust durability implementation 2. A library of high performance data structure and algorithms The fact this it's SQL is nice, but those two attributes are what make it great. For example, I'm implement an in-process event log that I want to be durable. I started simple, but soon saw some edge cases and instead of playing whackamole I just swapped to…

mirrors my own experience creating a persistent event log. I started with JSON, then JSONL, etc until finally landing on SQLite.

Re: SQLite is all you need for durable workflows

#196
post #22

I don't understand this obsession with SQLite for real, production apps. SQLite is an embedded database, completely unsuitable for managing concurrency. This is what database _servers_ are for, e.g., Postgres, MySQL, etc. Their entire job is to allow you to modify data from multiple processes, on different machines, at the same time. This is a foundational principle of computer science. It seems to me that the "SQLit…

there is a difference between concurrency in a distributed environment and concurrency on a single machine across processes. SQLite is incredibly useful for the latter.

you seem like the inexperienced one to me..

Re: SQLite is all you need for durable workflows

#197
post #157

Earlier quoted context omitted.

This could enforce dates are strings. They wanted to enforce dates are dates I thought.

create table events ( id integer primary key, name text not null, event_date text not null check ( -- YYYY-MM-DD event_date glob '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]' and date(event_date) is not null and date(event_date) = event_date ) ); In Python that raises this error if the date is invalid: sqlite3.IntegrityError: CHECK constraint failed: event_date glob '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]'

I see. The strict tables page did not mention the date and time functions.

Python would show the 1st line always? Or the failed part?

This is unreasonable for a very common type I think.

Re: SQLite is all you need for durable workflows

#199
post #112

Earlier quoted context omitted.

But now you have another process to babysit. How do you keep it healthy? And you have to ensure the client-server communication won't break. For me the main benefit of sqlite is that it's a library rather than an app.

> But now you have another process to babysit. How do you keep it healthy? I've been assured by many HN users that running apps/sites on a single VPS requires near-zero maintenance or monitoring to achieve acceptable uptime 24/7/365 for years on end, sooooo...just pretend it will never fail like your main server process?

Ive been assured by many HN users that you must have 24/7/365 uptime for everything in case one of your 10 bi-monthly users decides to log on.

Re: SQLite is all you need for durable workflows

#200
post #22

I don't understand this obsession with SQLite for real, production apps. SQLite is an embedded database, completely unsuitable for managing concurrency. This is what database _servers_ are for, e.g., Postgres, MySQL, etc. Their entire job is to allow you to modify data from multiple processes, on different machines, at the same time. This is a foundational principle of computer science. It seems to me that the "SQLit…

Thing is SQLite scales better than both those network databases [1] if you're prepared to stick with one big machine (+ a standby). This is even more obvious when you start doing transactions processing an row locks across the network limit you to 1-3k TPS that you cannot scale out of (Pareto distribution is merciless). [1] - https://andersmurphy.com/2025/12/02/100000-tps-over-a-billio...

Seeing as I can get about 200K TPS from a networked DB in my environment, I have to question your setup here.

In the real world we are looking at things like RPO (recovery point objective) and RTO (recovery time objective). You need to consider HA and DR. It’s in these areas where SQLite does not scale.

That’s why I struggle to see the fit for SQLite in any sort of multi-user server environment. If you need the data to be durable, then the bigger DB’s have the tools. If you don’t need the data to be durable, just keep it in memory. I’m sure there are niches I am missing.

Post reply on HN