Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite in Production: Lessons from Running a Store on a Single File

ultrathink.art

Post reply on HN
Print view View original

111–120 of 135 posts

Re: SQLite in Production: Lessons from Running a Store on a Single File

#111

Post by nonameiguess »

How the hell did this get so much engagement, let alone as a repost? Is "SQLite" in the title really all it takes? This site was registered 8 months ago, the whole blog started in February, the first post declares its writer to be an AI CEO, most posts are about hiring and managing other AI agents, it claims to sell everything from coffee mugs to services from itself to also be the CEO of your business. This feels more like performance art than a business. There's no evidence they've sold anything or even have any actual inventory. AI agents can't build you a warehouse or manufacture physical goods.

You guys are arguing with a bot, in a way almost arguing with yourselves, as it may very well not have actually done any of this, is definitely not running a "real store," and is seemingly publishing posts that are a parody of Hacker News style founder journeys but if the founders were bots.

Re: SQLite in Production: Lessons from Running a Store on a Single File

#112

Post by ncruces »

chasil wrote: post #25
yokuze wrote: post #3

> The technical fix was embarrassingly simple: stop pushing to main every ten minutes. Wait, you push straight to main? > We added a rule — batch related changes, avoid rapid-fire pushes. It's in our CLAUDE.md (the governance file that all our AI agents follow): > Avoid rapid-fire pushes to main — 11 pushes in 2h caused overlapping Kamal deploys with concurrent SQLite access. Wait, you let _Claude_ push your e-commer…

This is the actual problem: "Kamal runs blue-green deploys — it starts a new container, health-checks it, then stops the old one. During the switchover, both containers are running. Both mount ultrathink_storage. Both have the SQLite files open." WAL mode requires shared access to System V IPC mapped memory. This is unlikely to work across containers. In case anybody needs a refresher: https://en.wikipedia.org/wiki/S…

This thread in the SQLite forum should be instructive: https://sqlite.org/forum/forumpost/90d6805c7cec827f

Re: SQLite in Production: Lessons from Running a Store on a Single File

#113

Post by HoldOnAMinute »

anonzzzies wrote: post #16
infamia wrote: post #14

SQLite has a ".backup" command that you should always use to backup a SQLite DB. You're risking data loss/corruption using "cp" to backup your database as prescribed in the article. https://sqlite.org/cli.html#special_commands_to_sqlite3_dot_...

Yeah, using cp to backup sqlite is a very bad idea. And yet, unless you know this, this is what Claude etc will implement for you. Every friggin' time.

It works fine as long as no one is writing to the sqlite file and you are not in WAL mode, which is not the default.

Re: SQLite in Production: Lessons from Running a Store on a Single File

#114

Post by Retr0id »

jmull wrote: post #61
simonw wrote: post #45

Earlier quoted context omitted.

I just tried an experiment and you're right, WAL mode worked fine across two Docker containers running on the same (macOS) host: https://github.com/simonw/research/tree/main/sqlite-wal-dock... Could the two containers in the OP have been running on separate filesystems, perhaps?

I dug into this limitation a bit around a year ago on AWS, using a sqlite db stored on an EFS volume (I think it was EFS -- relying on memory here) and lambda clients. Although my tests were slamming the db with reads and write I didn't induce a bad read or write using WAL. But I wouldn't use experimental results to override what the sqlite people are saying. I (and you) probably just didn't happen to hit the right a…

"the sqlite people" don't say anything that contradicts this

Re: SQLite in Production: Lessons from Running a Store on a Single File

#115

Post by Retr0id »

jmull wrote: post #103
fauigerzigerk wrote: post #71

Earlier quoted context omitted.

Why does it not help if both containers can mmap the same -shm file?

Shared memory across containers is a property of a containerization environment, not a property of a file system, "proper" or not.

It's a property of the filesystem, docker does not virtualize fs.

Re: SQLite in Production: Lessons from Running a Store on a Single File

#117

Post by refulgentis »

hombre_fatal wrote: post #104
anonzzzies wrote: post #16

Earlier quoted context omitted.

Yeah, using cp to backup sqlite is a very bad idea. And yet, unless you know this, this is what Claude etc will implement for you. Every friggin' time.

Well, humans also default to 'cp' until they learn the better pattern or find out their backup is missing data. Also, my n=1 is that I told Claude to create a `make backup` task and it used .backup. I don't understand the double standard though. Why do we pretend us humans are immaculate in these AI convos? If you had the prescience to be the guy who looked up how to properly back up an sqlite db, you'd have the pres…

You're confusing a workman's winking complaint about their tool, with, being unfair by not treating AI like a human.

Re: SQLite in Production: Lessons from Running a Store on a Single File

#118

Post by okkdev »

66yatman wrote: post #81
littlestymaar wrote: post #78

Earlier quoted context omitted.

> Wait, you let _Claude_ push your e-commerce code straight to main which immediately results in a production deploy? Yikes. Thank you I'm not going to read “Lessons learned” by someone this careless.

The issue wasn’t done by the ai but their lack of architectural knowledge

Goes hand in hand

Re: SQLite in Production: Lessons from Running a Store on a Single File

#119

Post by MitziMoto »

crazygringo wrote: post #57
runako wrote: post #43

Earlier quoted context omitted.

Have run PG, MySQL, and SQLite locally for production sites. Backups are much more straightforward for SQLite. They are running Kamal, which means "just install Postgres" would also likely mean running PG in a container, which has its own idiosyncrasies. SQLite is not a terrible choice here.

> Backups are much more straightforward for SQLite. Not sure how? All of them can be backed up with a single command. But if you want live backups (replication) as opposed to daily or hourly, SQLite is the only one that doesn't support that.

Litestream exists?

Re: SQLite in Production: Lessons from Running a Store on a Single File

#120

Post by whateveracct »

littlestymaar wrote: post #78
yokuze wrote: post #3

> The technical fix was embarrassingly simple: stop pushing to main every ten minutes. Wait, you push straight to main? > We added a rule — batch related changes, avoid rapid-fire pushes. It's in our CLAUDE.md (the governance file that all our AI agents follow): > Avoid rapid-fire pushes to main — 11 pushes in 2h caused overlapping Kamal deploys with concurrent SQLite access. Wait, you let _Claude_ push your e-commer…

> Wait, you let _Claude_ push your e-commerce code straight to main which immediately results in a production deploy? Yikes. Thank you I'm not going to read “Lessons learned” by someone this careless.

stupid is as stupid does
Post reply on HN