Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite the only database you will ever need in most cases (2021)

unixsheikh.com

Post reply on HN
Print view View original

181–190 of 378 posts

Re: SQLite the only database you will ever need in most cases (2021)

#181

Post by guggle »

irskep wrote: post #103
guggle wrote: post #96

Earlier quoted context omitted.

I don't know why this is downvoted, it's probably the best advice on this page. Learning a few things about deployment definitely helps to make better software.

Well, he did cherry-pick one joking comment out of a well-intentioned post in order to say I have a bad attitude, so that's not exactly welcoming. And very patronizing, considering I have actually done that kind of work with Salt Stack a few years ago, and just don't want to bother with it at the moment because I have other priorities. I learn new things every day, and one day, Docker was that thing. I am now better…

Sure, that comment came a bit rough. But I'm not even sure it was triggered by the Docker comment, rather than the developer/release engineer opposition. At least that's how I see it. And maybe I just don't see the added value of PaaS because I've been spinning up public web servers for too long now ;-)

Re: SQLite the only database you will ever need in most cases (2021)

#182

Post by culopatin »

berdon wrote: post #151
JohnBooty wrote: post #134

Earlier quoted context omitted.

You don't need to be a full-blown sysadmin to know how to do basic deployments, and learning these things will make you a better developer. Yes. True. However, it is also true that we have a limited amount of time per week and a limited number of weeks on Earth. Time spent learning sysadmin-y stuff is less time spent mastering developer-y stuff. Think about what it means to be a "full stack" engineer in 2023: - Unix…

Fwiw, I’ve used all of those in my career, extensively here and there. I’ve written many thousands of lines of bash scripts. Even written my own php-for-bash-script style code tags that support arbitrary shells. I’ve written my own log based distributed kv store, gone down the YouTube trail of writing my own db, gone through the angular and react iterations, deep dove in docker, docker compose, k8s, crds and custom h…

Good for you guys, but there is always someone doing more, always someone better, why start a pissing contest? You turned someone’s comment on why deploying SQLite is not trivial into an opportunity to tap yourself in the back so we can all say “wow look at berdon”, but in the end, it doesn’t really matter.

Re: SQLite the only database you will ever need in most cases (2021)

#183

Post by stirfish »

irskep wrote: post #148
JohnBooty wrote: post #134

Earlier quoted context omitted.

You don't need to be a full-blown sysadmin to know how to do basic deployments, and learning these things will make you a better developer. Yes. True. However, it is also true that we have a limited amount of time per week and a limited number of weeks on Earth. Time spent learning sysadmin-y stuff is less time spent mastering developer-y stuff. Think about what it means to be a "full stack" engineer in 2023: - Unix…

I really appreciate you taking the time to respond with these details. I do go deep on application-level tech (CSS, HTML, JS with and without frameworks, iOS dev, etc) and it's always disheartening for people to claim that choosing not to focus on specific technologies makes you a worse human being. The other thing that's going completely over people's heads here is that I made this repo with the intent of helping ot…

>choosing not to focus on specific technologies makes you a worse human being.

maybe that'll make you a worse engineer, but engineers are not human beings.

Re: SQLite the only database you will ever need in most cases (2021)

#184

Post by irskep »

tptacek wrote: post #149
bravura wrote: post #35

I would love to use SQLite for all my Django webapps that have only several simultaneous users, but this article suggests there are too many footguns for me to be able to do that. Is there a "using SQLite for a multi-threaded webapp for dummies" package that does all the config I need so I can just drop it in and go and not tune anything? Paging fly.io founders etc! If I have a persistent volume can my fly.io apps us…

You can certainly put a SQLite database on a Fly.io persistent volume. You can use Litestream to keep that SQLite database backed up, transaction-by-transaction, on other volumes or in S3; soon, you'll be able to use LiteFS to replicate that SQLite database as well.

I'm extremely excited about LiteFS. Looking forward to trying it out once write forwarding lands. I think it has the potential to make hobby projects scale to zero but also serve reasonable traffic, which is exactly what I'm looking for. Thanks so much for hiring Ben!

Re: SQLite the only database you will ever need in most cases (2021)

#185

Post by nijave »

chillfox wrote: post #76
jxf wrote: post #28

> The only time you need to consider a client-server setup is: Where you have multiple physical machines accessing the same database server over a network. In this setup you have a shared database between multiple clients. Am I misunderstanding this or is this not the vast, vast majority of all cases?

Most of the time people separate the app and database into two different VMs that the infrastructure team then runs on the same box. edit: This is done not because of any considered technical reasons, but because that's how one learned to deploy apps.

Technical reason is to limit blast radius, easier permission management, easier scaling.

For blast radius, you have things like patching (you don't want to botch the database accidentally updating a shared dependency with your app), resource management (don't want your DB to eat all the RAM or I/O and kill your app), if someone botches maintenance there's less to break at once

You don't have to use VMs but they're one way to do it. Container orchestrators achieve many of the same goals and automatically restart workloads if a physical machine fails, too

Re: SQLite the only database you will ever need in most cases (2021)

#186

Post by benhoyt »

irskep wrote: post #29

This sentiment pops up regularly on HN, and I've seen at least one article per month for the past few months, but the trouble is, none of them seem to help you actually deploy it. They assume you're comfortable spinning up public web servers. If you want to use a PaaS to deploy an app, because you don't want to spend your time learning to be a sysadmin, then all the tutorials are going to put you on the Postgres path…

Just this morning I was wondering about where to host the next iteration of my side project (GiftyWeddings.com). It currently uses Go + SQLite, and I deploy it to a small $9/month instance on AWS, but the setup involves a bunch more Ansible and messing around than I want, and I'm not even sure the Ansible scripts I wrote a few years ago would work anymore (or work a second time). A simpler PaaS-like system that works with SQLite at a few bucks a month would be great. I'll check out your repo and fly.io in more depth -- thanks!

Re: SQLite the only database you will ever need in most cases (2021)

#187

Post by irskep »

ripdog wrote: post #105
Arathorn wrote: post #92

Earlier quoted context omitted.

The problem is that even a single user matrix server can be very resource intensive if that user joins big rooms with thousands of users spread over thousands of servers. Synapse is very database heavy, so the parallelism in Postgres helps a lot - plus some of the hot DB paths have special cased queries for Postgres to use some of its more obscure features that Sqlite lacks. Finally, we don’t dogfood or optimise Syna…

Thank you, Matthew! This would be fantastic information to include in the Synapse documentation, for those of us being seduced by the operational simplicity of SQLite, and posts like this. :)

Adding my voice to this - I was actually looking at Synapse a few days ago and was wondering why SQLite was not recommended. This would have answered a question I didn't know the answer to until just now.

Re: SQLite the only database you will ever need in most cases (2021)

#189

Post by berdon »

culopatin wrote: post #182
berdon wrote: post #151

Earlier quoted context omitted.

Fwiw, I’ve used all of those in my career, extensively here and there. I’ve written many thousands of lines of bash scripts. Even written my own php-for-bash-script style code tags that support arbitrary shells. I’ve written my own log based distributed kv store, gone down the YouTube trail of writing my own db, gone through the angular and react iterations, deep dove in docker, docker compose, k8s, crds and custom h…

Good for you guys, but there is always someone doing more, always someone better, why start a pissing contest? You turned someone’s comment on why deploying SQLite is not trivial into an opportunity to tap yourself in the back so we can all say “wow look at berdon”, but in the end, it doesn’t really matter.

You misunderstood my post - I was trying to make your point - there is always someone better. Just because you haven’t doesn’t mean others can’t.

Re: SQLite the only database you will ever need in most cases (2021)

#190

Post by jpgvm »

kragen wrote: post #165
erik_seaberg wrote: post #125

Earlier quoted context omitted.

Sounds like IMS runs on Z system mainframes with redundant hot-swappable CPUs and memory. They pay IBM a lot of money for the illusion of a single reliable machine, when a different OS would manage it as a small cluster. We economize by using racks of cheap, flaky commodity hardware, but we have to be ready for one computer to die by failing each application over to another.

in 01973 the s/360 did not have hot-swappable cpus or memory even in the 01990s i don't think ibm had such an offering, though tandem did (but it couldn't run ims)

it didn't run ims but it ran nonstop SQL instead which was a rdbms designed for their redundant hw architecture
Post reply on HN