>The biggest problem with using SQLite in production is disaster recovery.
No, SQLite has WAL and can recover from failures. The biggest problem with SQLite is concurrency. You can't write from multiple processes in parallel.
Why I Built Litestream
141–150 of 178 posts
Re: Why I Built Litestream
#142>The biggest problem with using SQLite in production is disaster recovery. No, SQLite has WAL and can recover from failures. The biggest problem with SQLite is concurrency. You can't write from multiple processes in parallel.
That only solves a narrow range of disasters. Your disk could die or the file system becomes corrupted or your ec2 instance is accidentally destroyed or etc.
Re: Why I Built Litestream
#143"Solutions such as Kubernetes tout the benefits of zero-downtime deployments but ignore that their inherent complexity causes availability issues." This is completely accurate. I've seen several teams do kubernetes, only to both spend 50% of their dev time on ops, AND cause outages due to kubernetes complexity. They do this all while boasting about zero downtime deployments. It's comical really.
I have always wondered if PostgreSQL for the data layer + HAProxy(multiple instances of web services) would be enough in most cases where Kubernetes is used. HAProxy provides blue-green deployment strategies for web applications for continuous deployments. Database is best kept outside of a distributed system anyway..
A good weekend project and some learning ahead.
Re: Why I Built Litestream
#144I would love to be able to read sqlite from s3 without loading the whole file.
Re: Why I Built Litestream
#145"Solutions such as Kubernetes tout the benefits of zero-downtime deployments but ignore that their inherent complexity causes availability issues." This is completely accurate. I've seen several teams do kubernetes, only to both spend 50% of their dev time on ops, AND cause outages due to kubernetes complexity. They do this all while boasting about zero downtime deployments. It's comical really.
I call these "aspirational features".
The tech aspires to provide that feature, and _technically_ can, but in practice you're always chasing the goose. (or spending so much on ops time/people that it becomes an invalid option, of course this becomes clear after you've fully invested in it)
Re: Why I Built Litestream
#146"Solutions such as Kubernetes tout the benefits of zero-downtime deployments but ignore that their inherent complexity causes availability issues." This is completely accurate. I've seen several teams do kubernetes, only to both spend 50% of their dev time on ops, AND cause outages due to kubernetes complexity. They do this all while boasting about zero downtime deployments. It's comical really.
Well then you just need multiple Kubernetes clusters for redundancy :-) :-) From another thread on the home page right now: The current trend goes to multi-cluster environments, because it's way too easy to destroy a single k8s cluster due to bugs, updates or human mistake. Just like it's not an very unlikely event to kill a single host in the network e.g. due to updates/maintenance. For instance, we had several outa…
This reads like a bad joke.
Where did the KISS principle go?
Re: Why I Built Litestream
#147One notable site which runs on SQLite is SQLite's own source code version tracking system called Fossil.
See https://sqlite.org/src/doc/trunk/README.md and https://www.fossil-scm.org
Re: Why I Built Litestream
#148> “But nobody writes production applications with SQLite, right?" We've been doing it for 5 years now. Basic tricks we employ are: Use PRAGMA user_version for purposes of managing automatic migrations, a. la. Entity Framework. This means you can actually do one better than Microsoft's approach, because you don't need a special unicorn table to store migration info. A simple integer compared with your latest integer a…
>We can afford to lose the last few minutes of work without anyone getting yelled at. Some modern virtualization technologies do help a lot in this regard. Running bare metal you need to be a little more careful. How does virtualization help with data loss? I would expect that a VM can't have guarantees better than the underlying physical hardware provides.
> How does virtualization help with data loss? I would expect that a VM can't have guarantees better than the underlying physical hardware provides.
E.g. storage virtualisation.
Re: Why I Built Litestream
#149Earlier quoted context omitted.
> If I make $50k, I mean I am taking home $50k If I've revenue of $50k, I am taking home less than $50k That is not normal native usage. If you make $50k, you're salaried at $50k, but you take home considerably less than that.
Another native speaker here: I think the fact that we can debate this is evidence enough that it's confusing :) Yes with salary I usually say I "make" my pre-tax income. But I don't know if that's really what "make" means per se, or if that's just a side effect of how most jobs advertise salaries in pre-tax terms. (Also I assume this usage is actually older than the income tax.) In my mind, if someone tells me they "…
I think the fact we can debate this is evidence that it's not confusing!
Re: Why I Built Litestream
#150"Solutions such as Kubernetes tout the benefits of zero-downtime deployments but ignore that their inherent complexity causes availability issues." This is completely accurate. I've seen several teams do kubernetes, only to both spend 50% of their dev time on ops, AND cause outages due to kubernetes complexity. They do this all while boasting about zero downtime deployments. It's comical really.
I have always wondered if PostgreSQL for the data layer + HAProxy(multiple instances of web services) would be enough in most cases where Kubernetes is used. HAProxy provides blue-green deployment strategies for web applications for continuous deployments. Database is best kept outside of a distributed system anyway.. A good weekend project and some learning ahead.
It was a few years ago now, but my team at Amazon just deployed to a bunch of servers behind a load balancer and very rarely experienced downtime due to infrastructure. I’m working on a system that isn’t live right now and the infrastructure is so much more complex than that without a single customer. I spend a huge amount of dev time debugging issues that have their root cause in a flakey kubernetes cluster.