Live data from Hacker News

Why I Built Litestream

litestream.io

21–30 of 178 posts

Re: Why I Built Litestream

#21

Awesome! I built a side-business that runs completely on Crystal + SQLite. Very light, fast service and makes ~$200k/mo. I just cp my sqlite file to S3 every 2 hours. From my app, i have a page[1] where i can load any snapshot database saved on S3. I can backup at anytime too with a click, which i do before deployment. [1]: https://i.imgur.com/Ls1Tnxc.png

Is this a public business? Would you mind sharing, if so?

We do local cookie delivery[1]. We run our own mapping software and have our own drivers so most of the software is internal facing.

Talked about it on Indie Hackers podcast[2], where i briefly mention SQLite as well.

[1]: https://cravecookie.com

[2]: https://www.indiehackers.com/podcast/166-sam-eaton-of-crave-...

Re: Why I Built Litestream

#22

> Anecdotally, I’ve run several VPS servers over the years which all have well over 99.9% uptime and have suffered no catastrophic failures. Without advocating for Kubernetes, there's a pretty big difference between running a single application like WordPress or whathaveyou that receives only occasional updates and a SaaS application that is actively developed by hundreds or thousands of engineers deploying dozens of…

Yes, I agree there's definitely a place for Kubernetes. If you have thousands of engineers working on an application then workflow management becomes a very real concern. I've seen a lot of organizations adopt Kubernetes well before then though and I feel like it's a mistake. I didn't mean to imply that all Kubernetes applications should move to SQLite but that SQLite is a viable option for most small to medium sized applications with moderate uptime requirements.

Re: Why I Built Litestream

#23

Awesome! I built a side-business that runs completely on Crystal + SQLite. Very light, fast service and makes ~$200k/mo. I just cp my sqlite file to S3 every 2 hours. From my app, i have a page[1] where i can load any snapshot database saved on S3. I can backup at anytime too with a click, which i do before deployment. [1]: https://i.imgur.com/Ls1Tnxc.png

Is this a public business? Would you mind sharing, if so?

Looks like it is Crave Cookies: https://www.indiehackers.com/product/crave-cookie

Re: Why I Built Litestream

#24
> “But nobody writes production applications with SQLite, right?"

We've been doing it for 5 years now. Basic tricks we employ are:

Use PRAGMA user_version for purposes of managing automatic migrations, a. la. Entity Framework. This means you can actually do one better than Microsoft's approach, because you don't need a special unicorn table to store migration info. A simple integer compared with your latest integer and executing SQL in the range is all it takes.

Use PRAGMA synchronous=NORMAL alongside PRAGMA journal_mode=WAL for maximum throughput while supporting most reasonable IT recovery concerns. If you are running your SQLite application on a VM somewhere and have RTO which is satisfied by periodic hot snapshots (which WAL is quite friendly to), this is a more than ideal way to manage recovery of all business data while also giving good throughput to writers. If you are more paranoid than we are, then you can do FULL synchronous for a moderate performance penalty. This would be more for situations where your RTO requires the exact state of the system be recoverable the moment it lost power. We can afford to lose the last few minutes of work without anyone getting yelled at. Some modern virtualization technologies do help a lot in this regard. Running bare metal you need to be a little more careful.

For development & troubleshooting, being able to copy a .db file (even while its in use) is tremendously powerful. I can easily patch up a QA database I mangled with a bad migrator in 5 minutes by stopping the service, pulling the .db local, editing, and pushing it back up. We can also ask our customers to zip up their entire db folder so we can troubleshoot the entire system state.

Being able to use SQLite as our exclusive data store also meant that our software delivery process could be trivialized. We use zero external hosts, even localhost, for our application to be installed or started. We don't even require a runtime exist on the base operating system. Unzip our latest release build to a blank Win2019 server, sc.exe the binary path, net start the service, and it just works. Anyone can deploy our software because it's literally that simple. We didn't even bother to write a script because its a bigger pain in the ass to set powershell execution mode.

So, its not just about the core data storage, but also about the higher-order implications of choosing to use a database solution that can be wholly embedded within your application. Because of decisions like these, we don't have to screw around with things like Docker or Kubernetes.

Re: Why I Built Litestream

#25

> Anecdotally, I’ve run several VPS servers over the years which all have well over 99.9% uptime and have suffered no catastrophic failures. Without advocating for Kubernetes, there's a pretty big difference between running a single application like WordPress or whathaveyou that receives only occasional updates and a SaaS application that is actively developed by hundreds or thousands of engineers deploying dozens of…

Yes, I agree there's definitely a place for Kubernetes. If you have thousands of engineers working on an application then workflow management becomes a very real concern. I've seen a lot of organizations adopt Kubernetes well before then though and I feel like it's a mistake. I didn't mean to imply that all Kubernetes applications should move to SQLite but that SQLite is a viable option for most small to medium sized…

Right, I didn't interpret you as trying to gloss over this nuance, but I've seen a lot of "always k8s vs never k8s" debates which suggests to me that people need the explicit disclaimer.

Re: Why I Built Litestream

#26

Earlier quoted context omitted.

Is this a public business? Would you mind sharing, if so?

We do local cookie delivery[1]. We run our own mapping software and have our own drivers so most of the software is internal facing. Talked about it on Indie Hackers podcast[2], where i briefly mention SQLite as well. [1]: https://cravecookie.com [2]: https://www.indiehackers.com/podcast/166-sam-eaton-of-crave-...

You make $200k/m delivering cookies in two cities? That’s incredible!

Always amazed by these niche businesses that make bank

Re: Why I Built Litestream

#27
post #6

Ben - What are some practical examples of use cases you see here over the long term?

Hi Ethan, I see two primary use cases currently:

1. Applications which are read-heavy & serve fewer than 10,000 requests per second.

2. SaaS applications which can be sharded so that the largest customer uses 10,000 requests per second or less.

Once read-only replicas functionality is added, I'm also excited about globally-distributed applications where local servers can serve users with low-latency read requests. For example, you could run a high-traffic e-commerce site with 20 PoPs spread across the world for only $100/month. Customers in India could get the same fast response times as someone in the US. I think that's pretty compelling.

Re: Why I Built Litestream

#28

Awesome! I built a side-business that runs completely on Crystal + SQLite. Very light, fast service and makes ~$200k/mo. I just cp my sqlite file to S3 every 2 hours. From my app, i have a page[1] where i can load any snapshot database saved on S3. I can backup at anytime too with a click, which i do before deployment. [1]: https://i.imgur.com/Ls1Tnxc.png

Also got this gem from the Crave Cookies website:

"The software is built "from scratch" like the cookies and is part of Crave's success story. No other food company has the software Crave has for managing deliveries."

You should definitely do a write-up on Not Invented Here syndrom. And we sometimes "reinventing" the wheel, in moderation and for core components, really is the best solution.

Re: Why I Built Litestream

#29
post #26

Earlier quoted context omitted.

We do local cookie delivery[1]. We run our own mapping software and have our own drivers so most of the software is internal facing. Talked about it on Indie Hackers podcast[2], where i briefly mention SQLite as well. [1]: https://cravecookie.com [2]: https://www.indiehackers.com/podcast/166-sam-eaton-of-crave-...

You make $200k/m delivering cookies in two cities? That’s incredible! Always amazed by these niche businesses that make bank

we only have a single kitchen and it seems just above 200k is our ceiling for a single location. Expanding soon.

we took down our revenue from indiehackers but somebody took a screenshot and posted on twitter of the stripe verified revenue. https://pbs.twimg.com/media/EXbNBVUX0AAotOo?format=jpg&name=...

Re: Why I Built Litestream

#30

Awesome! I built a side-business that runs completely on Crystal + SQLite. Very light, fast service and makes ~$200k/mo. I just cp my sqlite file to S3 every 2 hours. From my app, i have a page[1] where i can load any snapshot database saved on S3. I can backup at anytime too with a click, which i do before deployment. [1]: https://i.imgur.com/Ls1Tnxc.png

$200k/mo is remarkable for a "side" business! Can you share more of your story?
Post reply on HN