Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I Built Litestream

litestream.io

Post reply on HN
Print view View original

171–178 of 178 posts

Re: Why I Built Litestream

#171

Post by tonetheman »

Griffinsauce wrote: post #146
chubot wrote: post #140

Earlier quoted context omitted.

Well then you just need multiple Kubernetes clusters for redundancy :-) :-) From another thread on the home page right now: The current trend goes to multi-cluster environments, because it's way too easy to destroy a single k8s cluster due to bugs, updates or human mistake. Just like it's not an very unlikely event to kill a single host in the network e.g. due to updates/maintenance. For instance, we had several outa…

This reads like a bad joke. Where did the KISS principle go?

k8 is about as anti "keep it simple" as it can get.

Re: Why I Built Litestream

#172

Post by chmaynard »

uniqueid wrote: post #128

This is off-topic, but when I was a Mac user, I always wishes Apple built a barebones graphical tool into the OS to view and edit SQlite databases. They pop up now and then, including via Apple applications, and it would be convenient to view their contents in the GUI without third party tools.

During development, database tables often start out as spreadsheets. The Pages app has an excellent GUI. It's not difficult to import and export SQLite table data in either CSV or TSV format. Using pipes and scripting, I bet the transfer between SQLite and Pages could be implemented with one-line shell commands.

Re: Why I Built Litestream

#173

Post by lenkite »

popcorncowboy wrote: post #165
chillfox wrote: post #135

Earlier quoted context omitted.

because sqlite is super simple to use and most things won't ever need to scale beyond one server.

> most things won't ever need to scale beyond one server. Yep. But don't tell anyone. This is a secret weapon/super power most juniors (and many mid/seniors) have been conditioned by the GOOG/FB/AMZN/MSFT approved project literature to believe is simply untrue. The number of experienced, highly skilled founder devs I know that reach for shiny tech stacks that solve issues they don't have but can "scale" is a 100:1. T…

Well, as soon the solution doesn't scale - which will likely be within a couple of years - one needs to rearchitect and recode.

I suppose single node+db works fine for hobby SAAS and expert programmers who can squeeze every bit of performance from the machine. But for the average in-efficient programmer or teams, standard PostGres as DB and HA services fronted by a load-balancer work better.

Re: Why I Built Litestream

#174

Post by danbruder »

colinhb wrote: post #163

I've wanted to move a small, Postgres-backed web app (written in Go) to SQLite, but one of the hurdles is that we deploy to Heroku (which famously has an ephemeral filesystem). Since the web app runs on a single, always-on dyno, seems like it may work to use Litestream to (1) continuously replicate and (2) restore when we restart the dyno. For folks who have dug into Litestream further than me, any thoughts on this u…

I’m also interested in trying this out

Re: Why I Built Litestream

#176

Post by majkinetor »

motiejus wrote: post #92
majkinetor wrote: post #41

Earlier quoted context omitted.

Thanks for details. I yet wait to see how somebody serve 1M users on a web service using sqlite. Sounds like you can do all that since you create desktop app, you almost never need anything more then sqlite for that.

https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-q...

Surely that doesn't qualify as an answer being a read only test. Also, it looks like there is not much interaction between users in their core domain.

Re: Why I Built Litestream

#177

Post by wmanley »

lenkite wrote: post #173
popcorncowboy wrote: post #165

Earlier quoted context omitted.

> most things won't ever need to scale beyond one server. Yep. But don't tell anyone. This is a secret weapon/super power most juniors (and many mid/seniors) have been conditioned by the GOOG/FB/AMZN/MSFT approved project literature to believe is simply untrue. The number of experienced, highly skilled founder devs I know that reach for shiny tech stacks that solve issues they don't have but can "scale" is a 100:1. T…

Well, as soon the solution doesn't scale - which will likely be within a couple of years - one needs to rearchitect and recode. I suppose single node+db works fine for hobby SAAS and expert programmers who can squeeze every bit of performance from the machine. But for the average in-efficient programmer or teams, standard PostGres as DB and HA services fronted by a load-balancer work better.

> Well, as soon the solution doesn't scale - which will likely be within a couple of years - one needs to rearchitect and recode.

Your project is far more likely to die within a couple of years from having too few users than from having too many. If you find you're having scaling issues then that's a good time to solve them - your project is popular enough to justify the investment.

You'll likely be recoding your project quite a lot in the early phase of the development anyway as you learn more about the problem and (most importantly) more about what your users want/need. Anything that can reduce the cost of this early iteration will pay dividends later because it makes it more likely that you'll need to scale - and by the time you do need to scale you'll be scaling something that you understand and other people want.

Re: Why I Built Litestream

#178

Post by jlongtine »

benbjohnson wrote: post #62
polyrand wrote: post #8

Hi Ben, thank you so much for building this! I'm currently working on implementing a set of data structures (and more things later) on top of SQLite ( https://github.com/litements/ ) for some of the same reasons you mention in the article. I probably don't even have 1/10 of your experience, but your work is really motivating me to keep working on it, thanks!

I like where you're going with Litements. There are a lot of servers that could just be libraries running on SQLite. Job queues are one of those I find myself reimplementing sometimes.

This is an interesting idea... It'd be really cool to have these as SQLite extensions. I could see that being powerful (in part because of language agnosticism). Granted, a bit harder to build/install. https://sqlite.org/loadext.html
Post reply on HN