Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A database for 2022

tailscale.com

Post reply on HN
Print view View original

141–150 of 336 posts

Re: A database for 2022

#141

Post by tptacek »

ignoramous wrote: post #105
rablackburn wrote: post #100

I loved this article, despite the negativity. A discussion that keeps recurring on HN is how you can get a lot done with simple, boring solutions. This is a perfect example of that in practice. What’s the simplest thing that’ll work? Do that, monitor the solution, and when you start hitting a limitation reevaluate what the next appropriate solution is. There are plenty of comments here deriding this as insanity when…

> People aren’t perfect, not every engineer comes fully-formed grokking the industry’s best solutions and their trade offs. The thing is, David Crawshaw is being extremely honest and humble here in admitting their shortcomings and how they're learning from it... but an entire troupe of Silicon Valley engs seem to have lost sleep over it, for reasons beyond me.

Are these even shortcomings? They made JSON scale for 18 months, which is an achievement. They seamlessly switched to etcd, which is one of the "right things" for distributed state problems. Like us, they discovered that when you adopt a "distributed state" solution, you inherit all the API limitations those things come with because nothing really solves (or can solve) the "multiple writers multiple readers real-time consistency" problem. Nothing broke, but etcd was a pain to work with: anything you store in etcd, the developers that work with it have to understand the rituals of etcd. Wouldn't it be nice to have that just be a SQL database? Oh, look: that's easy to do: use sqlite, and ship WAL segments.

There is negativity on this thread, but it strikes me as incredibly, even embarrassingly, ill-informed.

Re: A database for 2022

#142

Post by mrkurt »

epolanski wrote: post #90
bradfitz wrote: post #28

Earlier quoted context omitted.

Tailscale employee here. Our database needs are tiny, as explained in the earlier post. So we optimize for things like: "can we run all our tests quickly and easily in many environments without containers and VMs?" All three of our storage schemes have had that property. We have MySQL and PostgreSQL veterans on the team. We know those options well.

> we run all our tests quickly and easily in many environments without containers and VM How's that an important metric to optimize for? Did you ever benchmark the tests with other solutions?

I believe "quickly" means "quick for a human to do" in this case.

Re: A database for 2022

#143

Post by tptacek »

ignoramous wrote: post #114
tptacek wrote: post #87

This thread has (at least at the moment) serious Bob Martin Sudoku Solver energy to it. Tailscale has solved an infamously complicated problem using, for the most part, simple tools. They're not just successful; they're remarkably successful, spookily successful, upsettingly successful. Consider whether the secret sauce here might not be au courant database choices, but rather something much harder for random teams t…

> This thread has (at least at the moment) serious Bob Martin Sudoku Solver energy to it. To the forum's credit, the top comment (by u/judofyr) the last time tailscale blogged about their unusal db setup, got it exactly right: https://news.ycombinator.com/item?id=25768042 > Interesting choice of technology, but [tailscale] didn't completely convince me to why this is better than just using SQLite or PostgreSQL with a…

I did commit the cardinal sin of HN metacommentary and referred to the state of the thread at a point in time as if it was a durable property of the thread, which was a mistake I own and take full responsibility for.

Re: A database for 2022

#144

Post by tptacek »

jeffffff wrote: post #119
mrkurt wrote: post #79

Earlier quoted context omitted.

Managed DBs have tremendous lock in. Just try migrating off RDS with zero downtime. You can't, because they've "managed" your ability to configure external replicas. Then built a whole brittle data migration service that probably won't work for your DB.

they've already had to migrate twice due to questionable technology decisions. having to migrate again due to needing to change cloud providers seems a lot less likely than having to migrate again because their outside the box technology choice didn't pan out.

"Questionable technology decisions". You're trying to dunk, but I don't think you understand where the hoop is. Their technology decisions have panned out Tailscale well. We should all be so fortunate. This isn't Twitter with the "fail whale"; the only reason you know about any of this stuff is because they wrote about it. They ran their entire service with a JSON file backend for 18 months, and switched from it to etcd without you even knowing about it.

Re: A database for 2022

#146

Post by reilly3000 »

tptacek wrote: post #143
ignoramous wrote: post #114

Earlier quoted context omitted.

> This thread has (at least at the moment) serious Bob Martin Sudoku Solver energy to it. To the forum's credit, the top comment (by u/judofyr) the last time tailscale blogged about their unusal db setup, got it exactly right: https://news.ycombinator.com/item?id=25768042 > Interesting choice of technology, but [tailscale] didn't completely convince me to why this is better than just using SQLite or PostgreSQL with a…

I did commit the cardinal sin of HN metacommentary and referred to the state of the thread at a point in time as if it was a durable property of the thread, which was a mistake I own and take full responsibility for.

If anything, this ^^^ is how you win the long game. History will surface itself when you make assertions. Gracefully revocability scales.

Re: A database for 2022

#147

Post by tptacek »

rob_c wrote: post #76
lostcolony wrote: post #51

Earlier quoted context omitted.

>> shoving it into a json screams of throw it at a disk and let the hardware deal with it Which...worked. For quite a long time. And required no dev effort. Freeing devs up to focus on other things. They did it as a POC, and it worked, better even than they had anticipated; why over-engineer it up front? Even when it turns out they are gonna need it (i.e., YAGNI no longer applies), they have been able to push off the…

Speaking as someone who has had to fix things like this... When you reach capacity and it hits a screaming halt as the hardware capacity has been reached and performance doesn't collapse its completely gone. This is not about best use. Hell json file performance may be well within spec of the system requirements, but jumping up and down without perf numbers or specs is typical of sweep it under the carpet for a gener…

That would be a good critique if it was was happening, but they gracefully transitioned from a JSON file(!) backend to etcd, and gracefully transitioned from etcd to sqlite. I'd say they're going to gracefully transition from sqlite, but I think the think people don't have their heads around is that it's possible they'll never need to do that, because of The Unreasonable Effectiveness of SQLite.

Re: A database for 2022

#148

Post by chubot »

benbjohnson wrote: post #77
chubot wrote: post #38

Honest question: what do the sqlite authors think of litestream? I seem to recall Richard Hipp on the "changelog" podcast mentioning it, but I don't remember what he said. 2 episodes here: https://changelog.com/person/drh/podcasts#feed I think it was very neutral, something to the effect of "there are multiple solutions". But I know essentially nothing about sqlite internals so I can't judge, or maybe that's why I di…

> Honest question: what do the sqlite authors think of litestream? Litestream author here. Dr Hipp and his team reached out when I first released Litestream and we had a video call together. They were fantastic. Really friendly and down to earth. I explained how I put together Litestream and we went back and forth on different approaches. They were really helpful with understanding some of the shared memory stuff tha…

Cool, thanks for the response!

Re: A database for 2022

#149

Post by ignoramous »

Aeolun wrote: post #41
jjeaff wrote: post #39

Earlier quoted context omitted.

I really hope they aren't taking the same tack when dealing with the encryption portion of their service.

What would be the analogue? Using openssl with a wrapper service? Doesn’t seem particularly dubious to me.

> Using openssl with a wrapper service?

Coincidentally, one of tailscale's founders already built this: https://github.com/sshuttle/sshuttle

Re: A database for 2022

#150

Post by billywhizz »

AnimalMuppet wrote: post #115
dahfizz wrote: post #112

This should be a new logical fallacy: appeal to current year. Every time I hear someone say "it's $CURRENT_YEAR, we obviously need to implement $PARTISAN_POLICY" I wonder about what fringe politics will be "obviously correct" in 3022.

Seconded. It's a logical fallacy because they are using the year as proof that we should adopt their policy. Except the year only proves what year it is - nothing else.

oh boy. is nobody allowed to make a little joke anymore. you should take yourself a lot less seriously my dude.
Post reply on HN