Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A database for 2022

tailscale.com

Post reply on HN
Print view View original

81–90 of 336 posts

Re: A database for 2022

#81

Post by cogman10 »

tptacek wrote: post #57
cogman10 wrote: post #54

Earlier quoted context omitted.

> Do you want to know how popular systems are actually built, from the inside, or do you want carefully groomed triumphal announcements of new, perfectly-formed features? It depends on what that tool is doing and if the thing it's doing is a core competency for your company. It absolutely makes sense to geek out about stuff like this if it's part of your core product. However, if it isn't, then do yourself and your e…

They picked (checks notes) sqlite. It's quite possibly the most boring choice available!

And they tacked on Litestream, which made for far more interesting failure modes. Hopefully they ran their jepson... oh wait, no they didn't really care about the reasons they ruled out traditional db choices.

Re: A database for 2022

#82

Post by ignoramous »

AtlasBarfed wrote: post #10

... this is just replication of a database? As in the full database per node/replica? Hasn't AWS's SQL db-as-a-services had this for years now?

Tailscale mention they want to run tests locally, not deal with MySQL/Postgres ops overhead themselves, and also avoid vendor lock-in (rules out RDS / Aurora, PlanetScale, CockroachDB, YugaByte, Spanner et al).

May be Oracle remains an option. ;)

Re: A database for 2022

#83

Post by cogman10 »

tptacek wrote: post #74
cogman10 wrote: post #65

Earlier quoted context omitted.

I suggest reading the article. The reason they haven't chosen "boring" tech has nothing to do with the clients running their software and everything to do with their internal development experiences. > MySQL (or PostgreSQL) would come next. I’m not particularly familiar with anything MySQL post 1998, but I’m sure it would work. The HA story for open source databases is somewhat surprising, though: you can either have…

If you're dinging them for not using Cockroach, they explained why they're not using Cockroach in the exact paragraph you quoted: if they use Cockroach, they're committing to the scaling and distribution system that Cockroach provides, which might or might not be a good fit for them 2 years from now. If it isn't a good fit, they're stuck with a huge engineering bill to get themselves out of Cockroach. We're moving to…

> If it isn't a good fit, they're stuck with a huge engineering bill to get themselves out of Cockroach.

Similar to the current engineering bills they are paying because they switched from a json file -> etcd -> sqlite?

Seems like the cost of switching isn't really an issue for them since they've pulled that lever multiple times now.

Re: A database for 2022

#84

Post by epolanski »

brycelarkin wrote: post #43

It looks like you guys use AWS. Why not just use RDS if you don’t want to deal with database management?

Some engineering teams seem to take overcomplication always one step too far. It's very hard to estimate future work and overconfident engineers consistently downplay the costs.

Re: A database for 2022

#85

Post by ripley12 »

musjleman wrote: post #45
Dylan16807 wrote: post #25

Earlier quoted context omitted.

Okay, but this is a database now. Is there a reason the replication needs to be built-in?

Sorry, I probably didn't phrase it very clearly. I don't think there is anything wrong with needing some external help for that, but the combination of their choices is IMO a bit bizarre: * SQLite which I mostly see as an embeddable database for applications, used on a server. * Single table with key-value pairs. * Having the tool for replication be a quite young project that's in active development.

> SQLite which I mostly see as an embeddable database for applications

This is common but incorrect view of SQLite. It's able to handle many server-side workloads, as long as you're aware of its limitations (and the Tailscale team certainly are).

https://www.sqlite.org/whentouse.html

Re: A database for 2022

#86

Post by musjleman »

tptacek wrote: post #46
musjleman wrote: post #16

So to recap their timeline: * Instead of an actual database use a JSON file. * Write a blog post about how that didn't scale. * Instead of an actual database hand-roll something else. * Write a blog post about how that didn't scale. * Instead of a database with built-in replication which is tailor built for key-value storage use SQLite with a single table containing key-value pairs and some fresh glue to make it repl…

Do you want to know how popular systems are actually built, from the inside, or do you want carefully groomed triumphal announcements of new, perfectly-formed features? I struggle with this a lot in my writing too, but I've drawn a conclusion I'm sticking with: it's better to relay what the team is actually doing, "warts" and all. We ranked on HN a few weeks ago with a post about user-mode WireGuard that was basicall…

> Do you want to know how popular systems are actually built, from the inside, or do you want carefully groomed triumphal announcements of new, perfectly-formed features?

IMO both this and "announcements of new, perfectly-formed features" are at the same level (if not lower, because a new feature is still something new). Both are essentially marketing with no technical content that's really worth your time. Except that it's now the third iteration of "we did an obviously questionable thing and replaced it with another questionable solution".

> it's better to relay what the team is actually doing, "warts" and all. We ranked on HN a few weeks ago with a post about user-mode WireGuard that was basically a litany of engineering mistakes, and it's one of the things I've most enjoyed writing.

I can agree with that, but I wouldn't bunch your writeup into the same group. That's some pretty decent technical content about the problems that were encountered. Meanwhile in these writeups what were the problems? "JSON file database too slow", "Hand-rolled software has to be maintained in-house"?

> Every serious product gets built the way Tailscale is describing. You start some components with the simplest thing that can reasonably work, and see how far you can take them. The JSON file Tailscale used to use was one of the most interesting and valuable things they wrote about.

Yet they keep changing it, migrating, writing tests for the new code and writing blog posts. The time all these things take even if the size of the new code is less than 100 lines is substantial.

Re: A database for 2022

#87

Post by tptacek »

This thread has (at least at the moment) serious Bob Martin Sudoku Solver energy to it. Tailscale has solved an infamously complicated problem using, for the most part, simple tools. They're not just successful; they're remarkably successful, spookily successful, upsettingly successful.

Consider whether the secret sauce here might not be au courant database choices, but rather something much harder for random teams to pick up and run with: just straight up good design and programming.

This is a thing that happens in other fields, too; some of the hardest dishes to cook seem extremely simple and have few ingredients. They're hard because there's no place to hide: your technique has to be flawless or they don't work. A famous entrance exam for new cooks at fine dining restaurants is "make an omelet". Tailscale is, thankfully (and, to me, irritatingly), more like an omelet than a truffled hash.

Re: A database for 2022

#88

Post by eyelidlessness »

brycelarkin wrote: post #43

It looks like you guys use AWS. Why not just use RDS if you don’t want to deal with database management?

Disclaimer: I hate doing ops, but I’ve been in a position to actively hate doing it fairly regularly for several years of my career. My perspective is as a person who doesn’t want to deal with any of this kind of stuff. So I’ve probably failed to acquire knowledge which would make it less frictionful for me, purely from lack of interest.

RDS has some significant downsides which I would personally consider no go if I were in a position to evaluate it. The one which stands out as particularly painful from my past experience is… it’s excruciatingly slow to provision or make configuration changes. Like lose whole days of work to a few iterations of trial and error slow. Combined with AWS’ sprawling and inscrutable set of authorization and configuration options, the weird idiosyncrasies between most of their offerings, and the absolutely opaque naming applied to most of those offerings… trying to use RDS effectively as a managed database service felt more to me like becoming a full time ops professional.

Re: A database for 2022

#89

Post by tptacek »

ripley12 wrote: post #85
musjleman wrote: post #45

Earlier quoted context omitted.

Sorry, I probably didn't phrase it very clearly. I don't think there is anything wrong with needing some external help for that, but the combination of their choices is IMO a bit bizarre: * SQLite which I mostly see as an embeddable database for applications, used on a server. * Single table with key-value pairs. * Having the tool for replication be a quite young project that's in active development.

> SQLite which I mostly see as an embeddable database for applications This is common but incorrect view of SQLite. It's able to handle many server-side workloads, as long as you're aware of its limitations (and the Tailscale team certainly are). https://www.sqlite.org/whentouse.html

Those limitations are also the limitations of the backing stores of many distributed state solutions; you just don't see them because they're hidden behind complicated APIs.

Re: A database for 2022

#90

Post by epolanski »

bradfitz wrote: post #28
musjleman wrote: post #16

So to recap their timeline: * Instead of an actual database use a JSON file. * Write a blog post about how that didn't scale. * Instead of an actual database hand-roll something else. * Write a blog post about how that didn't scale. * Instead of a database with built-in replication which is tailor built for key-value storage use SQLite with a single table containing key-value pairs and some fresh glue to make it repl…

Tailscale employee here. Our database needs are tiny, as explained in the earlier post. So we optimize for things like: "can we run all our tests quickly and easily in many environments without containers and VMs?" All three of our storage schemes have had that property. We have MySQL and PostgreSQL veterans on the team. We know those options well.

> we run all our tests quickly and easily in many environments without containers and VM

How's that an important metric to optimize for? Did you ever benchmark the tests with other solutions?

Post reply on HN