Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A database for 2022

tailscale.com

Post reply on HN
Print view View original

51–60 of 336 posts

Re: A database for 2022

#51

Post by lostcolony »

rob_c wrote: post #49
tptacek wrote: post #46

Earlier quoted context omitted.

Do you want to know how popular systems are actually built, from the inside, or do you want carefully groomed triumphal announcements of new, perfectly-formed features? I struggle with this a lot in my writing too, but I've drawn a conclusion I'm sticking with: it's better to relay what the team is actually doing, "warts" and all. We ranked on HN a few weeks ago with a post about user-mode WireGuard that was basicall…

Yes but userland wireguard is unique and useful. This is ignoring textbooks, competitors who write about their exploits warts and all and screams of get a more experienced db team to work with you... shoving it into a json screams of throw it at a disk and let the hardware deal with it

>> shoving it into a json screams of throw it at a disk and let the hardware deal with it

Which...worked. For quite a long time. And required no dev effort. Freeing devs up to focus on other things.

They did it as a POC, and it worked, better even than they had anticipated; why over-engineer it up front? Even when it turns out they are gonna need it (i.e., YAGNI no longer applies), they have been able to push off their "solve problems customers don't care about" to instead favor "problems customers do care about". That...seems like sound engineering to me?

Re: A database for 2022

#52

Post by dgb23 »

cogman10 wrote: post #47
bradfitz wrote: post #28

Earlier quoted context omitted.

Tailscale employee here. Our database needs are tiny, as explained in the earlier post. So we optimize for things like: "can we run all our tests quickly and easily in many environments without containers and VMs?" All three of our storage schemes have had that property. We have MySQL and PostgreSQL veterans on the team. We know those options well.

> without containers Why? The official docker postgres package weighs in at 100MB. Assuming you standardized on that, then every environment would end up having a fresh postgres image just waiting to be startup. Meaning, the actual cost is the memory for the server and startup time, not the image itself. Mount the data on tmpfs and you can startup and setup a db in very little time (I know, because that's what we do)…

Have a look at this headline:

> Zero config VPN. Installs on any device in minutes, manages firewall rules for you, and works from anywhere.

I don't know how their product works, but my intuition is that your suggestions aren't feasible.

Re: A database for 2022

#53

Post by nosefrog »

There's a lot of negativity here, so I just wanted to comment that I think the approach Tailscale is taking with sqlite is pretty neat. MySQL and Postgres are both pretty complicated, so gluing together two simpler systems, sqlite and litestream, sounds appealing, and I'm interested to hear how it turns out in the long run.

Re: A database for 2022

#54

Post by cogman10 »

tptacek wrote: post #46
musjleman wrote: post #16

So to recap their timeline: * Instead of an actual database use a JSON file. * Write a blog post about how that didn't scale. * Instead of an actual database hand-roll something else. * Write a blog post about how that didn't scale. * Instead of a database with built-in replication which is tailor built for key-value storage use SQLite with a single table containing key-value pairs and some fresh glue to make it repl…

Do you want to know how popular systems are actually built, from the inside, or do you want carefully groomed triumphal announcements of new, perfectly-formed features? I struggle with this a lot in my writing too, but I've drawn a conclusion I'm sticking with: it's better to relay what the team is actually doing, "warts" and all. We ranked on HN a few weeks ago with a post about user-mode WireGuard that was basicall…

> Do you want to know how popular systems are actually built, from the inside, or do you want carefully groomed triumphal announcements of new, perfectly-formed features?

It depends on what that tool is doing and if the thing it's doing is a core competency for your company.

It absolutely makes sense to geek out about stuff like this if it's part of your core product. However, if it isn't, then do yourself and your engineering team a favor and pick boring technologies.

Do you know what DB system tailscale would be using if they choose something boring postgres or Cassandra? Postgres or cassandra.

The cost of not doing that is they are on their 3rd iteration of switching db technologies and the development effort to keep rediscovering the same problems.

Re: A database for 2022

#56

Post by dan-robertson »

musjleman wrote: post #16

So to recap their timeline: * Instead of an actual database use a JSON file. * Write a blog post about how that didn't scale. * Instead of an actual database hand-roll something else. * Write a blog post about how that didn't scale. * Instead of a database with built-in replication which is tailor built for key-value storage use SQLite with a single table containing key-value pairs and some fresh glue to make it repl…

Doesn’t the blog post explain why they chose this step rather than doing the thing you suggest? In particular I don’t see how this comment really addresses any of their arguments.

And if they got on ok with the json file, maybe you should update towards the features of a complicated RDBMS not being worth the cost?

Re: A database for 2022

#57

Post by tptacek »

cogman10 wrote: post #54
tptacek wrote: post #46

Earlier quoted context omitted.

Do you want to know how popular systems are actually built, from the inside, or do you want carefully groomed triumphal announcements of new, perfectly-formed features? I struggle with this a lot in my writing too, but I've drawn a conclusion I'm sticking with: it's better to relay what the team is actually doing, "warts" and all. We ranked on HN a few weeks ago with a post about user-mode WireGuard that was basicall…

> Do you want to know how popular systems are actually built, from the inside, or do you want carefully groomed triumphal announcements of new, perfectly-formed features? It depends on what that tool is doing and if the thing it's doing is a core competency for your company. It absolutely makes sense to geek out about stuff like this if it's part of your core product. However, if it isn't, then do yourself and your e…

They picked (checks notes) sqlite. It's quite possibly the most boring choice available!

Re: A database for 2022

#58

Post by eminence32 »

flatiron wrote: post #42

How did they think going from a single json to either MySQL or Postgres? I know they say this isn’t April fools but I’m not sure I would put it past the engineering team. This can’t be real. “We looked at real solutions like cockroach but nobody could be bothered to rtfm”

I'm sympathetic to the idea of avoiding a database because no one on your team has experience with it.

On my old team, our product supported Oracle DB (along with some other databases), but on one on the team really was an oracle expert, and we would frequently run into questions that we didn't really know how to answer. We would have dropped it if we could.

It's easy to say "rtfm" but doing this doesn't make you an expert in the system. And as I bet many NH readers know, there's a big difference between "I read the quickstart guide and got a DB working in an afternoon" and "We've hit an error an error that the manual says nothing about, but our DB expert saw this once, 7 years ago, and remembers how to recover"

Re: A database for 2022

#59

Post by [deleted] »

musjleman wrote: post #16

So to recap their timeline: * Instead of an actual database use a JSON file. * Write a blog post about how that didn't scale. * Instead of an actual database hand-roll something else. * Write a blog post about how that didn't scale. * Instead of a database with built-in replication which is tailor built for key-value storage use SQLite with a single table containing key-value pairs and some fresh glue to make it repl…

[deleted]

Re: A database for 2022

#60

Post by [deleted] »

rudasn wrote: post #20

This is an April fool's joke, right? Edit: no it's not.. > Footnote: coworkers point out it’s April Fool’s today and request that I clarify this isn’t a joke. Joke’s on them: every day’s April Fool’s in the Tailscale Database Engineering department.

[deleted]
Post reply on HN