Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Unlikely Database Migration

tailscale.com

Post reply on HN
Print view View original

151–160 of 190 posts

Re: An Unlikely Database Migration

#151

Post by winrid »

a1369209993 wrote: post #135
Quekid5 wrote: post #103

Earlier quoted context omitted.

As long as you're guaranteeing correctness[0], it's hard to disagree with the "simple" approach. As long as you don't over-promise or under-deliver, there's no problem, AFAICS. [0] Via mutex in your case. Have you thought about durability, though. That one's actually weirdly difficult to guarantee...

> Have you thought about durability, though. That one's actually weirdly difficult to guarantee... Strictly speaking, it's literally impossible to guarantee[0], so it's more a question of what kinds and degrees of problems are in- versus out-of-scope for being able to recover from. 0: What happens if I smash your hard drive with a hammer? Oh, you have multiple hard drives? That's fine, I have multiple hammers.

What happened to the first hammer :D

Re: An Unlikely Database Migration

#152

Post by nine_k »

manigandham wrote: post #147
nine_k wrote: post #146

Earlier quoted context omitted.

IMHO, ORMs are a wrong abstraction. Full-fledged objects are a wrong abstraction of the data in more cases than not. The right tool is a wrapper / DSL over SQL, which allows to interact with the database in a predictable and efficient way, while not writing placeholder-ridden SQL by hand. Composable, typesafe, readable. ORMs do fine in small applications without performance requirements. The further you go from that,…

Do you have an example of the right tool? SQL is already a DSL and I can't see how creating another language isn't just adding more overhead without actually getting you to usable objects. The only reason objects are the "wrong" abstraction is because they don't match relational models exactly. That impedance mismatch is the entire reason for the object-to-relational mapping, otherwise you can use things like documen…

SQLAlchemy gives a nice composable DSL in Python; it has an ORM layer on top of it, too, but it's optional.

JOOQ allows for composable typesafe SQL in Java.

Re: An Unlikely Database Migration

#153

Post by mslm »

cavisne wrote: post #132

Honestly this feels like engineers that spent too long in FANG and get completely burnt out on dealing with SRE and HA requirements... so decide to built a setup so prone to 2AM pages even a PHP webshop would frown at it. Curiously though its a pattern I've seen twice in the last 12 months, there was that guide on the good bits of AWS that also recommended starting with a single host with everything running on it. Ma…

I'm sure your typical PHP webshop is getting plenty more 2AM pages than what these guys got with a solution they understood inside out because it was simple and got the job done.

Re: An Unlikely Database Migration

#154

Post by cakoose »

mwcampbell wrote: post #44
bradfitz wrote: post #35

Earlier quoted context omitted.

> and a tailetc cluster What do you mean by this part? tailetc is a library used by the client of etcd. Running an etcd cluster is much easier than running an HA PostgreSQL or MySQL config. (I previously made LiveJournal and ran its massively sharded HA MySQL setup)

> Running an etcd cluster is much easier than running an HA PostgreSQL or MySQL config. What if you used one of the managed RDBMS services offered by the big cloud providers? BTW, if you don't mind sharing, where are you hosting the control plane?

> > Running an etcd cluster is much easier than running an HA PostgreSQL or MySQL config.

> What if you used one of the managed RDBMS services offered by the big cloud providers?

Yeah, AWS RDS "multi-AZ" does a good job of taking care of HA for you. (Google Cloud SQL's HA setup is extremely similar.) But you still get 1-2 minutes of full unavailability when hardware fails.

I haven't operated etcd in production myself, but I assume it does better because it's designed specifically for HA. You can't even run less than three nodes. (The etcd docs talk about election timeouts on the order of 1s, which is encouraging.)

For many use cases, 1-2 minutes of downtime is tolerable. But I can imagine situations where availability is paramount and you're willing to give up scale/performance/features to gain another 9.

Re: An Unlikely Database Migration

#155

Post by enneff »

npiit wrote: post #9
hfern wrote: post #4

Earlier quoted context omitted.

I don't think it's shady marketing. The company is composed of several prominent software engineers (eg Brad Fitzpatrick of memcached fame). I think many folks on HN (myself included) are interested to see what they are working on. Especially as Brad left Google to work on it.

I understand. But given the product features compared to the rest of the industry, Tailscale brings no real value compared to Zero-tier hen it comes to meshes, it's not zerotrust like Cloudflare, Twingate, and many others, it claims to be open source while only the client is and it cannot be used without their closed source control plane where most of the feature are behind paywall, it's way more expensive than reput…

> Tailscale brings no real value compared to Zero-tier

This article has nothing to do with Tailscale the product and everything to do with the team's unconventional approach to engineering. That's what HN is interested in and why the post is being upvoted.

Re: An Unlikely Database Migration

#156

Post by _mjpassing_ »

hankchinaski wrote: post #27

seems like the reason for not going the MySQL/PSQL/DMBS route is lack of good Go libraries to handle relational databases (ORM/migration/testing)? from the story it seems more like a solution in search for a problem

How do you come to this conclusion from the article? Or is it just an attempt to discredit Go?

Re: An Unlikely Database Migration

#157

Post by gher-shyu3i »

manigandham wrote: post #137

> "Attempts to avoid this with ORMs usually replace an annoying amount of typing with an annoying amount of magic and loss of efficiency." People seem to keep using poorly-designed ORMs or are stuck with some strange anti-ORM ideology. Modern ORMs are fast, efficient, and very productive. If you're working with relational databases then you're using an ORM. It's a question of whether you use something prebuilt or wri…

> Modern ORMs are fast, efficient, and very productive

Which ones do you have experience with?

Re: An Unlikely Database Migration

#158

Post by bradfitz »

cbushko wrote: post #148
bradfitz wrote: post #141

Earlier quoted context omitted.

We have a lot of Kubernetes experience on the team. Multiple of us run Kubernetes clusters in our home labs (mine: https://github.com/bradfitz/homelab ), and one of us used to be on the Google GKE team as an SRE, and is the author of https://metallb.universe.tf/ (which multiple of us also use). Us _not_ using Kubernetes isn't because we don't know how to use it. It's because we _do_ know how to use it and when _not_…

Haha, it sounds like you have it covered. Even more so if you were to run on GKE (which I use and adore). When not to use it is a tough question. If I was ever in charge of a company, kubernetes would be the only way of running my product that I would consider. I am a fan of kubernetes as I use it every day but I have also been on the other side of the fence. I have run production systems on bare metal, VMs, EC2 inst…

Kubernetes support is a top Tailscale request and the community's starting to do it on their own (a bit, in less than ideal ways sometimes) so soon enough here we'll have to end our little Kubernetes vacation and get back into it and make it Tailscale support Kubernetes really well.

Re: An Unlikely Database Migration

#159

Post by bradfitz »

0xbadcafebee wrote: post #120

I am missing a lot of context from this post because this just sounds nonsensical. First they're conflating storage with transport. SQL databases are a storage and query system. They're intended to be slow, but efficient, like a bodybuilder. You don't ask a bodybuilder to run the 500m dash. Second, they had a 150MB dataset, and they moved to... a distributed decentralized key-value store? They went from the simplest…

I'd answer questions but I'm not sure where to start.

I think we're pretty well aware of the pros and cons of all the options and between the team members designing this we have pretty good experience with all of them. But it's entirely possible we didn't communicate the design constraints well enough. (More: https://news.ycombinator.com/item?id=25769320)

Our data's tiny. We don't want to do anything to access it. It's nice just having it in memory always.

Architecturally, see https://news.ycombinator.com/item?id=25768146

JSON vs compressed JSON isn't the point: see https://news.ycombinator.com/item?id=25768771 and my reply to it.

Re: An Unlikely Database Migration

#160

Post by AlfeG »

miki123211 wrote: post #99

I use the same system (a JSON file protected with a mutex) for an internal tool I wrote, and it works great. For us, file size or request count is not a concern, it's serving a couple (internal) users per minute at peak loads, the JSON is about 150 kb after half a year, and old data could easily be deleted/archived if need be. This tool needs to insert data in the middle of (pretty short) lists, using a pretty compli…

That's good. But single file could break on powerloss. I use sqllite. It's quite easy to use, not a single line though.
Post reply on HN