Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Unlikely Database Migration

tailscale.com

Post reply on HN
Print view View original

111–120 of 190 posts

Re: An Unlikely Database Migration

#111

Post by breck »

CleverLikeAnOx wrote: post #98
breck wrote: post #41

If you liked this I highly recommend "Clean Architecture" by Uncle Bob. He has a great story of how they kept putting off switching to a SQL DB on a project and then never needed to and it was a big success anyway.

I thoroughly enjoyed "Clean Architecture." I engage with all of Uncle Bob's work in an adversarial manner: he is trying to convince me that he is right and I am playing devil's advocate. This helped me gain insight from his work, develop my own thoughts on architecture, and avoid getting too caught up in doing things the "one true way."

I'm always surprised when people get so agitated when I bring up his books (Clean Code being one of my all time favorites). Even if you disagree with some of the specifics, if there's one big takeaway like you said, it's to develop your "thoughts". He has thought hard about all of these things, and will force you to as well.

I imagine it's like if you were trying to be an olympic marathon runner, you'd study things like humidity and shoes and arm motion deeply, and he kind of does that (function naming, nodes per block, comments, test coverage, et cetera). Even if you don't agree with him, as you suggest read it and play "devil's advocate", and you will be forced to think about details that will make you a better software engineer.

Re: An Unlikely Database Migration

#112

Post by worik »

bradfitz wrote: post #92
worik wrote: post #86

This jumped out at me: "The obvious next step to take was to move to SQL" No. Not unless your data is relational. This is a common problem, relational databases have a lot of over head. They are worth it when dealing with relational data. Not so much with non relational data.

Maybe "obvious" was the wrong word there. What we meant to say was that would be the typical route people would go, doing the common, well-known, risk-averse thing.

Yes, I agree.

It is the wrong "obvious" thing. Sounds like they did better though

Re: An Unlikely Database Migration

#113

Post by bullen »

I also use JSON files... but one per value! It has it's ups and downs: pro it's incredibly fast and scales like a monster. con it's uses alot of space and inodes, so better use type small with ext4!

The only feature it misses is to compress the data that is not actively in use, that way there is really not much of a downside.

http://root.rupy.se

Re: An Unlikely Database Migration

#114

Post by JacobiX »

After reading the comments and the blog post, I think that the requirements boils down to fast persistence to disk, minimum dependencies and fast test-runs. Fortunately the data is very small 150MB and it fits very easily in memory. According to the post the data changes often so they need to write the data many times in a second. But I'm not sure why do they need to flush every time the entire 150MB ? Why not structure the files/indexes such that we write only the modified data ?

Re: An Unlikely Database Migration

#115

Post by dekhn »

jeffbee wrote: post #90
dekhn wrote: post #81

"""Even with fast NVMe drives and splitting the database into two halves (important data vs. ephemeral data that we could lose on a tmpfs), things got slower and slower. We knew the day would come. The file reached a peak size of 150MB and we were writing it as quickly as the disk I/O would let us. Ain’t that just peachy?""" Uh, you compressed it first, right? because CPUs can compress data faster than disk I/O.

Hrmm. Even lz4 level 1 compresses at "only" about 500-1000MB/s on various CPU types, which isn't quiet as fast as NVMe devices demand.

yes, that's a very recent change (afaict, the big change was going from SATA interface to NVME).

Re: An Unlikely Database Migration

#116

Post by Quekid5 »

jrockway wrote: post #110
Quekid5 wrote: post #104

Earlier quoted context omitted.

Not OP, but if I could ask further... How much consistency can you tolerate on your reads? From the use case you mention I imagine... quite a lot, but you could risk locking yourself out of networks/systems if you get it wrong? EDIT: Negation is important

I've always found it hard to reason about relaxing consistency, and I think people underestimate how much complexity they take on by moving away from serializable isolation towards something looser. (Fun fact! Many databases out of the box don't correctly handle the classic transaction example -- read an account balance, subtract an amount from the balance, and then add the amount to another account's balance.) Usual…

Interesting perspective, thanks!

Not sure whether I agree or disagree, actually...

AFAICT Linearilazable is about the best we can expect in reality (at least for a distributed system), but as you point out: Very few people actually check their assumptions... and even fewer actually think about DB transactions correctly in the first place. It's actually really, really hard and people have these rules of thumb in their heads that aren't actually correct.

Which gets me to wondering if we could formalize some of this stuff... (in relevant "code scopes", dgmw!)

EDIT: If there is one thing I am certain about it is the fact that a lot of consistency can be relaxed around human interaction. It's lossy anyway, and people will call you (eventually, depending on anxiety/shyness) if you haven't fulfilled an order. The browser is the first order of that and that's already out of date once you show a page, so... Anyway, that's just to say it's amusing how much people worry about consistency on the front end

Re: An Unlikely Database Migration

#117

Post by cbushko »

I think a lot of people are missing the point that a traditional DB (MYSQL/Postgress) are not a good fit for this scenario. This isn't a CRUD application but is instead a distributed control plane with a lot of reads and a small dataset. Joins and complex queries are not needed in this case as the data is simple.

I am also going to go out on a limb and guess that this is all running in kubernetes. Running etcd there is dead simple compared to even running something like Postgress.

Congrats on a well engineered solution that you can easily test on a dev machine. Running a DB in a docker container isn't difficult but it is just one more dev environment nuance that needs to be maintained.

Re: An Unlikely Database Migration

#118

Post by cbushko »

npiit wrote: post #2

Unfortunately this company is known for its shady agressive marketing on hacker news. The upvotes count is really suspicious and this is not the first time. EDIT: Why is the downvoting? the post was given like 9 upvotes in the first 5 minutes. I frequently go to "new" and this is a highly suspicious behaviur.

I upvoted because the content was technical, well written and different than what we normally see on HN.

Re: An Unlikely Database Migration

#119

Post by ac2u »

bradfitz wrote: post #84
nmjohn wrote: post #51

Earlier quoted context omitted.

Can you put some numbers on how much time is too much? I've never seen anyone go this far to avoid using a database for what sounds like the only "real" reason is to avoid testing latency (a problem which has many other solutions) so I am really confused, but curious to understand!

Running all of our control server tests (including integration tests) right now takes 8 seconds, and we're not even incredibly happy with that. There's no reason it should even be half that. So it's not really in our patience budget for adding a mysqld or postgres start up (possible docker pull, create its schema, etc).

>right now takes 8 seconds, and we're not even incredibly happy with that

With the amount of explaining and skepticism you're having to deal with in most of the threads here (plenty of reasonable questions, some seem to approach the question with the assumption that your approach is totally wrong) I feel compelled to comment on how nice such a fast feedback loop would be just so it's known that you're not listing these benefits into an ether that doesn't appreciate them.

Re: An Unlikely Database Migration

#120

Post by 0xbadcafebee »

I am missing a lot of context from this post because this just sounds nonsensical.

First they're conflating storage with transport. SQL databases are a storage and query system. They're intended to be slow, but efficient, like a bodybuilder. You don't ask a bodybuilder to run the 500m dash.

Second, they had a 150MB dataset, and they moved to... a distributed decentralized key-value store? They went from the simplest thing imaginable to the most complicated thing imaginable. I guess SQL is just complex in a direct way, and etcd is complex in an indirect way. But the end results of both are drastically different. And doesn't etcd have a whole lot of functional limitations SQL databases don't? Not to mention its dependence on gRPC makes it a PITA to work with REST APIs. Consul has a much better general-purpose design, imo.

And more of it doesn't make sense. Is this a backend component? Client side, server side? Why was it using JSON if resources mattered (you coulda saved like 20% of that 150MB with something less bloated). Why a single process? Why global locks? Like, I really don't understand the implementation at all. It seems like they threw away a common-sense solution to make a weird toy.

Post reply on HN