Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Distributed Databases Should Work More Like CDNs

cockroachlabs.com

Post reply on HN
Print view View original

71–80 of 86 posts

Re: Distributed Databases Should Work More Like CDNs

#71

Post by cookiecaper »

prepend wrote: post #7
timkpaine wrote: post #5

Imagine a globally replicated and version controlled object store. You could use this for data, assets, source code, anything you like. Would be incredibly useful for both the development side of things, as well as production.

We could call it the World Wide Web.

Yes. I've thought about a "distributed web" and mocked up some prototypes for a while. At some point, it loops back around and you come to something that is not hugely different from what we already have.

While some of the technical underpinnings could be improved, the state of the internet is determined by social and legal considerations, not technical ones.

Re: Distributed Databases Should Work More Like CDNs

#73

Post by manigandham »

marknadal wrote: post #33
astine wrote: post #23

Sooo, being able to distribute data globally is good for performance? Who knew? The thing about distributed systems, including distributed databases is that they need to navigate around the the CAP theorem(Consistency, Availability, Partition Tolerance, pick two, essentially,) and every solution is ultimately a trade off. This article would be a lot more interesting if it showed how CockroachDB made a better trade of…

Also a distributed database engineer, and while I disagree with CockroachDB's CAP Theorem tradeoff decisions, they are definitely well reasoned and principled. RethinkDB was Master-Slave (strongly consistent) with an amazing developer community and actually survived Aphyr's tests better than most systems. CockroachDB is also in the Master-Slave (strongly consistent) camp, but more enterprise focused, and therefore wi…

Banking is eventually consistent and the most incorrectly used example when it comes to any database scenarios.

CRDB is also master-master, all the nodes are the same and can serve reads and writes. That has no bearing on AP or CP.

Re: Distributed Databases Should Work More Like CDNs

#74

Post by b4lancesh33t »

Chickenosaurus wrote: post #66
zjaffee wrote: post #56

Earlier quoted context omitted.

It's my understanding that Google Spanner is leveraging hardware (direct lines between machines), so the degree at which there is a network partition is different than compared to other systems.

According to a paper by Eric Brewer, the author of the original CAP theorem, Google Spanner technically is not a CA system [1]. It is advertised as such because partitons are supposed to be exceedingly rare as the infrastructure is outstanding and a lot of operating experience is present [1]. I believe at the end of the day the CAP theorem is too fuzzy for such discussions. [1] https://research.google.com/pubs/pub458…

Theorems are all well and good, but ultimately the thing that matters is the experience of using the system "IRL" as the kids say. If spanner is up 99.99% of the time, and is consistent at all times, it is probably not going to be the weak link in your software chain.

(Fwiw their slo for multi-regional instances is 99.999%, although I have no idea what their measured performance is against that objective.)

Re: Distributed Databases Should Work More Like CDNs

#75

Post by Joeri »

elvinyung wrote: post #70
cube2222 wrote: post #49

Earlier quoted context omitted.

No, you can just have partitioned replicated topics like kafka has, just make sure that all the events for one user get put into the same partition, so that they get totally ordered.

So I guess the specific issue that grandparent raised was if multiple users wanted to edit the same record. So even if the queue is sharded on some primary key, then for each partition, either a) you only have one machine ingesting and thus a SPOF, or b) you ingest data using multiple machines, which means you need consensus/conflict resolution somewhere, right?

You ingest the log partition in all machines to reconstruct the state from the log. You do this even on the machine that writes to the log, treating the log as the source of truth. Since every user would belong to one partition there would be a globally agreed ordering of their events in the log.

Basically though that makes a CP tradeoff, since a network partition that hides the primary kafka replica from some of the writers causes writes to fail.

There’s simply no way to have a globally distributed system that’s always available and always consistent, either you get inconsistency or you drop writes.

Re: Distributed Databases Should Work More Like CDNs

#76

Post by dantiberian »

daxfohl wrote: post #62

Does GDPR really require all Europeans' data to stay on EU soil without explicit consent?

I don't think so, you need consent to process personal data anywhere in the world, and you can only transfer it outside of the EU if there are appropriate safeguards - https://gdpr-info.eu/art-46-gdpr/

Re: Distributed Databases Should Work More Like CDNs

#77

Post by ithkuil »

Chickenosaurus wrote: post #66
zjaffee wrote: post #56

Earlier quoted context omitted.

It's my understanding that Google Spanner is leveraging hardware (direct lines between machines), so the degree at which there is a network partition is different than compared to other systems.

According to a paper by Eric Brewer, the author of the original CAP theorem, Google Spanner technically is not a CA system [1]. It is advertised as such because partitons are supposed to be exceedingly rare as the infrastructure is outstanding and a lot of operating experience is present [1]. I believe at the end of the day the CAP theorem is too fuzzy for such discussions. [1] https://research.google.com/pubs/pub458…

The paper says:

"Does this mean that Spanner is a CA system as defined by CAP? The short answer is “no” technically, but “yes” in effect and its users can and do assume CA.

The purist answer is “no” because partitions can happen and in fact have happened at Google, and during (some) partitions, Spanner chooses C and forfeits A. It is technically a CP system."

which I believe is another way to word what I say in my post above.

Re: Distributed Databases Should Work More Like CDNs

#78

Post by elvinyung »

Joeri wrote: post #75
elvinyung wrote: post #70

Earlier quoted context omitted.

So I guess the specific issue that grandparent raised was if multiple users wanted to edit the same record. So even if the queue is sharded on some primary key, then for each partition, either a) you only have one machine ingesting and thus a SPOF, or b) you ingest data using multiple machines, which means you need consensus/conflict resolution somewhere, right?

You ingest the log partition in all machines to reconstruct the state from the log. You do this even on the machine that writes to the log, treating the log as the source of truth. Since every user would belong to one partition there would be a globally agreed ordering of their events in the log. Basically though that makes a CP tradeoff, since a network partition that hides the primary kafka replica from some of the…

Yep, this was the answer I was expecting. Basically, I really doubt that event sourcing magically solves this, which is what the above comment claimed [1].

[1] https://news.ycombinator.com/item?id=16328127

Re: Distributed Databases Should Work More Like CDNs

#79

Post by ec109685 »

icebraining wrote: post #20
dstroot wrote: post #19

CDN: no trade offs. Faster everywhere. More reliable overall Cockroach DB: trade some performance for geographic redundancy. The trade off may work in your favor - e.g. read heavy workloads (or not). I plugged in CDB I place of Postgres for some testing this week, was surprised it worked so well.

no trade offs This is almost never the case, and CDNs are no exception. A CDN like Cloudflare that reuses your domain(s) means that they become just a useless point that your dynamic requests have to travel to and from the main server. A CDN that uses its own domains requires extra DNS queries, extra TCP & SSL connection setup, etc, plus it only starts loading when the browser has started processing the HTML.

CDN’s can terminate a user’s ssl connection close to the user (and keep a persistent one open to the origin), so it is more than a useless hop.

There are also http headers that can instruct the browser to fetch cdn resource before the html is delivered.

Re: Distributed Databases Should Work More Like CDNs

#80

Post by susegado »

russell_h wrote: post #28

I'm curious, what sort of read latency is achievable with CockroachDB? Does it support some notion of tunable read consistency in order to achieve lower read latency at the expense of consistency?

Reads from CockroachDB go through a lease holder for the piece of data being read without needing confirmation from any replicas about consistency, so there is no overhead from replication. But read latency can be affected by writes on the data (conflicts), because it is fundamentally a consistent system (serializable). This is not tunable.
Post reply on HN