Live data from Hacker News

Distributed Databases Should Work More Like CDNs

cockroachlabs.com

61–70 of 86 posts

Re: Distributed Databases Should Work More Like CDNs

#61

The only distributed database I'm familiar with is CouchDB. Can anyone give me a birds eye view on how Cockroach is different?

Unfortunately, it's not on the list for the table of feature comparisons, but here's a start: https://www.cockroachlabs.com/docs/stable/cockroachdb-in-com...

Might be helpful if you already know the couchdb answer for each feature.

Re: Distributed Databases Should Work More Like CDNs

#65
post #13

Imagine a globally replicated and version controlled object store. You could use this for data, assets, source code, anything you like. Would be incredibly useful for both the development side of things, as well as production.

Like IPFS? https://github.com/ipfs/ipfs

Only if others replicate your content. At the moment it's totally voluntary. However filecoin may change that.

Re: Distributed Databases Should Work More Like CDNs

#66
post #56
post #53

Earlier quoted context omitted.

> the only way to have a CA database is to only have a single node. I know that this is meant to mean: in the real world you cannot just write off partition resilience and still call your system highly available, since partitions will happen sooner or later and when they do your CA system won't be available. But in the other hand, having a system that is always available _except_ during a network partition is a usefu…

It's my understanding that Google Spanner is leveraging hardware (direct lines between machines), so the degree at which there is a network partition is different than compared to other systems.

According to a paper by Eric Brewer, the author of the original CAP theorem, Google Spanner technically is not a CA system [1]. It is advertised as such because partitons are supposed to be exceedingly rare as the infrastructure is outstanding and a lot of operating experience is present [1].

I believe at the end of the day the CAP theorem is too fuzzy for such discussions.

[1] https://research.google.com/pubs/pub45855.html

Re: Distributed Databases Should Work More Like CDNs

#67
post #26

> When partitions heal, you might have to make ugly decisions: which version of your customer’s data to you choose to discard? If two partitions received updates, it’s a lose-lose situation. When partitions heal you simply merge all versions through conflict-free replicated data types. No ugly decisions, no sacrificing neither latency nor consistency. We call it strong eventual consistency [1] nowadays. And it's exac…

My application is just plain old CRUD. What if two users want to change e.g. the telephone number of an existing record during a network partition. There just is no obvious way to merge a telephone number. One of them is correct, the other is incorrect. Can CRDTs solve my simple problem?

Last write wins in that case?

Re: Distributed Databases Should Work More Like CDNs

#68
post #31

Earlier quoted context omitted.

My application is just plain old CRUD. What if two users want to change e.g. the telephone number of an existing record during a network partition. There just is no obvious way to merge a telephone number. One of them is correct, the other is incorrect. Can CRDTs solve my simple problem?

I think the issue is solved (while still complex) with an event sourcing system. The chances are really high that the users did not push / updated the number at the exact same time. (so they are at least 1ns apart, even when not it's not that bad). So you can actually restore to a sane state by reapplying the log from scratch. i.e. this is solved with CQRS and Event Sourcing and it probably works in like all database…

1ns according to whom? Clock skew is still a thing :)

Re: Distributed Databases Should Work More Like CDNs

#69
post #39

Earlier quoted context omitted.

Does this imply that there's a single place that log entries get ingested at? Doesn't that make it a single point of failure?

well you can either use a single database for log entries, or cassandra or a mysql cluster or a pg cluster. that doesn't matter.. basially instead of having a table where you update/delete entries you basically append only to a table. so insert only. you can still have another table that will be aggregated from your log table. where you store (i.e. which database system, cluster whatever) your stuff doesn't matter.

That doesn't answer my question. If you're appending to the end of a table, regardless of system you still need one machine somewhere to ingest new records for the same primary key, right?

Re: Distributed Databases Should Work More Like CDNs

#70

Earlier quoted context omitted.

Does this imply that there's a single place that log entries get ingested at? Doesn't that make it a single point of failure?

No, you can just have partitioned replicated topics like kafka has, just make sure that all the events for one user get put into the same partition, so that they get totally ordered.

So I guess the specific issue that grandparent raised was if multiple users wanted to edit the same record. So even if the queue is sharded on some primary key, then for each partition, either a) you only have one machine ingesting and thus a SPOF, or b) you ingest data using multiple machines, which means you need consensus/conflict resolution somewhere, right?
Post reply on HN