Live data from Hacker News

Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

github.com

41–50 of 70 posts

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#41
post #2

Why the 'Ex-Googlers' in the title. Is it like a seal of approval or something?

A requirement of a successful project/product is visibility. Ex-Googlers does not imply much more then "used to work there" but if it drives more attention to them, it's good MarCom.

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#42
post #26

Many things that I don't agree with: https://github.com/cockroachdb/cockroach/ MySQL: Weak consistency Cassandra: No availability or weak consistency with datacenter failure

MySQL provides strong consistency when run on a single machine, but that breaks down when you handle failover using asynchronous replication between DCs.

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#43
post #29
post #21

Earlier quoted context omitted.

Does it? I don't think ex-Googlers gives anything of value here.

It indicates that the project is worth a closer look rather than waiting or dismissing it outright.

Why? Being an ex-googler could just as easily imply they were incompetent and fired.

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#44
post #17
post #2

Why the 'Ex-Googlers' in the title. Is it like a seal of approval or something?

Yes, it's called social proof. If you're observant you'll notice that people use it everywhere. Of course, social proof doesn't mean guarantees.

Nah, it's honor-by-association fallacy. Social proof is a behavioral thing, not false reasoning.

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#45
post #18

Are there distributed data stores like this that are also resilient to intentional sabotage? I've been looking recently at long-term digital preservation systems -- tools designed to archive large amounts of data for decades. This is the Library of Alexandria problem -- how do we preserve all this data we're generating against once-in-a-century disasters? So this 2005 paper lists thirteen different threats to long-te…

The problem with internal/external attacks is that we (the society) don't really want to prevent it. The reason is simple: child porn. To date, Bitcoin block chain (and related ideas) is the only data-storage that is 100% resistant to attacks (i.e. changing history), but luckily it cannot handle amounts of data large enough to be viable for child porn (or most other forms of media). Tor, on the other hand, gets a bad rep precisely because it doesn't prevent it (despite its numerous other, beneficial, uses).

The core of the issue is that humans view different information differently (child porn vs. Mona Lisa), whereas for computers, bits are bits and numbers are numbers. As long as child porn remains illegal and socially unacceptable, we'll want to enable attacks on data, i.e. for someone (usually internal operators) to be able to delete some kind of information, corrupt it or at least track it. Of course, this necessarily means that all information stored in the same data-store will be vulnerable.

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#46
It's really strange to see networked databases go through this iterative design fad. It was file transfers back in the 90s/00s; everyone had their own distributed decentralized file transfer solution. Little known fact, Gentoo's Portage almost became an internet-wide distributed decentralized public file system. Thank god they abandoned that idea. Can you imagine trying to debug a file transfer error just to get an mp3 player to install on your machine? I can't wait until databases go back to being mainframes.

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#47
post #43
post #29

Earlier quoted context omitted.

It indicates that the project is worth a closer look rather than waiting or dismissing it outright.

Why? Being an ex-googler could just as easily imply they were incompetent and fired.

So, you're assuming that half of all ex-googlers were fired? Otherwise, there aren't equal probabilities.

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#49
post #16

Earlier quoted context omitted.

The design doc https://docs.google.com/document/d/11k2EmhLGSbViBvi6_zFEiKzu... says, "TBD: how to avoid partitions? Need to work out a simulation of the protocol to tune the behavior and see empirically how well it works" .. so they've gone for CA and forgotten about P.

You're taking a sentence out of context and giving it the most uncharitable possible interpretation. They certainly haven't "forgotten" about network partitions because if you actually read the design document, instead of ctrl-F'ing for the word "partition", they talk about the mechanisms they use to ensure sequential consistency. The software is not yet at the point of being testable AFAIK, but clearly the intent is…

I'll admit to aggressive use of ctrl-F, but only because I too felt they should be a lot clearer about what the intended properties are.

From the intro page:

"Cockroach is a distributed key/value datastore which supports ACID transactional semantics and versioned values as first-class features. The primary design goal is global consistency and survivability, hence the name. Cockroach aims to tolerate disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention."

If we read 'survivability' as 'availability', then that would suggest they've gone for CA. Although closer inspection reveals that their architecture seems to be made of shards each of which is maintained with Raft/Paxos. An evaluation of this by the Cambridge Computer laboratory is here: http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-857.pdf

That report makes two points relevant to this discussion. One is that a hard definition of C A and P can be difficult and that it's possible to achieve all three almost all of the time in real conditions. The other is from the conclusion:

"In particular, a [Raft] cluster can be rendered useless by a frequently disconnected node or a node with an asymmetric partition"

Re: Ex-Googlers CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore

#50
post #26

Many things that I don't agree with: https://github.com/cockroachdb/cockroach/ MySQL: Weak consistency Cassandra: No availability or weak consistency with datacenter failure

To add to this "Postgres: Limited scalability". I wish someone told us before we ticked over to 31TB... 5 Months ago.
Post reply on HN