Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CockroachDB 1.0

cockroachlabs.com

Post reply on HN
Print view View original

281–290 of 366 posts

Re: CockroachDB 1.0

#281

Post by irfansharif »

ww520 wrote: post #278
sixdimensional wrote: post #138

How does Cockroach efficiently handle the shuffle step when data is on many nodes on the cluster and has to move to be joined? Does Cockroach need high capacity network links to function well? I always see companies making the claim of linear speedup with more nodes but surely that can't be the case if the nodes are geographically disjointed over anything less than gigabit links? Perhaps linear speedup with more node…

15 years ago I was working on a similar distributed DB product. At the time, the idea was to send the query execution plan to each node to execute any filtering criteria to trim down the candidate row set. Then compute a Bloom Filter on the joining keys on the node with the largest candidate set (using some heuristic statistics), ship the Bloom Filter to other nodes with smaller data set to greatly reduce the non-mat…

haven't come across this idea before, interesting - will definitely have to give it some more thought. our 'distributed joins', so to speak, run through our distributed query execution model (distsql) setting up incremental 'stages' of computation with the results pipelined and plumbed through individual computes. viewing it through this model our implementation more closely resembles the Grace Hash Join[1] algorithm. you might be interested in the PR[2] that landed this changeset, there's a cool visualization in one of the comments[3] showing the query execution plan.

[1]: https://en.wikipedia.org/wiki/Hash_join#Grace_hash_join

[2]: https://github.com/cockroachdb/cockroach/pull/12221

[3]: https://github.com/cockroachdb/cockroach/pull/12221#issuecom...

Re: CockroachDB 1.0

#283

Post by raarts »

johnwheeler wrote: post #28

I think the name "Cockroach" was a really poor decision from a marketing standpoint. The team intended to convey durability, since cockroaches can live through anything. But when I think of a cockroach, I think, gross, disgusting, etc.

I think we will actually see real cockroaches' reputation improved because of this product

Re: CockroachDB 1.0

#284

Post by irfansharif »

raarts wrote: post #282

On a three node cluster will it survive two nodes going down?

short answer: nope. cockroachdb replicates data for availability and in order to guarantee consistency across the replicas, it uses Raft[1] internally. Raft necessitates a majority of the replicas remain available in order to operate. it ensures that a new 'leader' for each group of replicas is elected if the former leader fails, so that transactions can continue and affected replicas can rejoin their group once they're back online.

[1]: https://raft.github.io/raft.pdf

Re: CockroachDB 1.0

#285

Post by SilasX »

EnFinlay wrote: post #244
SilasX wrote: post #234

Earlier quoted context omitted.

I elaborated in much (probably too much) greater detail on the relevance of the bikeshedding metaphor here: https://news.ycombinator.com/item?id=14310444 I would appreciate your thoughts. With respect to your specific point: if we could resolve how much it matters, then yes, that would obviate the debate. But the bikeshedding metaphor doesn't add much there because it's precisely in dispute about how much it matters.

Well know I feel like a bit like an asshole with how abrasive my response was, sorry. Kudos to you for not escalating. I agree that it resolves to how much it matters, and I guess I disagree with you on how much it matters. How it relates to the bike shedding metaphor is starting to feel like a semantic argument, which is not something I want to continue. In response to your escalated names like PubesDB... my opinion…

>Well know I feel like a bit like an asshole with how abrasive my response was, sorry. Kudos to you for not escalating.

I didn't feel it was abrasive at all.

For my part, I'm just upset that I went to such great lengths (in the comment I linked) to unpack where the bikeshed metaphor does or doesn't apply, disentangling the various issues and merging them into a general understanding, right where that comment was needed, and yet that's the one that no one is responding to... (what's worse, it was downvoted less than a minute after I posted it ).

>In response to your escalated names like PubesDB... my opinion is that I agree I wouldn't work with them, not because of any internal disgust reaction, but because the name signals a level of maturity that I don't want in my stack. Some people might have the same reaction to Cockroaches.

Right, like I said, "we're haggling over the details"; it should be regarded as a question of which names are so disgusting to be out of the question, yet people are dismissing the entire naming issue as "lol emotional primates".

Re: CockroachDB 1.0

#286

Post by Teichopsia »

SilasX wrote: post #264
genkaos wrote: post #197

Earlier quoted context omitted.

I'm a spaniard and mongo is an insult. [removed unnecessary snarky comment]

Okay, there are separate issues going on here; let me try to clarify: Is "mongo" the equivalent of English "retard", in terms of being a low-class insult that invokes a visceral reaction among the majority of the population? I didn't believe that at first; if so, why didn't anyone ever put it in Wikipedia? English has "retard" (in the pejorative sense): https://en.wikipedia.org/wiki/Retard#Other_uses And why doesn't…

First of all, Wikipedia isn't infallible. Second, genkaos and myself grew up on different sides of the world. Culturally different, and yet, in our own respective cultures we learned, however wrong it is, the implicit meaning of mongo when used derogatively. It doesn't have several meanings as you pointed out, it has one, which does not mean it is the same for genkaos. As you pointed out in the Wikipedia link, it refers to certain type of people. When used as an insult, towards to a, whether the person is white, Hispanic, black, whatever, it is implied that that person is that sort of person and a retard. Thus, my comment of it being racist as well.

Now, even if I had associated MongoDB with that explanation, and now that I do remember it's inherent meaning under a certain context, I take no offense in it since the people behind MongoDB didn't have that intent. Obviously this is an assumption on my part.

Let us not get derailed from the main point, which is the 'visceral' feelings that cockroachDB has on so many people as you mentioned in several comments. It is true, it happens to me as well. But not the word itself, but when I'm around one. Those feelings of fear, whatever, when around one are irrational. I don't remember the explanation why it's irrational, I've never worked in the field of psychology.

Re: CockroachDB 1.0

#289

Post by philsnow »

socmag wrote: post #78
irfansharif wrote: post #31

Earlier quoted context omitted.

You might find our post[1] on atomic clocks, rather having to do without them, partially interesting. [1]: https://www.cockroachlabs.com/blog/living-without-atomic-clo...

Hey guys, I'm a fellow developer of distributed systems here. First of all I think what you are doing is great. My question is what's the point of clocks at all? The current time is a very subjective matter and I'm sure you know this, the only real time is at the point when the cluster receives the request to commit. Anything else should be considered hearsay. Specifically the time source of any client is totally mea…

> Specifically the time source of any client is totally meaningless since as you say further in the discussion that client machine times can be off by huge margins.

distributed systems like cockroach shouldn't use the client's conception of current time for anything at all, except possibly to store it (_verbatim_, don't interpret it) and relay it back to the client or to other clients (and let the client interpret it however they want).

Re: CockroachDB 1.0

#290

Post by melloc »

jordanlewis wrote: post #164
daliwali wrote: post #156

CockroachDB looks like a great alternative to PostgreSQL, congrats to the team for doing so much in such a short time. The wire protocol is compatible with Postgres, which allows re-using battle-tested Postgres clients. However it's a non-starter for my use case since it lacks array columns, which Postgres supports [0]. I also make use of fairly recent SQL features introduced in Postgres 9.4, but I'm not sure if ther…

I'm an engineer on the SQL team at CockroachDB. We're very aware of our missing support for array column types - and in fact beginning to add support for arrays is one of my team's priorities for the next release cycle. What kind of other recent SQL features introduced in Postgres 9.4 do you use? Postgres has a ton of features, as I'm sure you're aware, and while we strive for wire compatibility with Postgres it's no…

Postgres' network types [0] are very useful, especially when you use the > operators to query for addresses contained in a subnet.

[0] https://www.postgresql.org/docs/9.5/static/datatype-net-type...

Post reply on HN