CockroachDB 2.0 Performance Makes Significant Strides • Hacker News Board

Re: CockroachDB 2.0 Performance Makes Significant Strides

#141

Post by ComputerGuru » Fri, Mar 30, 2018, 4:25 AM UTC

elvinyung wrote: post #108

d4l3k wrote: post #104

Earlier quoted context omitted.

Yeah, though from what I understand this benchmark is measuring both transactional read and write performance rather than just join performance. Transactional writes are likely the slowest thing since they need to talk to all replicas.

Actually hmm, do reads need to talk to all replicas in this case (serializable isolation)?

As I understand it, reads only need to talk to the lease owner, which in turn bypasses raft since write consensus guarantees atomicity of after completion of write intents. Cockroach tries best-effort to have the lease owner and the raft leader be the same.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#142

Post by grzm » Fri, Mar 30, 2018, 4:29 AM UTC

ComputerGuru wrote: post #140

Gigablah wrote: post #138

Earlier quoted context omitted.

I think people are generally allowed to judge themselves without industry credentials...

That was tongue-in-cheek (that still had no business on HN), a reference to @lacker's two Putnam awards.

The Putnam is also the source of one of the best burns on HN: https://news.ycombinator.com/item?id=35079

Re: CockroachDB 2.0 Performance Makes Significant Strides

#143

Post by gregwebs » Fri, Mar 30, 2018, 4:35 AM UTC

atombender wrote: post #105

wilbeibi wrote: post #93

The thing I really don't get is why CockroachDB is avoid benchmarking with it's rival tidb ( https://github.com/cockroachdb/docs/issues/1412 ). tidb already pretty mature, used in many big companies (Let's say, Didi, which on the similar scale data with Uber, and banks). Even if I like CockroachDB's pg sql more, it would be helpful to have the comparison/benchmark to show something more.

TiDB looks promising, but it doesn't have serializable transactions at all, which makes it something of an apples-to-oranges comparison at the moment when it comes to OLTP. TiDB has a weird kind of variation on "read committed" where you get phantom reads (though they're not called that in the documentation, which is actually ambiguous on this point). This is a problem for apps that expect consistency.

The documentation states that phantom reads are not possible (under the default isolation level or repeatable read): https://github.com/pingcap/docs/blob/master/sql/transaction-...

Re: CockroachDB 2.0 Performance Makes Significant Strides

#144

Post by SoulMan » Fri, Mar 30, 2018, 4:53 AM UTC

joris wrote: post #122

etaioinshrdlu wrote: post #94

Project idea: globally hosted / managed CockroachDB that lets developers quickly start building small apps cheaply or free using this database. This database has the potential to dethrone Spanner in a major way.

That’s on their roadmap: https://www.cockroachlabs.com/docs/stable/frequently-asked-q...

My Org/team is too conservative to use this is they have to hire ops and too froogle to use spanner.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#145

Post by tomerbd » Fri, Mar 30, 2018, 4:59 AM UTC

AlimJaffer wrote: post #96

netghost wrote: post #86

Earlier quoted context omitted.

What kind of workload are you using it for? What's been your biggest win while using it?

A couple of our use-cases include: good KV access (stored user data etc.) and listing blocks of data that has been pre-sorted on disk at insert time (leaderboard records, chat message history etc.). As well, the clustering technology is particularly useful at scale. We work in the games space with some very large games in production, which allows us to spread the load across multiple database nodes and offers us peac…

What do you mean by KV access isn't it a relational data store? do you store the value as a blog? json? if so how do you then do queries on that value's values?

Re: CockroachDB 2.0 Performance Makes Significant Strides

#146

Post by tedmiston » Fri, Mar 30, 2018, 5:41 AM UTC

_wmd wrote: post #20

xstartup wrote: post #18

Earlier quoted context omitted.

We use clickhouse cluster with 1000 nodes and 50000 GB clickstream data.

That's only 50gb per node. Why do you/Clickhouse need so many nodes?

Maybe some space is dedicated to replication? Or for query execution temp space like Redshift. Or he could be trying to keep everything in memory.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#147

Post by thinkloop » Fri, Mar 30, 2018, 6:31 AM UTC

etaioinshrdlu wrote: post #94

Project idea: globally hosted / managed CockroachDB that lets developers quickly start building small apps cheaply or free using this database. This database has the potential to dethrone Spanner in a major way.

I just started playing with spanner. The API is nice. Simple txs are a bit slow - seconds instead of milliseconds - ok for my use-case. I don't hear much about it though, few articles on HN, I was wondering how mature it was and how widely used it is. Is it common knowledge that it is used by a lot of orgs?

Re: CockroachDB 2.0 Performance Makes Significant Strides

#148

Post by kevincox » Fri, Mar 30, 2018, 7:21 AM UTC

SoulMan wrote: post #144

joris wrote: post #122

Earlier quoted context omitted.

That’s on their roadmap: https://www.cockroachlabs.com/docs/stable/frequently-asked-q...

My Org/team is too conservative to use this is they have to hire ops and too froogle to use spanner.

Why do you think that hosted cochroachdb would be cheaper then hosted spanner? Google has been optimizing spanner performance for years so I would expect that it will be cheaper to run for quite a while. Of course the markup can be different but I wouldn't expect it to make a huge difference.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#149

Post by pieterhg » Fri, Mar 30, 2018, 7:22 AM UTC

Great stuff but this name really doesn’t work. Make it a name with positive connotations.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#150

Post by minxomat » Fri, Mar 30, 2018, 7:26 AM UTC

wilbeibi wrote: post #93

The thing I really don't get is why CockroachDB is avoid benchmarking with it's rival tidb ( https://github.com/cockroachdb/docs/issues/1412 ). tidb already pretty mature, used in many big companies (Let's say, Didi, which on the similar scale data with Uber, and banks). Even if I like CockroachDB's pg sql more, it would be helpful to have the comparison/benchmark to show something more.

CockroachDB's SQL implementation is still heavily unoptimized for actual business queries (subqueries, unnest, (recursive) CTE, schema changes). Features like this either take way too long or are not even implemented yet. Some of them are in the 2.1 milestone though. Let's hope.

I'd really love some kind of distributed-for-performance database using the exact optimizer and query planner of SQLite plus std plugins (FTS5, JSON, transitive_closure, spatial). Something like a mix between Bloomberg's comdb2 (which uses a modified SQLite frontend) and rqlite (distributed-for-safety).

Note: You can save most of the shortcomings of CDB on the SQL client side today, but don't underestimate the time it takes to implement CDB-specific workarounds...