CockroachDB 2.0 Performance Makes Significant Strides • Hacker News Board

Re: CockroachDB 2.0 Performance Makes Significant Strides

#171

Post by gregwebs » Fri, Mar 30, 2018, 6:48 PM UTC

manigandham wrote: post #170

jinqueeny wrote: post #169

Earlier quoted context omitted.

As a distributed database system, the highly-layered architecture of TiDB is not a disadvantage because it makes TiDB easier to debug and diagnose when issues occur. The complicity resulted from the separated modules can be easily tackled by automatic deployment tools and microservice framework. For example, TiDB can be easily deployed either using Ansible: https://pingcap.com/docs/op-guide/ansible-deployment/ or Doc…

Sure, I get the advantages of layers in architecture, especially the growing trend of separating compute and storage. However there's a difference between architecture and deployment, and having everything in a single package makes operations much easier. CockroachDB also uses a KV storage layer (using RocksDB) with SQL on top.

TiKV uses Rocks also which is not distributed. TiKV is distributed and used by TiDB. TiSpark also uses TiKV directly which starts to demonstrate some of the advantages of the layered architecture.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#172

Post by Groxx » Fri, Mar 30, 2018, 7:22 PM UTC

thanatos_dem wrote: post #127

Groxx wrote: post #126

Earlier quoted context omitted.

Why write conflicts? Contention, sure, but contention isn't an issue until you have literal tons of it.

From OP's description > issues a write every time someone types into a text field With more than a handful of people, this is getting into conflict territory pretty rapidly, especially if the document is structured as a single row (hopefully it's more granular than that). Time for some back of the envelope maths: Assuming that an average person types at around 200 words per minute (number pulled from https://www.live…

If you're talking about "insert char X at location Y" leading to undesirable changes: sure. But that has nothing to do with the DB type, and everything to do with how changes are resolved. CRDT / etc can all be written on top of any DB.

Updating a DB every 30ms should be trivial. Heck, you should be able to grab an exclusive row lock, double check your state, and write your change without even coming close - 100% conflict or deadlock free, regardless of the number of writers, simply by using a DB as it is designed to be used. In this case, by using the biggest reasonable hammer available: make everything sequential at the DB level. You can absolutely build other systems on a relational DB that don't have that limitation.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#173

Post by manigandham » Fri, Mar 30, 2018, 7:42 PM UTC

gregwebs wrote: post #171

manigandham wrote: post #170

Earlier quoted context omitted.

Sure, I get the advantages of layers in architecture, especially the growing trend of separating compute and storage. However there's a difference between architecture and deployment, and having everything in a single package makes operations much easier. CockroachDB also uses a KV storage layer (using RocksDB) with SQL on top.

TiKV uses Rocks also which is not distributed. TiKV is distributed and used by TiDB. TiSpark also uses TiKV directly which starts to demonstrate some of the advantages of the layered architecture.

I get it. Both databases are layered, but I'm saying deployment is still easier with a single package/binary instead of having TiDB, TiKV and Placement Driver instances.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#174

Post by guipsp » Fri, Mar 30, 2018, 7:54 PM UTC

tyingq wrote: post #90

qaq wrote: post #74

Earlier quoted context omitted.

you do realize it's ancient hardware thats $300-400 USD on ebay now.

http://www.tpc.org/tpcc/results/tpcc_result_detail.asp?id=11... Yeah, but 1700 cores worth. That's still a lot of $300 boxes. Like qty 53 Sparc T3-2's for example. Which seem to be $1200 to $2k on eBay. And unsupported, end of life, etc. I'd compare CockroachDB's number to some more recent result with a similar number of cores. (If you can find one)

Minor correction: They used 54 SPARC T3-2. You can see exactly which components they used in http://c970058.r58.cf2.rackcdn.com/fdr/tpcc/Oracle_SPARC_Sup...

Re: CockroachDB 2.0 Performance Makes Significant Strides

#175

Post by minxomat » Fri, Mar 30, 2018, 9:52 PM UTC

jimbokun wrote: post #168

minxomat wrote: post #150

Earlier quoted context omitted.

CockroachDB's SQL implementation is still heavily unoptimized for actual business queries (subqueries, unnest, (recursive) CTE, schema changes). Features like this either take way too long or are not even implemented yet. Some of them are in the 2.1 milestone though. Let's hope. I'd really love some kind of distributed-for-performance database using the exact optimizer and query planner of SQLite plus std plugins (FT…

"I'd really love some kind of distributed-for-performance database using the exact optimizer and query planner of SQLite plus std plugins..." Why do you think the identical optimizer and query planner would work in a distributed environment, with no changes from the single server implementations?

Sure there have to be (minor) changes and some removals, but at the core that's exactly what Comdb2 did, and it works pretty well.

If Comdb2 7 ever reaches stable, and BLP invests some time into deployment, ops and monitoring tools/docs, it'll be a strong competitor.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#176

Post by atombender » Sat, Mar 31, 2018, 12:06 AM UTC

siddontang wrote: post #167

atombender wrote: post #105

Earlier quoted context omitted.

TiDB looks promising, but it doesn't have serializable transactions at all, which makes it something of an apples-to-oranges comparison at the moment when it comes to OLTP. TiDB has a weird kind of variation on "read committed" where you get phantom reads (though they're not called that in the documentation, which is actually ambiguous on this point). This is a problem for apps that expect consistency.

Siddon Tang from TiDB here. Thanks for your interest in TiDB! TiDB uses Snapshot Isolation ( https://en.wikipedia.org/wiki/Snapshot_isolation ), which is similar to Repeated Read in MySQL. It doesn’t allow Read Phantom but can’t avoid Write Skew. IMO, this can work well for OLTP in most of the cases. If you really care about serializability, you can use `select for update` to promote the Read as Write explicitly like…

Thanks, that's helpful!

Re: CockroachDB 2.0 Performance Makes Significant Strides

#177

Post by lobster_johnson » Sat, Mar 31, 2018, 12:08 AM UTC

orangechairs wrote: post #163

atombender wrote: post #103

Looks very promising! We've looked at Cockroach for a particular project, and we've been concerned that performance wasn't good enough. Cockroach performance seems to scale linearly, but single-connection performance, especially for small transactions, seems rather dismal. Some casual stress testing against a 3-node cluster on Kubernetes showed that small transactions modifying a single row could take as much as 7-8…

[cockroach employee here] Have you hit us up on Gitter or Stack Overflow to help debug and tune? We'd also love to learn more about how you're using K8s, what your setup looks like, surprises you're running into with it, etc.

Thanks, I'll do that when I get back to testing.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#178

Post by siddontang » Sat, Mar 31, 2018, 1:17 AM UTC

manigandham wrote: post #173

gregwebs wrote: post #171

Earlier quoted context omitted.

TiKV uses Rocks also which is not distributed. TiKV is distributed and used by TiDB. TiSpark also uses TiKV directly which starts to demonstrate some of the advantages of the layered architecture.

I get it. Both databases are layered, but I'm saying deployment is still easier with a single package/binary instead of having TiDB, TiKV and Placement Driver instances.

TiDB provides many some ways to let the deployment easier. If you just want to have a try on the single machine, we provide docker-compose (https://github.com/pingcap/tidb-docker-compose). But if you don't like docker, the single binary may be easier.

For the business users, we support ansible and K8s deployment, both can help them run the TiDB easily and quickly in production. The deployment is not a problem so far.

IMO, for the distributed system, if you need to operate many instances (10, 100 or even more), what you pay attention to most is not whether the instance is one single binary or not, but is how to operate all these easily. At that time, there is no much difference.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#179

Post by etaioinshrdlu » Sat, Mar 31, 2018, 2:28 AM UTC

thinkloop wrote: post #147

etaioinshrdlu wrote: post #94

Project idea: globally hosted / managed CockroachDB that lets developers quickly start building small apps cheaply or free using this database. This database has the potential to dethrone Spanner in a major way.

I just started playing with spanner. The API is nice. Simple txs are a bit slow - seconds instead of milliseconds - ok for my use-case. I don't hear much about it though, few articles on HN, I was wondering how mature it was and how widely used it is. Is it common knowledge that it is used by a lot of orgs?

De-throne in the sense of being the top-tier multi-master consistent transactional DB. I doubt too many people use Spanner outside of Google in reality. Google would likely eventually adopt cockroackdb if it was clearly better long term.

Re: CockroachDB 2.0 Performance Makes Significant Strides

#180

Post by shaklee3 » Sat, Mar 31, 2018, 3:27 PM UTC

manigandham wrote: post #79

shaklee3 wrote: post #54

Earlier quoted context omitted.

kdb+?

Sure, but it's far more expensive and not as generally usable as the mysql-flavored MemSQL for common data warehouse scenarios. Performance will be similar but there are differences in functionality like kdb's asof joins that can't really be compared. kdb+ is much better for numeric/financial analysis apps, especially when used with the integrated query language and interpreter environment.

The statement was that you will not get better performance than memsql, not about which is cheaper.