Serializable, Lockless, Distributed: Isolation in CockroachDB • Hacker News Board

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#21

Post by teraflop » Thu, May 05, 2016, 2:17 AM UTC

andrewvc wrote: post #13

tyingq wrote: post #4

Earlier quoted context omitted.

It's not a nosql database if that's what you're thinking. https://github.com/cockroachdb/cockroach/issues/2970

Joins in distributed databases all must make some sort of unsavory tradeoff be it speed or space or limiting what you can join. So, while any distributed database can do a join, it may not be fast or flexible enough to be worth it.

One could say exactly the same thing about joins in non-distributed databases. "You have to make some kind of tradeoff" is a platitude.

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#22

Post by ams6110 » Thu, May 05, 2016, 3:17 AM UTC

tdrd wrote: post #16

lobster_johnson wrote: post #15

Earlier quoted context omitted.

Can Cockroach do the equivalent of a "select ... for update" (e.g., PostgreSQL), where you lock one thing while applying changes elsewhere? Concrete example: We have app that has a "documents" table and a "translog" table. The translog is like a series of diff-patches, representing changes to the documents. When we write to the translog, we first lock the document with a "select ... for update", so that no intervenin…

(employee here) It seems to me that your use-case does not require locking specifically - you just want to make sure no concurrent transactions can clobber your "update A". As mrtracy explained, such overlapping transactions are linearizable in CockroachDB, so this invariant is preserved without the need for explicit locking.

I think it does require locking, because in PostGres (or Oracle) readers do not block writers and writers do not block readers. So to be sure you update the same version you read, you have to select...for update.

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#23

Post by biokoda » Thu, May 05, 2016, 5:42 AM UTC

andrewvc wrote: post #13

tyingq wrote: post #4

Earlier quoted context omitted.

It's not a nosql database if that's what you're thinking. https://github.com/cockroachdb/cockroach/issues/2970

Joins in distributed databases all must make some sort of unsavory tradeoff be it speed or space or limiting what you can join. So, while any distributed database can do a join, it may not be fast or flexible enough to be worth it.

Joins in ActorDB work great. Of course that is because we use an entirely different way of making an SQL database distributed and our joins aren't actually distributed even though the database is.

Best way to solve a problem is to avoid it.

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#24

Post by chris_wot » Thu, May 05, 2016, 5:52 AM UTC

petermattis wrote: post #19

chillaxtian wrote: post #3

Earlier quoted context omitted.

> Looking forward to when joins are implemented so I can try using it with something non-trivial. these types of databases typically don't have joins. you'll be waiting a while.

(employee/founder here) Re joins: we have top men working on it right now. Joins are definitely an interesting problem for a distributed database.

Hi Peter, sounds pretty awesome! Quick question though - on your front page you say that CockroachDB does SQL - but if it can't do a join, then how can you say it uses SQL? Or is distributed SQL a different thing entirely? It does sounds like a very limited SQL subset though... I'm sure I must be missing something as I'm not familiar with your product.

Also, what levels of isolation do you actually offer? Serialized snapshot isolation appears to be MVCC, but I see you also have just snapshot isolation - what is the difference?

Edit: oh brother, the proof you link to, I just realized I bought that book some time ago and never got around to reading it... Transactional Information Systems by Weikum & Vossen, right? time for me to hit the books I guess. Still trying to get my head around that second graph you have drawn, can't work out how you have gotten it :(

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#25

Post by LoSboccacc » Thu, May 05, 2016, 5:55 AM UTC

By coincidence I was looking around for a datastore. I almost settled on couchbase when I saw this; easy cluster deployment is my main draw after looking at how bad setting up master-master system is in other solutions (hbase, mongo, couchdb all have eterogeneus nodes and weird failure modes)

One quick question since I saw the devs around and I can't find a final answer on it on google: are there any strong roadblock or performance drawbacks against storing principally medium size binary data like images, say, Yeah I could use something more appropriate but I'd be back to figuring out hadoop installation or fighting s3 eventual consistency.

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#26

Post by teraflop » Thu, May 05, 2016, 6:31 AM UTC

ams6110 wrote: post #22

tdrd wrote: post #16

Earlier quoted context omitted.

(employee here) It seems to me that your use-case does not require locking specifically - you just want to make sure no concurrent transactions can clobber your "update A". As mrtracy explained, such overlapping transactions are linearizable in CockroachDB, so this invariant is preserved without the need for explicit locking.

I think it does require locking, because in PostGres (or Oracle) readers do not block writers and writers do not block readers. So to be sure you update the same version you read, you have to select...for update.

Having serializable transactions is equivalent to adding "FOR UPDATE" to every SELECT statement, so it sounds like CockroachDB already does what you want.

A typical RDBMS will prevent conflicts by forcing queries to block until they can be executed in a conflict-free ordering. CockroachDB instead detects conflicts after the fact and prevents inconsistent transactions from committing, forcing them to retry. The end result -- that is, the set of possible outcomes of a series of transactions -- is the same, but the performance characteristics will be different.

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#27

Post by jandrewrogers » Thu, May 05, 2016, 6:42 AM UTC

chillaxtian wrote: post #3

tyingq wrote: post #2

I love the transparency this project operates with. Looking forward to when joins are implemented so I can try using it with something non-trivial.

> Looking forward to when joins are implemented so I can try using it with something non-trivial. these types of databases typically don't have joins. you'll be waiting a while.

It is possible to massively distribute joins but it requires a much more sophisticated database design than you see in most distributed database implementations because the whole system has to be designed for that use case e.g. support for ad hoc inter-node orchestration and data flows. At that point you basically have a parallel database instead of just a distributed one.

And if you add an additional requirement to correctly execute ad hoc consistent joins under very high write workloads and continuous failures, then it is definitely non-trivial engineering. But it is possible, I've designed implementations like this before, it just requires a very high degree of effort and skill that is rarely applied to the design of typical NoSQL databases.

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#28

Post by knz42 » Thu, May 05, 2016, 7:47 AM UTC

chris_wot wrote: post #24

petermattis wrote: post #19

Earlier quoted context omitted.

(employee/founder here) Re joins: we have top men working on it right now. Joins are definitely an interesting problem for a distributed database.

Hi Peter, sounds pretty awesome! Quick question though - on your front page you say that CockroachDB does SQL - but if it can't do a join, then how can you say it uses SQL? Or is distributed SQL a different thing entirely? It does sounds like a very limited SQL subset though... I'm sure I must be missing something as I'm not familiar with your product. Also, what levels of isolation do you actually offer? Serialized…

SQL is not a single language but rather a family of languages. Each RDBMS that advertises support for SQL ends up implementing a different flavor of SQL, most of the times they are not even compatible :)

For now CockroachDB's supports a subset of the SQL implemented by other databases, with some extensions of its own. This subset will grow over time.

The two levels of isolation offered are snapshot (SI) and serializable (SSI). Snapshot means that concurrent transactions are atomic with regards to each other and "see" the same initial state from the DB. Serializable adds into this that they can't introduce write skews, ie if there are concurrent transactions they will "see" each others effects in some order.

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#29

Post by ansible » Thu, May 05, 2016, 8:05 AM UTC

LoSboccacc wrote: post #25

By coincidence I was looking around for a datastore. I almost settled on couchbase when I saw this; easy cluster deployment is my main draw after looking at how bad setting up master-master system is in other solutions (hbase, mongo, couchdb all have eterogeneus nodes and weird failure modes) One quick question since I saw the devs around and I can't find a final answer on it on google: are there any strong roadblock…

I don't know that you would be happy trying to store blobs of that size in any distributed database, current or future.

I'd suggest having an immutable distributed blob store, and just store the index in the distributed database.

Of course, now you are half way towards a distributed filesystem...

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#30

Post by shin_lao » Thu, May 05, 2016, 8:13 AM UTC

petermattis wrote: post #19

chillaxtian wrote: post #3

Earlier quoted context omitted.

> Looking forward to when joins are implemented so I can try using it with something non-trivial. these types of databases typically don't have joins. you'll be waiting a while.

(employee/founder here) Re joins: we have top men working on it right now. Joins are definitely an interesting problem for a distributed database.

You may find that you will have to choose at best between two of these three: joins, speed and convenience. In other words, you won't be able to join quickly in a convenient way for the users, of if you want the user to join like he can in SQL database, it will be much slower.