11–20 of 54 posts
Sorry if this is a little bit off topic This is a startup with employees that is making a non-commercial open source database system. How will they make a financial return on such a product?
Joyent/Node Sonatype Nexus
Sorry if this is a little bit off topic This is a startup with employees that is making a non-commercial open source database system. How will they make a financial return on such a product?
It's common in the industry to build an open source product and then do paid support, hosting, etc with it. Joyent/Node Sonatype Nexus
Earlier quoted context omitted.
> Looking forward to when joins are implemented so I can try using it with something non-trivial. these types of databases typically don't have joins. you'll be waiting a while.
It's not a nosql database if that's what you're thinking. https://github.com/cockroachdb/cockroach/issues/2970
So, while any distributed database can do a join, it may not be fast or flexible enough to be worth it.
I love the transparency this project operates with. Looking forward to when joins are implemented so I can try using it with something non-trivial.
> Looking forward to when joins are implemented so I can try using it with something non-trivial. these types of databases typically don't have joins. you'll be waiting a while.
For instance, suppose you want to join on a column that's not a primary key. No big deal in a typical RDBMS; just do an index lookup on the join column. But if your data is sharded across machines, that doesn't work, because you can't update the index without running the risk that readers will observe it in an inconsistent state. Unless you can do transactions, that is.
So I'm optimistic that Cockroach will get join support before too long. The necessary primitives are available, making it "merely" a matter of figuring out the engineering details.
How does this work if the clocks drift between the nodes? Does this allow incorrect behavior because one transaction looks like it happened before another?
(blog author here) Interestingly, clock drift does not affect the serializability of the transaction history; this system guarantees that the history is serializable, regardless of clock drift. However, "serializable" only means that the history is equivalent to some serial ordering of transactions - it makes no guarantee that the equivalent serial ordering is consistent with the real-time ordering of the involved tr…
Concrete example: We have app that has a "documents" table and a "translog" table. The translog is like a series of diff-patches, representing changes to the documents. When we write to the translog, we first lock the document with a "select ... for update", so that no intervening translog entries can be written concurrently against the same document, then we patch the document, and then we write the translog entry and commit.
We do this with Postgres, and we can do the same thing with Redis' MULTI since Redis is completely single-threaded. I can't think of any other NoSQL data store that allows a similar "lock A, update A, insert B, unlock A"; for example, Cassandra's "lightweight transactions" are only transactional in the context of a single row.
(By "lock" I'd also accept optimistic locking, where you can retry on failure.)
Earlier quoted context omitted.
(blog author here) Interestingly, clock drift does not affect the serializability of the transaction history; this system guarantees that the history is serializable, regardless of clock drift. However, "serializable" only means that the history is equivalent to some serial ordering of transactions - it makes no guarantee that the equivalent serial ordering is consistent with the real-time ordering of the involved tr…
Can Cockroach do the equivalent of a "select ... for update" (e.g., PostgreSQL), where you lock one thing while applying changes elsewhere? Concrete example: We have app that has a "documents" table and a "translog" table. The translog is like a series of diff-patches, representing changes to the documents. When we write to the translog, we first lock the document with a "select ... for update", so that no intervenin…
It seems to me that your use-case does not require locking specifically - you just want to make sure no concurrent transactions can clobber your "update A".
As mrtracy explained, such overlapping transactions are linearizable in CockroachDB, so this invariant is preserved without the need for explicit locking.
Earlier quoted context omitted.
It's not a nosql database if that's what you're thinking. https://github.com/cockroachdb/cockroach/issues/2970
Joins in distributed databases all must make some sort of unsavory tradeoff be it speed or space or limiting what you can join. So, while any distributed database can do a join, it may not be fast or flexible enough to be worth it.
Please forgive me, but what are you really trying to get at? Otherwise it just seems like naysaying and rabble rousing to me..
Earlier quoted context omitted.
Can Cockroach do the equivalent of a "select ... for update" (e.g., PostgreSQL), where you lock one thing while applying changes elsewhere? Concrete example: We have app that has a "documents" table and a "translog" table. The translog is like a series of diff-patches, representing changes to the documents. When we write to the translog, we first lock the document with a "select ... for update", so that no intervenin…
(employee here) It seems to me that your use-case does not require locking specifically - you just want to make sure no concurrent transactions can clobber your "update A". As mrtracy explained, such overlapping transactions are linearizable in CockroachDB, so this invariant is preserved without the need for explicit locking.
What I need is for our translog to reflect the order of updates. So if diff A was applied before B, then the translog order also needs to be A, B. (The order only needs to be consistent per document.)
This is because we have listeners — through APIs — that play the translog as it happens and maintain various state based on it.
Currently, the translog is ordered by a sequential number (because it's cheap in Postgres), but every entry also records the ID of the previous entry (so B will point at A). One could sort by time and then reorder by causality before emitting the linear log to consumers, but that would of course be more complicated than one that is already linear.
I love the transparency this project operates with. Looking forward to when joins are implemented so I can try using it with something non-trivial.
> Looking forward to when joins are implemented so I can try using it with something non-trivial. these types of databases typically don't have joins. you'll be waiting a while.
Re joins: we have top men working on it right now. Joins are definitely an interesting problem for a distributed database.
Earlier quoted context omitted.
(blog author here) Interestingly, clock drift does not affect the serializability of the transaction history; this system guarantees that the history is serializable, regardless of clock drift. However, "serializable" only means that the history is equivalent to some serial ordering of transactions - it makes no guarantee that the equivalent serial ordering is consistent with the real-time ordering of the involved tr…
Can Cockroach do the equivalent of a "select ... for update" (e.g., PostgreSQL), where you lock one thing while applying changes elsewhere? Concrete example: We have app that has a "documents" table and a "translog" table. The translog is like a series of diff-patches, representing changes to the documents. When we write to the translog, we first lock the document with a "select ... for update", so that no intervenin…
The transaction would: 1. Read the current document (i'm assuming this needs to be done to compute the translog). 2. Read the latest ID in the translog table 3. Write a new entry to translog with ID+1 4. Write the document.
If any other transaction interleaves with this process (by either reading or writing one of the same keys in a way that would violate isolation), one of the two transactions will be aborted.