Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serializable, Lockless, Distributed: Isolation in CockroachDB

cockroachlabs.com

Post reply on HN
Print view View original

51–54 of 54 posts

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#51

Post by LoSboccacc »

hnkimb3558 wrote: post #50
LoSboccacc wrote: post #25

By coincidence I was looking around for a datastore. I almost settled on couchbase when I saw this; easy cluster deployment is my main draw after looking at how bad setting up master-master system is in other solutions (hbase, mongo, couchdb all have eterogeneus nodes and weird failure modes) One quick question since I saw the devs around and I can't find a final answer on it on google: are there any strong roadblock…

(employee/founder here) Anything under 64K is perfectly reasonable to store in a CockroachDB column. Between 64K and maybe 1M is trending towards trouble. Values greater than this are going to go through CockroachDB like a goat through a python. Why is this the case? For starters, at the level of RocksDB, values greater than 64K are not jammed into SSTables (to avoid constantly rewriting them during compactions of th…

Thanks! That's fantastic to hear even if it's going to materialize later/eventually/never at least it's great to know the need is recognized.

Disregarding write amplification issues for a sec, would it make things better to split binaries in 64k chunks and have them in a chunk table keyed by name and offset?

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#52

Post by hnkimb3558 »

LoSboccacc wrote: post #51
hnkimb3558 wrote: post #50

Earlier quoted context omitted.

(employee/founder here) Anything under 64K is perfectly reasonable to store in a CockroachDB column. Between 64K and maybe 1M is trending towards trouble. Values greater than this are going to go through CockroachDB like a goat through a python. Why is this the case? For starters, at the level of RocksDB, values greater than 64K are not jammed into SSTables (to avoid constantly rewriting them during compactions of th…

Thanks! That's fantastic to hear even if it's going to materialize later/eventually/never at least it's great to know the need is recognized. Disregarding write amplification issues for a sec, would it make things better to split binaries in 64k chunks and have them in a chunk table keyed by name and offset?

I'm not really sure which strategy would benchmark best between 64K chunks, 1M chunks, or even 8M chunks. I think this requires some experimentation. Pushing them all through as 64K chunks has a lot of overhead, and you'd reap the full write amplification. Could you tell me a bit more about your use case? You can email me spencer at cockroachlabs.com.

I've been meaning to work on the "CockroachDB Egg Store" (my disgusting name for a blob storage subsystem) as a Free Fridays side project for a while, but have been distracted with all manner of other enticing options. There aren't enough hours in a week...

Re: Serializable, Lockless, Distributed: Isolation in CockroachDB

#53

Post by chris_wot »

petermattis wrote: post #44
chris_wot wrote: post #24

Earlier quoted context omitted.

Hi Peter, sounds pretty awesome! Quick question though - on your front page you say that CockroachDB does SQL - but if it can't do a join, then how can you say it uses SQL? Or is distributed SQL a different thing entirely? It does sounds like a very limited SQL subset though... I'm sure I must be missing something as I'm not familiar with your product. Also, what levels of isolation do you actually offer? Serialized…

Hi Chris, we describe CockroachDB as a SQL database because that is what we're aspiring to. The missing functionality (i.e. joins) is on our near-term roadmap.

Sweet :-)

Joins over a distributed database aren't easy. Love to see what you have going! The main issue I see with distributed joins are that they need to be done in a single transaction - if even one table gets an insert, delete or update then it invalidated the join. But this distributed serialised snapshot isolation, that sounds like it might be the best way around it.

Post reply on HN