Why Large Hadron Collider Scientists are Using CouchDB

#1

Post by jchrisa » Thu, Aug 26, 2010, 5:16 PM UTC

Why Large Hadron Collider Scientists are Using CouchDB
readwriteweb.com

Re: Why Large Hadron Collider Scientists are Using CouchDB

#2

Post by jcsalterego » Thu, Aug 26, 2010, 5:43 PM UTC

I hope they didn't upgrade to 1.0.0.

Re: Why Large Hadron Collider Scientists are Using CouchDB

#3

Post by couchdb » Thu, Aug 26, 2010, 5:54 PM UTC

jcsalterego wrote: post #2

I hope they didn't upgrade to 1.0.0.

If they cared for data-integrity above performance, they would be running with delayed_commits=false anyway, and thus would not be subject to the bug.

Re: Why Large Hadron Collider Scientists are Using CouchDB

#4

Post by jchrisa » Thu, Aug 26, 2010, 6:03 PM UTC

jcsalterego wrote: post #2

I hope they didn't upgrade to 1.0.0.

For the record I should note that CouchDB's durable storage format ensured that all the data affected by the 1.0.0 bug is recoverable. We think that in the end, because we announced the bug loudly and clearly, and provided a repair tool within days, no one lost any critical data.

Re: Why Large Hadron Collider Scientists are Using CouchDB

#5

Post by rb2k_ » Thu, Aug 26, 2010, 6:08 PM UTC

I have no idea why they don't use something dynamo based like Riak or Cassandra which actually can automatically shard data and scale really well... CouchDB only supports replication, no sharding unless you use third party libraries. With Riak they could use the internal map/reduce support and with Cassandra they might be able to use Hadoop to analyze data and split the workload over several nodes.

Re: Why Large Hadron Collider Scientists are Using CouchDB

#6

Post by nborwankar » Thu, Aug 26, 2010, 6:20 PM UTC

rb2k_ wrote: post #5

I have no idea why they don't use something dynamo based like Riak or Cassandra which actually can automatically shard data and scale really well... CouchDB only supports replication, no sharding unless you use third party libraries. With Riak they could use the internal map/reduce support and with Cassandra they might be able to use Hadoop to analyze data and split the workload over several nodes.

If you read the case study you will see that building apps rapidly was a huge win for them. CouchDB gets compared with other DB's but it is a full stack app dev environment. The other NoSQL DB's need a Rails or PHP or Python or Java or C# or ... stack in front and app dev is nowhere as fast as with CouchApp.

Re: Why Large Hadron Collider Scientists are Using CouchDB

#7

Post by nivertech » Thu, Aug 26, 2010, 7:06 PM UTC

It's most likely just handles metadata, aggregated data or serves as a proxy to their main DB.

I just can't believe they converted 10 PB of binary data into JSON ;)

Re: Why Large Hadron Collider Scientists are Using CouchDB

#8

Post by ergo98 » Thu, Aug 26, 2010, 7:31 PM UTC

rb2k_ wrote: post #5

I have no idea why they don't use something dynamo based like Riak or Cassandra which actually can automatically shard data and scale really well... CouchDB only supports replication, no sharding unless you use third party libraries. With Riak they could use the internal map/reduce support and with Cassandra they might be able to use Hadoop to analyze data and split the workload over several nodes.

They use it to store aggregated reports for distribution and analysis. It isn't doing any heavy lifting, and it's not a terribly compelling case study of anything.

Re: Why Large Hadron Collider Scientists are Using CouchDB

#9

Post by ibejoeb » Thu, Aug 26, 2010, 8:10 PM UTC

Obviously CERN is doing heisenbug research.

Re: Why Large Hadron Collider Scientists are Using CouchDB

#10

Post by chunkbot » Thu, Aug 26, 2010, 8:28 PM UTC

I am happy to see that CERN uses Oracle extensively, presumably for the data that matters. Our tax dollars should only go to a reliable, proven database solution.