On moving from CouchDB to Riak • Hacker News Board

Re: On moving from CouchDB to Riak

#41

Post by JoachimSchipper » Tue, Mar 08, 2011, 1:41 PM UTC

prodigal_erik wrote: post #39

JoachimSchipper wrote: post #38

Earlier quoted context omitted.

I thought people just built sharding into the application level when using MySQL stuff - I'd imagine the same thing works just fine for PostgreSQL. Or were you talking about something else?

Anyone who volunteers for app level sharding is going to find themselves deep in Special Hell. Step one is to rewrite all your queries so they no longer expect really exotic use cases like "SELECT ... WHERE" to actually work until you somehow figure out which nodes to run them on (and if it was supposed to be a join between entities crossing shards, good luck with that). Step two is to find working XA-aware drivers (…

Actually, if you use MySQL as a "great hashtable in the sky (cloud)", it's not all that much more painful to shard things. That said, if you do that you're already in a state of sin/pain...

Re: On moving from CouchDB to Riak

#42

Post by philwise » Tue, Mar 08, 2011, 4:59 PM UTC

defroost wrote: post #21

philwise wrote: post #15

"We store a lot of data... 2TB" Given that a pair of 2TB drives is less than $250 on ebuyer right now, 2TB of data is not 'big data'. You could comfortably stuff that in any decent database (SQL Server for example, I'm sure PostgreSQL would work too). Just because a tiny machine on slicehost isn't big enough doesn't mean that your data won't fit in a normal database.

> You could comfortably stuff that in any decent database (SQL Server for example, I'm sure PostgreSQL would work too). You may have noticed that the OP's requirements were: * a REST interface * sharding The costs of SQL Server are unbelievably high, and which you can do horizontal partitioning and sharding, Riak is designed to be used in this manner. On the other hand, re: SQL Server according to this article the "h…

'Sharding' and 'a rest interface' are not really business requirements. Sharding is only a solution to handling certain types of high load, and REST is but one solution for IPC.

Re: On moving from CouchDB to Riak

#43

Post by tillk » Tue, Mar 08, 2011, 11:29 PM UTC

I'm a little let down by the detail of this blog post.

For anyone who cares: we've had 6x as many documents (as linkfluence) in a single CouchDB instane before we moved to Cloudant. We now have over 360 million documents on Cloudant.

Our database is very write intense, lots of new documents, little updates to the actual documents and a lot of reads of the documents in between. We also have a couple larger views to aggregate the data for our users.

The ride with CouchDB has been bumpy at times, but not at the 'scale' where linkfluence is at.