Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On moving from CouchDB to Riak

labs.linkfluence.net

Post reply on HN
Print view View original

31–40 of 43 posts

Re: On moving from CouchDB to Riak

#31

Post by aphyr »

davidhollander wrote: post #25
aphyr wrote: post #20

Earlier quoted context omitted.

To my knowledge, there is no such thing as an in-memory bitcask backend for Riak. Could you have been using regular Bitcask (keeps values on disk in log-structured files) or the ETS memory backend?

Bitcask is an in memory datastore because it keeps a copy of the entire dataset in memory. 4GB of data==4GB of memory>trivial amount of memory used by a file descriptor. > keeps values on disk in log-structured files Of course it does. Log files are the easiest way to make an in memory datastructure persistant. Logs are generally used for periodically dropping fast writes as a backup in case you want to reload the me…

Bitcask is an in memory datastore because it keeps a copy of the entire dataset in memory.

Begging your pardon, but I think you may be misunderstanding bitcask. The bitcask keydir is stored in memory, but the values are stored on disk. The keydir is a hash mapping each key to a file ID and the offset/size in that file at which the value is stored. The only time values are stored in memory on is when the kernel's fs cache or readahead buffer provide them.

http://downloads.basho.com/papers/bitcask-intro.pdf

Since a filesystem directory listing is likely held by your OS cache, you should see similar performance between bitcask and files on disk: an in-memory lookup to obtain the inode/offset, and a disk seek+read.

Riak will be substantially slower than directly using bitcask, however, because you may need to talk over the network to as many as N nodes, wait for all their responses, and compute the resultant vclock/metadata, and then serialize it for HTTP (huge TCP overhead) or protocol buffers (relatively fast). If you're running on a single machine, then you may incur additional time for that machine to run what would normally be distributed over three nodes. Without knowing more about your benchmark, however, it's difficult to say.

Re: On moving from CouchDB to Riak

#32

Post by grncdr »

JoachimSchipper wrote: post #23
defroost wrote: post #21

Earlier quoted context omitted.

> You could comfortably stuff that in any decent database (SQL Server for example, I'm sure PostgreSQL would work too). You may have noticed that the OP's requirements were: * a REST interface * sharding The costs of SQL Server are unbelievably high, and which you can do horizontal partitioning and sharding, Riak is designed to be used in this manner. On the other hand, re: SQL Server according to this article the "h…

You don't necessarily need to shard if the data fits comfortably on a standard disk... and the parent did mention PostgreSQL.

Sharding (for free) on PostgreSQL is surprisingly painful

Re: On moving from CouchDB to Riak

#33

Post by pjscott »

davidw wrote: post #10

Hrm... with N "nosql" things, there's great potential for writing. Redis vs Riak. Cassandra to CouchDB. Why I gave up on Mongo and moved to Mnesia. And so on and so forth;-)

Your comment applies to any job for which there are multiple competing tools.

Re: On moving from CouchDB to Riak

#34

Post by davidhollander »

aphyr wrote: post #31
davidhollander wrote: post #25

Earlier quoted context omitted.

Bitcask is an in memory datastore because it keeps a copy of the entire dataset in memory. 4GB of data==4GB of memory>trivial amount of memory used by a file descriptor. > keeps values on disk in log-structured files Of course it does. Log files are the easiest way to make an in memory datastructure persistant. Logs are generally used for periodically dropping fast writes as a backup in case you want to reload the me…

Bitcask is an in memory datastore because it keeps a copy of the entire dataset in memory. Begging your pardon, but I think you may be misunderstanding bitcask. The bitcask keydir is stored in memory, but the values are stored on disk. The keydir is a hash mapping each key to a file ID and the offset/size in that file at which the value is stored. The only time values are stored in memory on is when the kernel's fs c…

>you may be misunderstanding bitcask

Yes I was, I apologize. It must have been MongoDB that required total data be no larger than amount of RAM.

The test was run using protocol buffers and Python client on the same computer testing reads vs. a naive map reduce. The naive map reduce was storing 4,000 files in a folder, treating the filename as the key and parsing the text file contents from JSON into a python dictionary to see if an attribute matches. Basically I figure doing a loop of thousands of blocking disk accesses from a laptop harddrive on a standard filesystem buffering nothing in memory should always be much slower than any database.

This was last year so maybe Riak's performance has increased since then. I'd be interested if TokyoCabinet was added as a backend.

Re: On moving from CouchDB to Riak

#35

Post by cdavid »

rnewson wrote: post #22
biot wrote: post #18

Earlier quoted context omitted.

My reading was that they found CouchDB to have problems and adding complexity on top of a problematic base wasn't an appealing proposition for them.

My counterpoint is that Cloudant test and harden the CouchDB that we embed, it's not, for various reasons, a verbatim copy of CouchDB itself. That said, vanilla CouchDB is quite stable. While there are always bugs, I don't recognize the system that the original poster is describing.

What kind of data do you record (number of documents, average size of document) ? We have to use couchdb at my current job, and I have encoutered all those issues for even moderate size (replication failing more often than @ ~30-40 Gb). I wonder if that's an environment issue.

Re: On moving from CouchDB to Riak

#36

Post by aphyr »

davidhollander wrote: post #34
aphyr wrote: post #31

Earlier quoted context omitted.

Bitcask is an in memory datastore because it keeps a copy of the entire dataset in memory. Begging your pardon, but I think you may be misunderstanding bitcask. The bitcask keydir is stored in memory, but the values are stored on disk. The keydir is a hash mapping each key to a file ID and the offset/size in that file at which the value is stored. The only time values are stored in memory on is when the kernel's fs c…

> you may be misunderstanding bitcask Yes I was, I apologize. It must have been MongoDB that required total data be no larger than amount of RAM. The test was run using protocol buffers and Python client on the same computer testing reads vs. a naive map reduce. The naive map reduce was storing 4,000 files in a folder, treating the filename as the key and parsing the text file contents from JSON into a python diction…

Riak, by default, uses a replication value of three. Your single test machine has to do ~three times the work, so you should expect slower performance here. (I'm oversimplifying somewhat.)

You'll see significantly improved performance on a linear test (in my informal testing, 3-4x speedups) by adding an extra two nodes. Parallelized tests pretty much scale linearly with nodes.

In practice, I've found Riak to be slightly slower than MySQL. Direct reads/writes tend to be fast, but JSON parsing can bite you and denormalization requires more writes. The major advantage is that the Riak system can scale linearly with nodes, and that it can fail in predictable and resolvable ways.

As an example, the feed system I'm currently building on Riak will survive a total network partition and allow full reads and writes from every node with no data lost. Everything is automatically merged when the partition ends. The vclock-tagged multi-value functionality of Riak is exceptionally powerful when you want to design these types of systems, and is, in my mind, worth the performance hit and additional design complexity for certain classes of problems.

This was last year so maybe Riak's performance has increased since then. I'd be interested if TokyoCabinet was added as a backend.

There are also InnoDB and multiple in-memory backends, which may provide performance characteristics more in line with what you are looking for.

Re: On moving from CouchDB to Riak

#37

Post by [deleted] »

pkulak wrote: post #12
ot wrote: post #8

Earlier quoted context omitted.

> (and each key has, on the order of, 20 bits of overhead, IIRC) 20 bits ? Really? Less than an integer? Or did you mean bytes? (Not nitpicking here, I'm just curious)

20 bits can support a million records. As an average, per node or cluster, it seems reasonable to me.

[deleted]

Re: On moving from CouchDB to Riak

#38

Post by JoachimSchipper »

grncdr wrote: post #32
JoachimSchipper wrote: post #23

Earlier quoted context omitted.

You don't necessarily need to shard if the data fits comfortably on a standard disk... and the parent did mention PostgreSQL.

Sharding (for free) on PostgreSQL is surprisingly painful

I thought people just built sharding into the application level when using MySQL stuff - I'd imagine the same thing works just fine for PostgreSQL. Or were you talking about something else?

Re: On moving from CouchDB to Riak

#39

Post by prodigal_erik »

JoachimSchipper wrote: post #38
grncdr wrote: post #32

Earlier quoted context omitted.

Sharding (for free) on PostgreSQL is surprisingly painful

I thought people just built sharding into the application level when using MySQL stuff - I'd imagine the same thing works just fine for PostgreSQL. Or were you talking about something else?

Anyone who volunteers for app level sharding is going to find themselves deep in Special Hell. Step one is to rewrite all your queries so they no longer expect really exotic use cases like "SELECT ... WHERE" to actually work until you somehow figure out which nodes to run them on (and if it was supposed to be a join between entities crossing shards, good luck with that). Step two is to find working XA-aware drivers (haha). For step three you need a knife and a goat....

Apparently really high-end clustered databases solve this problem correctly, i.e., the schema is extended to specify where any record can be found, the cluster uses that to work out a minimally-stupid query plan, and record->shard mapping becomes merely a tuning decision. I've never had the opportunity to work with one. But I'm not bitter.

Re: On moving from CouchDB to Riak

#40

Post by [deleted] »

cdavid wrote: post #35
rnewson wrote: post #22

Earlier quoted context omitted.

My counterpoint is that Cloudant test and harden the CouchDB that we embed, it's not, for various reasons, a verbatim copy of CouchDB itself. That said, vanilla CouchDB is quite stable. While there are always bugs, I don't recognize the system that the original poster is describing.

What kind of data do you record (number of documents, average size of document) ? We have to use couchdb at my current job, and I have encoutered all those issues for even moderate size (replication failing more often than @ ~30-40 Gb). I wonder if that's an environment issue.

[deleted]
Post reply on HN