The key value store we ignored (Postgresql) • Hacker News Board

Re: The key value store we ignored (Postgresql)

#21

Post by maxpert » Sun, Dec 11, 2011, 6:32 PM UTC

grncdr wrote: post #16

maxpert wrote: post #8

Earlier quoted context omitted.

I think for schema-less storage systems we know 2 major competitors in market. MongoDB and CouchDB. - CouchDB by default has not ability to scale out except master master replication, solution? Sharding for distribution and Replication for reliability. Or you BigCouch with prayers that it won't trash out your data. - MongoDB is know to stand on its Sharding server mongos and you have to issue sharding commands whenev…

> I think for schema-less storage systems we know 2 major competitors in market. MongoDB and CouchDB. Um, what about Riak, Cassandra, Voldemort, and Hbase? (I'm sure there's a bunch more I'm forgetting)

Cassandra, Hbase -> Column Oriented! Not schema less! Riak -> Actually a key/value store with link walking, you can just write map reduce for that document oriented feel! Again I won't bother writing a map-reduce job just to fetch out document with particular values. Voldermort -> Distributed key value.

Again you are missing the point of maturity and a proven user base, and it's comparing apple with bananas! Try putting in same joins and relations in your NoSQL stores that you are bragging about and see how quickly they will lose scaling! Want an example? Neo4J!

Re: The key value store we ignored (Postgresql)

#22

Post by moe » Sun, Dec 11, 2011, 6:33 PM UTC

henrikschroder wrote: post #6

The main selling point of the various NoSQL products out there today isn't the schemaless storage, instead it's the ability to grow beyond a single server that's compelling. 228MB of data is nothing, it fits in RAM of any machine. What would the examples in this blog post look like if it was 228GB of data spread across 10 servers instead? How would you grow/shrink such a cluster? How would you perform a query that cu…

various NoSQL products it's the ability to grow beyond a single server that's compelling.

You make it sound as if these products scale "magically". This is definitely not the case. There's quite a bit of massaging needed to make riak, hbase et al scale beyond certain thresholds - and you better pay close attention to their respective peculiarities.

How would you perform a query that cuts across servers and aggregates data from some of them?

That's actually not very hard to implement once you understand how it needs to be done. And you do need this understanding with any k/v-store, otherwise you'll be very sad when the magic stops working (as documented time after time for MongoDB).

Starting out with a known-good, mature store such as PostgreSQL can make a lot of sense when the trade-offs match your usage pattern.

Re: The key value store we ignored (Postgresql)

#23

Post by maxpert » Sun, Dec 11, 2011, 6:36 PM UTC

imbriaco wrote: post #20

I agree with the author that hstore is very interesting, but the data structures are not the key selling point in the NoSQL space in my opinion. The most overlooked advantage to things like Cassandra and Riak are the fact that you have no single point of failure in the system. If an individual node fails, there is no operational impact. Postgres does have (finally!) a nice replication story, so you have data protecti…

I totally agree with just what your said, this post actually is meant for know your choices. I actually hate it when people make a lame excuse of using a NoSQL store when they don't need it and they don't know the consequences!

Re: The key value store we ignored (Postgresql)

#24

Post by maxpert » Sun, Dec 11, 2011, 6:37 PM UTC

jedberg wrote: post #19

reddit used (and still for the most part uses) postgres as a schemaless storage (although they don't use hstore). Also, I'm pretty sure Heroku uses hstore, because they're the ones that taught me about it. So not everyone forgot about it.

Just read the comments around and you will see the dogma people face on daily basis despite the huge databases of reddit!

Re: The key value store we ignored (Postgresql)

#25

Post by keeran » Sun, Dec 11, 2011, 6:58 PM UTC

Thoughtbot posted a similar Postgres article recently:

http://robots.thoughtbot.com/post/13829210385/the-durable-do...

Re: The key value store we ignored (Postgresql)

#26

Post by matwood » Sun, Dec 11, 2011, 7:00 PM UTC

henrikschroder wrote: post #6

The main selling point of the various NoSQL products out there today isn't the schemaless storage, instead it's the ability to grow beyond a single server that's compelling. 228MB of data is nothing, it fits in RAM of any machine. What would the examples in this blog post look like if it was 228GB of data spread across 10 servers instead? How would you grow/shrink such a cluster? How would you perform a query that cu…

228MB of data is nothing, it fits in RAM of any machine. What would the examples in this blog post look like if it was 228GB

You bring up an interesting point. At what data size do the companies using the various NoSQL DBs feel they have to move beyond a traditional RDBMS? I work with some traditional RDBMS stores now that are >500GB in size with tables that add 20M-30M rows/month and querying still isn't an issue. Admittedly it takes expertise to optimize the system and make it work efficiently, but that's going to be the case with any datastore.

Re: The key value store we ignored (Postgresql)

#27

Post by iFire » Sun, Dec 11, 2011, 7:14 PM UTC

henrikschroder wrote: post #6

The main selling point of the various NoSQL products out there today isn't the schemaless storage, instead it's the ability to grow beyond a single server that's compelling. 228MB of data is nothing, it fits in RAM of any machine. What would the examples in this blog post look like if it was 228GB of data spread across 10 servers instead? How would you grow/shrink such a cluster? How would you perform a query that cu…

You can buy a 2u server with 256GB of ram. Think about this.

Re: The key value store we ignored (Postgresql)

#28

Post by mulander » Sun, Dec 11, 2011, 7:23 PM UTC

matwood wrote: post #26

henrikschroder wrote: post #6

The main selling point of the various NoSQL products out there today isn't the schemaless storage, instead it's the ability to grow beyond a single server that's compelling. 228MB of data is nothing, it fits in RAM of any machine. What would the examples in this blog post look like if it was 228GB of data spread across 10 servers instead? How would you grow/shrink such a cluster? How would you perform a query that cu…

228MB of data is nothing, it fits in RAM of any machine. What would the examples in this blog post look like if it was 228GB You bring up an interesting point. At what data size do the companies using the various NoSQL DBs feel they have to move beyond a traditional RDBMS? I work with some traditional RDBMS stores now that are >500GB in size with tables that add 20M-30M rows/month and querying still isn't an issue. A…

I can sustain this claim. I work with a system implemented almost entirely in Oracle PL/SQL. Some tables in the system are nearing 800-900 columns, their size often exceeds 600 GB per table (not many of such large tables though). Querying isn't a problem at all. Large schema changes are also mostly painless. The only point at which one has to be really careful is when a schema change requires actual calculations based on historic data with an additional write on each record.

Re: The key value store we ignored (Postgresql)

#29

Post by iFire » Sun, Dec 11, 2011, 7:24 PM UTC

tptacek wrote: post #14

7952 wrote: post #12

I tried hstore on a project for many of the reasons mentioned. I just hated having to always represent data as strings.

Isn't this effectively what Redis does too? There are whole Salvatore posts about doing binary data structures in Redis string values.

http://redis.io/topics/data-types

Strings are the most basic kind of Redis value. Redis Strings are binary safe, this means that a Redis string can contain any kind of data, for instance a JPEG image or a serialized Ruby object.

Re: The key value store we ignored (Postgresql)

#30

Post by sqrt17 » Sun, Dec 11, 2011, 7:48 PM UTC

henrikschroder wrote: post #6

The main selling point of the various NoSQL products out there today isn't the schemaless storage, instead it's the ability to grow beyond a single server that's compelling. 228MB of data is nothing, it fits in RAM of any machine. What would the examples in this blog post look like if it was 228GB of data spread across 10 servers instead? How would you grow/shrink such a cluster? How would you perform a query that cu…

The trick is, you wouldn't necessarily. If your workload can be handled with one beefed-up lots-of-RAM-and-solid-state-drives server, you could spend your money on two of these instead of having 10 smaller servers, and be perfectly happy with it.

I don't care about what sells NoSQL products to enterprise users - there are lots of workloads where the data fits in RAM (for some reasonable quantity of RAM that money can buy) and you still prefer the durability and consistency that comes with standard SQL databases, even if you denormalize and/or use schemaless (XML or JSON) storage.