101–110 of 150 posts
Riak looks interesting but its overkill. They recommend at least three nodes. We went back to PostgreSQL.
"No schemas. This was wonderful. What are schemas even for? They just make things hard to change for no reason." While they do mention the need to enforce constraints on your data, it's comments like these that make me wish all application developers were required to work as a DBA for a few months. A properly normalized and "constrained" database prevents data loss from stupid mistakes.
"no schemas" means "schema in the application layer". sometimes its nice for the additional flexibility, but its never as reliable.
if (data.schema_version === 1) { ... } else if (data.schema_version === 2) { ... }
UGH!
Earlier quoted context omitted.
This is InnoDB's sweet spot really -- a mostly read-only in memory data set where the lookups are done primarily by PK. MySQL 5.5 can scale this kind of workload to 32 cores. I'm pretty sure given this kind of workload MySQL will outperform PostgreSQL handily.
And PostgreSQL 9.2 will be able to scale this workload linearly to 64 cores. So while MySQL may or may not win it will certainly not "outperform PostgreSQL handily". http://rhaas.blogspot.se/2012/04/did-i-say-32-cores-how-abou...
The fact is current versions of PG are unable to use more than 60% CPU on a 24 core machine. Do you know anyone who uses a dev version of an RDBMS in production?
http://archives.postgresql.org/message-id/BANLkTimVboKxzGS9B...
Earlier quoted context omitted.
Postgresql will scale to 32 cores with a real workload, and has done so for a few years. Mysql performance still tanks at 8 cores. Both of those statements are not accurate, but hey, what's it matter? Without benchmarks we're both talking out our ass anyway.
The postgres part of the statement is accurate; http://rhaas.blogspot.de/2012/04/did-i-say-32-cores-how-abou...
The lock manager bottlenecks that stopped PG from using more than 60% of the cpu power on a 24 core box were discovered a little less than a year ago.
http://rhaas.blogspot.com/2011/07/read-scaling-out-to-32-cor...
Earlier quoted context omitted.
Seriously, after that line I gave up on reading the rest. There's plenty to be said about schema vs no schema but its pretty ignorant to just dismiss the entire concept out of hand. Guess what - in lots of applications data integrity is more important than developer convenience.
I made it to the next one before I gave up: "Relational databases grew up solving problems where data integrity was paramount and availability was not a big concern."
Earlier quoted context omitted.
I don't like MySQL either, but they're basically using it as a networked hash table. It's not so bad that it can't do that .
This is InnoDB's sweet spot really -- a mostly read-only in memory data set where the lookups are done primarily by PK. MySQL 5.5 can scale this kind of workload to 32 cores. I'm pretty sure given this kind of workload MySQL will outperform PostgreSQL handily.
"No schemas. This was wonderful. What are schemas even for? They just make things hard to change for no reason." While they do mention the need to enforce constraints on your data, it's comments like these that make me wish all application developers were required to work as a DBA for a few months. A properly normalized and "constrained" database prevents data loss from stupid mistakes.
A properly written application layer also prevents data loss from stupid mistakes. A stupid mistake made while setting up a properly normalized database also causes data loss.
You have to be very smart to be able to design a normalized constrained DB well. The fact that only smart people can do it doesn't mean that people who don't do it aren't smart.
"No schemas. This was wonderful. What are schemas even for? They just make things hard to change for no reason." While they do mention the need to enforce constraints on your data, it's comments like these that make me wish all application developers were required to work as a DBA for a few months. A properly normalized and "constrained" database prevents data loss from stupid mistakes.
Seriously, after that line I gave up on reading the rest. There's plenty to be said about schema vs no schema but its pretty ignorant to just dismiss the entire concept out of hand. Guess what - in lots of applications data integrity is more important than developer convenience.
They're doing what works for them and good for them for that. But...I think a LOT of people are really missing out by passing over Riak. Many of the issues they found with CouchDB have been resolved with Riak. I think the sync API for CouchDB is really cool, but Riak has the auto-sharding thing down cold. Riak runs map reduce queries across multiple nodes, so performance and capability can grow as you add nodes. Couc…
I investigated using Riak for dealing with our metrics a few months ago, but with the data sizes we are dealing with, even the Riak people told us that Hadoop was likely a better solution. Once you are dealing with more than 500k keys or so, Riak starts to fall over. EDIT: The 500k key limit pertains to mapreduce jobs, not the overall data size.
Riak will handle billions of keys just fine. We had, I dunno, a half a billion in a six node bitcask-backed cluster and were only at half capacity. Much much bigger installs exist. The limit I was referring to is for a single mapreduce job; Riak MR just isn't well-suited to operations over millions of keys at a time. It can do it, but Riak MR isn't really designed for bulk processing: and I wouldn't be surprised to see MR become unusably slow over millions of keys. You'll get better performance out of Hadoop, generally, for bulk analytics.
The other tough point is key-listing. Listing buckets, listing keys, key filters, MR over buckets, all those features are essentially useless in production. Where the number of keys is large and unguessable it can become a logistical nightmare to keep track of them. 2I key indexes can help, though.
Earlier quoted context omitted.
Yes. I should clarify that I meant 500k keys used in a single m/r job. We needed to be able to run m/r over roughly 200 million keys at the time.
And it turns out you are misrepresenting the situation completely. You can run M/R over key sets in the billions of keys. It sounds like you've not organized your data at all. You're bashing a product here based on your lack of knowledge, not the products lack of capabilities.