61–70 of 91 posts
+1 for "do not use joins"
why
(the stackoverflow response is a much better response than i could have written, hence the link)
If sharding, rebalancing and partitions didn't exist... And if 1600 rps were enough when nosql can reach 1M rps (600 times more)... And if that homemade replication infrastructure was as mature and supported as nosql dbs... Then using MySQL as a nosql might work.
What use case and hardware are your hypothetical relational/non-relational database under where you get your 600 times speed-ups?
I can run a benchmark of a few hundred fast machines with sharded sqlite databases doing key-value and operating in RAM and get large numbers (any number I want), but they don't mean anything.
What about ease of setup? I'd take the NoSql setup any day. If in need of a super light, super quick db that is for a "non-enterprise" application, it's tough to beat NoSql...
MySQL may work well for this small data set (200GB). Start working with 10s of TBs of data and you will start to understand why NoSQL stores were built.
Meanwhile, NoSQL does not mean "10s TBs of data" automatically. Check this slideshow explaining challenges of MongoDB (poster NoSQL database) "scaling to 100GB and beyond"
http://www.slideshare.net/mongodb/partner-webinar-the-scalin...
Earlier quoted context omitted.
why
expense - see http://stackoverflow.com/questions/2065754/is-there-any-gene... which actually shows it fairly well. (the stackoverflow response is a much better response than i could have written, hence the link)
Earlier quoted context omitted.
Does anyone have a real, globally accepted NoSQL definition yet?
Well, it used to be databases that do not do SQL. But now, that Cassandra has SQL like query language, hadoop has SQL like hive and other NoSQLs implement semi-SQL interfaces, I believe a good definition is a database that is not one of 1. ACID 2. Strongly typed table schema.
Having a SQL-esque query language is very far from the implications of "doing SQL":
- Any external software that relies on SQL (not a similar language that looks like but it isn't) won't work.
- Any tool which expects SQL and expects SQL-based drivers (like JDBC, for instance) won't neither work.
- SQL is a huge standard with a significant number of features. This semi-SQL interfaces, at best, implement a tiny part of that used to be SQL 92, a standard by the Windows 3.1 era. So not a big deal if you compare with modern SQL implementations in databases like PostgreSQL.
So I think they are still very bound by the definition that they don't do SQL.
I'm going to question your knowledge of the domain immediately when your solution to a problem is to take away all of the advantages of SQL without any of the benefits that Cassandra or Redis provide. To me this is akin to saying we don't need Haskell because Java now has lambda expressions.
I know nothing about Cassandra. I question however the ability of Redis to index JSON fields and store/retrieve the JSON document in a way that is easy to program and reason about. I'm not an expert though so can you please correct me?
As for the second point, you can store the JSON data in a Redis "hash" datatype: https://matt.sh/introduction-to-redis-data-types
Although MySQL also supports in-memory KV storage with (nearly?) constant time lookups (http://dev.mysql.com/doc/refman/5.7/en/innodb-adaptive-hash....), I think Redis is a bit easier to partition.
I'm going to question your knowledge of the domain immediately when your solution to a problem is to take away all of the advantages of SQL without any of the benefits that Cassandra or Redis provide. To me this is akin to saying we don't need Haskell because Java now has lambda expressions.
What are the benefits of Cassandra or Redis that I am missing, for this use case? We have the latency figures as good as Cassandra can get and we get a reliable engine to store data (something that Redis is not - read Aphir post about Redis) The main advantage of MySQL that we keep is the rock solid platform with all the know-how to operate and manage.
Just comparing a single box vs single box.. this is about 3,333 queries per second if I understand the metric used (200k req per minute). A Redis instance (non-sharded) can handle 50 times that in a single thread, while maintaining sub-ms latency.
Just comparing the two: for this volume of data, and given the low queries per second, I completely agree that MySQL is a better choice since Redis wouldn't scale for cost (of RAM -- it would be silly to pay for that RAM if SSD or spinning disk can handle the load.)
Redis is really nice when used in conjunction with other servers. At Userify (SSH key management for cloud instances), we used to function with a purely MySQL environment, but MySQL couldn't keep up with our requirements (tens of thousands of qps) on low-end hardware for mostly small bits of data. We converted the whole thing to Redis + S3 and are scaling very smoothly, even though we actually encrypt and gzip data before writing to Redis (and we actually write-through to S3, which is mostly invisible except for sequentially pipelined operations). There are circumstances where we could have the lost-write problem, but they are rare in practice and would basically be the same effect as rolling-back a commit.
If I was going to scale the WiX model higher, I'd probably keep MySQL for the blob data (or use S3) and use Redis for lookups, or pursue other paths to keep MySQL in place (or perhaps pgsql). But don't fix it if it ain't broke. ;)
And, of course, there is still a very long way you can go to take MySQL (or Postgres) to insane heights (just ask Facebook), including NDB/MySQL Cluster, innovative caching solutions with Redis front-ending MySQL (the way you used to do with Memcached), maxscale, or additional sharding.
In other words, your design works and works well, and shows the power of MySQL (or especially Postgresql) as a general purpose data store. I really agree that you should always start on the datastore that you think you can get up and running with quickly and easily, and focus on optimization later, because optimization is always possible later within any complex system.
although Redis is just a dream to work with -- unlike some other nosql solutions..