How MySQL is able to scale to 200M QPS – MySQL Cluster
highscalability.com
How MySQL is able to scale to 200M QPS – MySQL Cluster
1–10 of 25 posts
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#2> MySQL Cluster is a scalable, real-time in-memory, ACID-compliant transactional database
To what degree can something be called durable if it's in-memory?
> It is possible to choose how to store data; either all in memory or with some on disk (non-indexed data only). […] Disk-based data can be used to store data with less strict performance requirements, where the data set is larger than the available RAM.
It's great to have all the concepts spelled out in this article, particularly the way nodes act as transaction managers when needed, synchronously replicate to a "buddy", and have management capabilities for recovering from partition events; however, it's almost misleading to use the MySQL name: this appears to be first and foremost an in-memory solution with some afterthought given to disk-based durability.
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#3A modern and properly implemented database engine should be bottlenecked on network saturation for these types of workloads even if backed by disk storage, never mind in-memory. These workloads are trivially distributable, so scale-out is expected unless you are doing something really wrong.
In that sense, similar "look how fast my simple lookups are" benchmarks published by other database vendors are really a function of their hardware budget for the benchmark assuming the implementation is competent. Queries with complex constraints or joins would be a more interesting indication of implementation scalability and performance.
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#4> distributed transactions based on 2-phase commit.
i just stop reading from there.
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#5> MySQL Cluster is a scalable, real-time in-memory, ACID-compliant transactional database To what degree can something be called durable if it's in-memory? > It is possible to choose how to store data; either all in memory or with some on disk (non-indexed data only). […] Disk-based data can be used to store data with less strict performance requirements, where the data set is larger than the available RAM. It's grea…
I don't know how the MySQL one works, and if it does provide disk-based durability at all, but the way Microsoft SQL server does it is explained Section 7 (at page 7) of [1]
They use a log stream and checkpoints stream that are writing to disk.
[1] http://research.microsoft.com/pubs/193594/Hekaton%20-%20Sigm...
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#6> distributed transactions based on 2-phase commit. i just stop reading from there.
Can you elaborate on this?
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#7> distributed transactions based on 2-phase commit. i just stop reading from there.
That's the way Megastore does it too. Optional 2PC for transactions that span entity groups. So what's wrong with 2PC?
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#8> MySQL Cluster is a scalable, real-time in-memory, ACID-compliant transactional database To what degree can something be called durable if it's in-memory? > It is possible to choose how to store data; either all in memory or with some on disk (non-indexed data only). […] Disk-based data can be used to store data with less strict performance requirements, where the data set is larger than the available RAM. It's grea…
VoltDB replicates to k nodes so you can choose the level of durability. Combined with UPSes in separate parts of the datacenter, it's durable in some senses. (I'd be more concerned with a kernel or app bug that crashes the system, if there wasn't a "save RAM on unclean boot".
This only applies to the commercial version, though. The open source one ditched that feature.
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#9I couldn't care less.
I found a bug and 5 months later still not resolved.
https://bugs.mysql.com/bug.php?id=75293
Maybe start fixing things people may want to use, instead of claiming "yeah we can do that".
I no longer use MySQL, I have upgraded to something more "enterprise" and more useful.
Re: How MySQL is able to scale to 200M QPS – MySQL Cluster
#10I'm sorry but MySQL Cluster is pure refined salty horse shit. Nothing more. We're currently at the office trying to recover our production data from its filthy hands.