Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How MySQL is able to scale to 200M QPS – MySQL Cluster

highscalability.com

Post reply on HN
Print view View original

21–25 of 25 posts

Re: How MySQL is able to scale to 200M QPS – MySQL Cluster

#21

Post by MichaelGG »

jhugg wrote: post #18
MichaelGG wrote: post #8

Earlier quoted context omitted.

VoltDB replicates to k nodes so you can choose the level of durability. Combined with UPSes in separate parts of the datacenter, it's durable in some senses. (I'd be more concerned with a kernel or app bug that crashes the system, if there wasn't a "save RAM on unclean boot". This only applies to the commercial version, though. The open source one ditched that feature.

VoltDB, like MySQL Cluster and several other in-memory system, uses snapshot checkpoints and logging to maintain power-loss protection. VoltDB is a bit different because of how it uses determinism + logical logging, but the point is that most of these systems don't have to lose data on power loss.

Sure but those are mostly backup, right? If a system could log to disk at a fast enough pace, then it'd not really be a main memory DB, right? It'd be essentially the same as a normal DB with a really good RAM cache. I've got to imagine there are some VoltDB users going fast enough to make logging not 100% loss proof.

Re: How MySQL is able to scale to 200M QPS – MySQL Cluster

#22

Post by jhugg »

MichaelGG wrote: post #21
jhugg wrote: post #18

Earlier quoted context omitted.

VoltDB, like MySQL Cluster and several other in-memory system, uses snapshot checkpoints and logging to maintain power-loss protection. VoltDB is a bit different because of how it uses determinism + logical logging, but the point is that most of these systems don't have to lose data on power loss.

Sure but those are mostly backup, right? If a system could log to disk at a fast enough pace, then it'd not really be a main memory DB, right? It'd be essentially the same as a normal DB with a really good RAM cache. I've got to imagine there are some VoltDB users going fast enough to make logging not 100% loss proof.

Actually no. The key innovation is that data is stored in a memory friendly way, then logged to disk in an append-only, log friendly way. VoltDB can be fast and persistent; the catch is that your data must fit in memory.

Re: How MySQL is able to scale to 200M QPS – MySQL Cluster

#23

Post by MichaelGG »

jhugg wrote: post #22
MichaelGG wrote: post #21

Earlier quoted context omitted.

Sure but those are mostly backup, right? If a system could log to disk at a fast enough pace, then it'd not really be a main memory DB, right? It'd be essentially the same as a normal DB with a really good RAM cache. I've got to imagine there are some VoltDB users going fast enough to make logging not 100% loss proof.

Actually no. The key innovation is that data is stored in a memory friendly way, then logged to disk in an append-only, log friendly way. VoltDB can be fast and persistent; the catch is that your data must fit in memory.

Oh really? I'm not sure how I missed that all this time. Is it a newer feature? I don't recall it in the H Store paper or in any of the docs I read. In fact I recall sync vs async logging to be a decision/tradeoff. And there used to be a rather large emphasis on choosing s good k factor, and it certainly seemed to be about durability. Otherwise, it's just another normal HA thing, no big deal? As in k 0 is a perfectly fine setup, if logging is enabled?

Re: How MySQL is able to scale to 200M QPS – MySQL Cluster

#24

Post by bballard »

MichaelGG wrote: post #23
jhugg wrote: post #22

Earlier quoted context omitted.

Actually no. The key innovation is that data is stored in a memory friendly way, then logged to disk in an append-only, log friendly way. VoltDB can be fast and persistent; the catch is that your data must fit in memory.

Oh really? I'm not sure how I missed that all this time. Is it a newer feature? I don't recall it in the H Store paper or in any of the docs I read. In fact I recall sync vs async logging to be a decision/tradeoff. And there used to be a rather large emphasis on choosing s good k factor, and it certainly seemed to be about durability. Otherwise, it's just another normal HA thing, no big deal? As in k 0 is a perfectly…

Command logging has worked this way since VoltDB 2.0. Sync mode needs hardware that can support more frequent fsyncs, such as disk controller with BBWC (does the job and inexpensive), or a server-class SSD, and should be configured to write the command log and snapshots to separate spindles. Async mode is the default setting so that out of the box VoltDB will perform as expected in the cloud, on a VM, or on a laptop.

Re: How MySQL is able to scale to 200M QPS – MySQL Cluster

#25

Post by no1youknowz »

cengizIO wrote: post #10

I'm sorry but MySQL Cluster is pure refined salty horse shit. Nothing more. We're currently at the office trying to recover our production data from its filthy hands.

Preach it loud and true brother!

I had MySQL clustering in the lab with some serious live traffic for about a month. Unless you have a team of DBs whose job is just to monitor it and keep it going MacGyver style, seriously forget it.

There are other better options out there, with less headaches!

Post reply on HN