Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MySQL is bazillion times faster than MemSQL

dom.as

Post reply on HN
Print view View original

51–60 of 148 posts

Re: MySQL is bazillion times faster than MemSQL

#51

Post by mariusmg »

calinet6 wrote: post #44
timaelliott wrote: post #40

Earlier quoted context omitted.

You seem to be missing the main reason why people have an issue with MemSQL. The issue are quotes like this on your website and video: "MEMSQL IS 30 TIMES FASTER THAN MYSQL." It's an extremely invalid and biased comparison. You're quite literally comparing the speed of writing to RAM versus the speed of writing to disk. If you didn't make such ridiculous assertions, people would accept your product for the actual awe…

This is exactly right, and should be addressed. memcached is probably 30 times faster than MySQL too, but they don't go around claiming it's a durable database.

It fucking says "Durable by Default" on their main page !!!!

edit: memSQL main page says "Durable by Default"

Re: MySQL is bazillion times faster than MemSQL

#52

Post by nikita »

HarrisonFisk wrote: post #41
nikita wrote: post #33

MemSQL CTO here. Great article- Domas has done a good job of digging into the internals of MemSQL! A few questions/comments: 1.) The range query issue you pointed out can be explained by a well known limitation of skip lists. Unlike B-Trees, skip lists are unidirectional. By default, our indexes are ascending, so indeed you have to skip to the end to run a MAX() or "ORDER BY id DESC" query. To fix this, just change t…

Regarding #2, we do normally run MySQL with full transactional durability at Facebook. We have for a very long time (several years at least). For example, here is a FB note regarding our enhancing performance under full durability in MySQL from Oct 2010: https://www.facebook.com/note.php?note_id=438641125932

Harrison, what are these settings for InnoDB? innodb_flush_log_at_trx_commit, innodb_flush_methodm, innodb_doublewrite,

Re: MySQL is bazillion times faster than MemSQL

#53

Post by cluda01 »

16s wrote: post #43
gouranga wrote: post #3

Earlier quoted context omitted.

Or in the case of a lot of vendors we won't mention - blatant lies.

Vendors lie. That is the sole truth of IT purchasing. When testing network gear, some of our network engineers would say, "Vendors lie, but packets don't".

Nice quote! More generally when considering any solution you should always test it in your environment according to your performance needs BEFORE you buy it. Any vendor worth their salt will let you do this.

Re: MySQL is bazillion times faster than MemSQL

#54

Post by bsg75 »

Domenic_S wrote: post #42
bsg75 wrote: post #36

Earlier quoted context omitted.

> People who hate MySQL, usually hate it because of SQL. Or because of the shortcuts it has taken to avoid the "hard stuff", which results in developers taking shortcuts by using MySQL to avoid the "hard stuff". Love or hate SQL for what it is, not the MySQL implementation of it.

> Or because of the shortcuts it has taken to avoid the "hard stuff", which results in developers taking shortcuts by using MySQL to avoid the "hard stuff". I am deeply interested in what you think this is

Avoiding RI (in part due to poor FK support depending on version / storage engine), designs that avoid simple schema changes because they can't be done online, point version migration difficulties.

Consider unsupported SQL constructs from other engines (CTEs, window functions, better join and subquery support), and the effort and mess to accomplish what should be straightforward tasks generates SQL hate.

Some NOSQL projects have merit as they provide mechanisms for working with data structures that do not map well to rows and columns. However, difficulties faced with MySQL have also pushed some projects to use equivalently broken data storage technology, when a "better" RDBMS would be more appropriate.

Re: MySQL is bazillion times faster than MemSQL

#56

Post by ankrgyl »

nikita wrote: post #46
timaelliott wrote: post #40

Earlier quoted context omitted.

You seem to be missing the main reason why people have an issue with MemSQL. The issue are quotes like this on your website and video: "MEMSQL IS 30 TIMES FASTER THAN MYSQL." It's an extremely invalid and biased comparison. You're quite literally comparing the speed of writing to RAM versus the speed of writing to disk. If you didn't make such ridiculous assertions, people would accept your product for the actual awe…

Hi Tim, we're not comparing "the speed of writing to RAM versus the speed of writing to disk." You can run InnoDB with a buffer pool large enough to keep the entire database in memory and we'll still outperform it significantly. We're actually working on a blog post right now with that comparison.

@calinet126

We have the durability interface and configuration options here http://developers.memsql.com/docs/1b/durability.html

This should give you a good idea about how durability will react to tuning, but it doesn't dive deep into the internal design. If there's enough interest (seems like there is) we'd be happy to discuss it in a blog post.

Re: MySQL is bazillion times faster than MemSQL

#57

Post by dhruvbird »

nikita wrote: post #33

MemSQL CTO here. Great article- Domas has done a good job of digging into the internals of MemSQL! A few questions/comments: 1.) The range query issue you pointed out can be explained by a well known limitation of skip lists. Unlike B-Trees, skip lists are unidirectional. By default, our indexes are ascending, so indeed you have to skip to the end to run a MAX() or "ORDER BY id DESC" query. To fix this, just change t…

Even though skip-lists as described in literature are singly linked (horizontally), they can be augmented with a back pointer to make them doubly linked. This would allow one to re-use an index as either a forward or backward index, and the order of the index is no longer important. I am not sure how easy this is to do in a lock free fashion. However, if one were to adopt a strategy which combines functional data structures and RCU, I am guessing that it would be possible to implement it with relatively good readability.

Additionally, skip-lists can be used for performing range queries as well. This means that "SELECT * FROM table ORDER BY id DESC LIMIT 5;" can be executed very efficiently as long as there is a skip-list on "id". Additionally, even COUNT(), MAX(), MIN(), etc... queries can be optimized in a similar fashion. Again, however, doing it in a lock-free manner might not be the most fun thing to do.

Re: MySQL is bazillion times faster than MemSQL

#58

Post by jeffdavis »

Domenic_S wrote: post #42
bsg75 wrote: post #36

Earlier quoted context omitted.

> People who hate MySQL, usually hate it because of SQL. Or because of the shortcuts it has taken to avoid the "hard stuff", which results in developers taking shortcuts by using MySQL to avoid the "hard stuff". Love or hate SQL for what it is, not the MySQL implementation of it.

> Or because of the shortcuts it has taken to avoid the "hard stuff", which results in developers taking shortcuts by using MySQL to avoid the "hard stuff". I am deeply interested in what you think this is

One thing he may be referring to is a cost-based optimizer (CBO). Dismissing the significance of a CBO is a mistake.

For one thing, it's not an optimizer like gcc at all. It doesn't take your existing algorithm and speed it up, it chooses a better algorithm. How does it know which is better? It collects statistics and runs them through a cost model.

This is what allows something like PostgreSQL to offer nested loop joins, merge joins, and hash joins. Without a CBO, how would it know which to choose? Similarly, it can offer multiple ways to use the same index (normal index scan, bitmap index scan) and multiple ways to aggregate (Sort+Group or HashAggregate).

Those are just a few of the many algorithms postgres offers that are chosen in a principled way by the CBO.

Re: MySQL is bazillion times faster than MemSQL

#59

Post by spudlyo »

jstclair wrote: post #17
endersshadow wrote: post #7

Earlier quoted context omitted.

For what it's worth SQL Server doesn't meet his definition of "durable." It writes a transaction log and then writes those transactions to disk on a checkpoint [1]. Typically, that checkpoint automagically happens, but you can also force it to clear the buffer with a "checkpoint" command. InnoDB also does this, too [2]. I don't know if MemSQL actually has checkpointing or what, but it's just worth noting. [1]: http:/…

This is a little misleading; an internal checkpoint happens upon transaction commit[1]. The comment makes it sound as if one or more transactions can commit before a checkpoint writes them to disk. [1]: http://msdn.microsoft.com/en-us/library/ms186259(v=sql.105)....

The transaction is actually written to disk twice. Once when it is written to the transaction log, and then again when a checkpoint happens and it's written to the underlying data blocks. Transaction log writes are sequential and fast, writes to the underlying data blocks are random and slow. The whole point of the checkpoint is to convert this sequential i/o to random i/o in an efficient manner.

Still, that's not what we're talking about. We're talking about the difference between writing data to the transaction log, and writing data to the transaction log and then flushing it. Once you call fsync() and the data is flushed you know for a fact (provided the hardware isn't lying to you) the bytes are on the damn platter, and not in some OS buffer cache.

Re: MySQL is bazillion times faster than MemSQL

#60

Post by midom »

nikita wrote: post #52
HarrisonFisk wrote: post #41

Earlier quoted context omitted.

Regarding #2, we do normally run MySQL with full transactional durability at Facebook. We have for a very long time (several years at least). For example, here is a FB note regarding our enhancing performance under full durability in MySQL from Oct 2010: https://www.facebook.com/note.php?note_id=438641125932

Harrison, what are these settings for InnoDB? innodb_flush_log_at_trx_commit, innodb_flush_methodm, innodb_doublewrite,

1, O_DIRECT, 1
Post reply on HN