Why Quora uses MySQL rather than No-SQL • Hacker News Board

Re: Why Quora uses MySQL rather than No-SQL

#51

Post by dangelo » Wed, Jun 30, 2010, 5:51 PM UTC

brown9-2 wrote: post #28

10ren wrote: post #11

> Schemas allow the data to persist in a typed manner across lots of new versions of the application as it's developed, they serve as documentation, and prevent a lot of bugs. So, schemaless vs. schemaful is similar to static typed vs. dynamic typed? (or is it more like typed vs. untyped?) > I think the "NoSQL" fad will end when someone finally implements a distributed relational database with relaxed semantics. What…

Also interesting is citing Facebooks giant usage of MySQL (1800 servers, 2 DBAs) but ignoring that they advocate pretty strongly for schemaless structures.

The built-in MySQL schemas have problems - mostly that schema changes can take hours or days on large tables. As a result of this you need to build a system that doesn't require schema changes at the MySQL level. This doesn't mean you can't use schemas at a higher level though.

There are two ways to build on top of MySQL - you can either make a flexible schemaless structure with objects and properties and associations between them, which is what Facebook advocates for, or you can make a flexible virtual schema that has column names, types, and arbitrary indexes. I'm in favor of the latter.

Re: Why Quora uses MySQL rather than No-SQL

#52

Post by astrange » Wed, Jun 30, 2010, 5:57 PM UTC

ericd wrote: post #44

known wrote: post #23

Isn't it possible to configure MySQL server and data to run in memory if we've enough RAM?

Yep, but I believe you have to handle persisting to disk yourself (there might be a Redis-like solution to this that I'm not aware of, though).

What does Redis do?

Re: Why Quora uses MySQL rather than No-SQL

#53

Post by bad_user » Wed, Jun 30, 2010, 8:22 PM UTC

vsync wrote: post #33

bad_user wrote: post #25

Earlier quoted context omitted.

It's just inertia ... I am familiar and worked with both ... but I still chose Mysql over Postgres simply because I have more confidence that I can control it. That's partly because Mysql always was more user friendly, and I also attended one Mysql conference and got to know some people working on Mysql. That said, overall Postgres is better, with one exception ... Mysql supports multiple storage engines for various…

> That's partly because Mysql always was more user friendly Examples? I started with Postgres and ignored MySQL from day one because of MySQL's deficiencies, and have had 0 problems learning what I needed in the years I've used it. Not to mention I can read a general SQL reference, or the standard, and the information actually applies. Unlike MySQL (quotation marks anyone?). > That said, overall Postgres is better, w…

> "various needs": what, for those times when you don't care whether your data gets silently corrupted?

I never had data-corruption in Mysql. Of course, I'm always working with InnoDB, maybe MyIsam is weaker

And if Mysql would have been such a weak RDBMS, maybe it wouldn't have been used by companies like Google or Amazon (especially Amazon RDS, which are already configured Mysql instances offered to clients in Amazon's cloud).

> "various needs": what, for those times when you don't care whether your data gets silently corrupted?

Ebay is using the memory tables engine in Mysql instead of Memcached. According to their benchmarks, the Mysql client is more stable, and the combination is more scalable.

There's also a whole ecosystem of third party storage engines. Like when Adobe developed an in-house distributed storage ... they also had an (internal project) storage engine for Mysql that interacted with it.

Also, the soon to be replacement for InnoDB is Falcon ... http://forge.mysql.com/wiki/Falcon (optimized for multiple cores).

Or RethinkDB (YC startup) ... a storage engine optimized for solid-state drives ... http://rethinkdb.com/

And I don't really care about what others think ... Mysql served me well.

Re: Why Quora uses MySQL rather than No-SQL

#54

Post by ericd » Thu, Jul 01, 2010, 12:30 AM UTC

astrange wrote: post #52

ericd wrote: post #44

Earlier quoted context omitted.

Yep, but I believe you have to handle persisting to disk yourself (there might be a Redis-like solution to this that I'm not aware of, though).

What does Redis do?

It's an in-memory hash store that is persisted to the disk when possible. Sort of like memcached that can survive restarts and is suitable for long-term storage.

Re: Why Quora uses MySQL rather than No-SQL

#55

Post by ericd » Thu, Jul 01, 2010, 12:32 AM UTC

bad_user wrote: post #48

ericd wrote: post #43

Earlier quoted context omitted.

Yeah, MySQL could definitely be improved upon. The table-level write locking is a pain. I was just saying that 100% consistency/data integrity isn't really a major concern for a lot of applications. I've thought about switching to Postgres, but it doesn't currently appear to be worth the pain in my case.

Locking in Innodb is row-level.

Yeah, I know, our conversation about read performance at the cost of data integrity being OK was about MyISAM (which doesn't have great concurrent write performance due to the table-level locking).

Re: Why Quora uses MySQL rather than No-SQL

#56

Post by astrange » Thu, Jul 01, 2010, 1:19 AM UTC

ericd wrote: post #54

astrange wrote: post #52

Earlier quoted context omitted.

What does Redis do?

It's an in-memory hash store that is persisted to the disk when possible. Sort of like memcached that can survive restarts and is suitable for long-term storage.

Well, I know that much. Did you mean that Redis just persists when you ask it to?

(I was counting that under "you have to do it yourself" and thought it might do something more automatic but still infrequent.)

Re: Why Quora uses MySQL rather than No-SQL

#57

Post by ericd » Thu, Jul 01, 2010, 2:16 AM UTC

astrange wrote: post #56

ericd wrote: post #54

Earlier quoted context omitted.

It's an in-memory hash store that is persisted to the disk when possible. Sort of like memcached that can survive restarts and is suitable for long-term storage.

Well, I know that much. Did you mean that Redis just persists when you ask it to? (I was counting that under "you have to do it yourself" and thought it might do something more automatic but still infrequent.)

Ah OK. I believe redis persists automatically when it can, from the memory, though I haven't actually used it. With a MySQL memory table, if you want to write to the memory table AND have it persist, you have to keep track of what has changed yourself, and handle writing those things to disk.

Re: Why Quora uses MySQL rather than No-SQL

#58

Post by jseifer » Thu, Jul 01, 2010, 2:45 AM UTC

fookyong wrote: post #5

rbranson wrote: post #4

Earlier quoted context omitted.

Sounds like pre-mature optimization, a bad idea which hackers need to be constantly reminded of.

yes that's exactly what it was. it was outsourced to a vendor that pretty much took advantage of the contracting startup's lack of DB knowledge. so they got the kitchen sink and a fat invoice. of course it's easy for the vendor to defend this - "what, we want you to be scalable!".

Flip side: How many startups ask for a solution that will scale to millions of users per day once their app goes live?

Re: Why Quora uses MySQL rather than No-SQL

#59

Post by zepolen » Thu, Jul 01, 2010, 6:51 AM UTC

Retric wrote: post #31

zepolen wrote: post #17

Earlier quoted context omitted.

So worst case they will be doing 4 req/s instead of 2. I know what you're trying to say, but at Quora's numbers, as OP said, it truly is irrelevant.

No, 5-20x * 2req/s av = 10 to 40 req/s.

Peak on a decent traffic site will be roughly 2x-3x the average. 5x-20x would be a blog that gets 10 visitors a day and after being posted on news.yc gets 100 visitors.

Re: Why Quora uses MySQL rather than No-SQL

#60

Post by c1sc0 » Thu, Jul 01, 2010, 12:29 PM UTC

mahmud wrote: post #47

generalk wrote: post #27

Earlier quoted context omitted.

Can't answer for Quora, but I know why I'm not: activation energy. My team and I know MySQL pretty well, and we can stand by our decision to use it. PostgreSQL we can't. It's not the most sound technical reasoning.

Postgres is easy to convert to from MySQL, if you keep that conversion cheatsheet handy. However, mastery of its performance quirks (all servers have quirks) can take years. It took me 48 hours to convert to it, 10 months later and I don't feel like I know much about it.

Isn't running a system for over 10 months without being forced to learn all the innate details a plus? Gives you nice 'look at the uptime of this baby' bragging rights without too much effort ;-)