Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling to 100M: MySQL Is a Better NoSQL

blog.wix.engineering

Post reply on HN
Print view View original

91–100 of 183 posts

Re: Scaling to 100M: MySQL Is a Better NoSQL

#91

Post by kyledrake »

fapjacks wrote: post #52
grossvogel wrote: post #21

Earlier quoted context omitted.

By "taken hostage their images," do you mean that literal graphic files uploaded to Wix servers were somehow made inaccessible to the user?

Yes, exactly. I don't know how they do it now, but Wix did the "one big Flash blob as a website" and did not make data available to clients to download once they had been uploaded. So images and other data that had been "compiled" into the Flash blob were erased or something. There was no warning about this, and it took many by surprise. This effectively forced people to renew their subscription to Wix who otherwise…

Weird. I wouldn't ever do anything like that with Neocities. Downloading a site is a button click on the bottom of everyone's dashboard (it spits out a zipball of the files). Flash is anachronistic as all hell anyways.

It's weird to me that they're hosting web sites that need inline MySQL at all, but I suppose they probably do a lot of fancy backend stuff. Much better to async queue any data sending unless it needs to be inline. Much better than that is to static cache.

We're straight up static + nginx for all site serving, via an anycast geo CDN. Logging is passively provided by the nginx logfiles, which are parsed hourly asynchronously. It's a pretty good system, and about as fast as site delivery can get. I guess I don't know what Wix offers for site features, but for simple web sites, why bother with a database for web site display at all?

That said, I agree with the article's premis. I'm partisan to PostgreSQL but it's dumb to say it doesn't work and that some wacky new TrendDB that doesn't handle fsync or atomicity correctly (and therefore is TEH SCALE!) is somehow better for this job.

Re: Scaling to 100M: MySQL Is a Better NoSQL

#92

Post by 3pt14159 »

suneilp wrote: post #60
luka-birsa wrote: post #9

Earlier quoted context omitted.

We've drank the NoSQL coolaid mostly as we prefered a schemaless approach to our database and couchdb looked like a cool thing to use. Tested, deployed in production abd after a while we figured out that most of the promises about performance, stability, etc we're mostly bull. HStore was released, we've migrated to PG and we can't be happier. Zero issues so far.

Couchdb isn't exactly the best thing to judge NoSQL by today especially by it's old performance issues, lack of automatic compaction, indexing on demand instead of proactive background indexing, etc.

At what point do I get to tout out the No True Scotsman paradox? Every NoSQL solution whether as its own DB or built into another one (like Postgres) I've worked with I've either hated (Couch, Mongo, Cassandra, Neo4j), or relegated to _very_ specific use-cases (Riak, Redis, Postgres).

I agree with the Wix team. I've used Mongo in a very high volume site and it was a constant source of headaches. We even had an engineer we called "MongoDB" because he was constantly having to deal with scaling it. It didn't even handle more than 10% of the application data, it was strictly for timelines; the rest of it was in trusty old MySQL. If we'd just done blobs as JSON in MySQL in the first place we would have been completely fine.

Even though I prefer Postgres, knowing nothing else, I would prefer 100% SQL in MySQL than a Postgres backed application that splits the storage between tables and KV store.

Re: Scaling to 100M: MySQL Is a Better NoSQL

#93

Post by spotman »

koolba wrote: post #56

So much to disagree with here ... > Locks limit access to the table, so on a high throughput use case it may limit our performance. Then use a proper database that implements MVCC. > Do not use transactions, which introduce locks. Instead, use applicative transactions. Or just use a database that handle transactions more efficiently. > `site_id` varchar(50) NOT NULL, Why varchar(50)? UUIDs are 16-bytes. The best way…

>> Locks limit access to the table, so on a high throughput use case it may limit our performance.

> Then use a proper database that implements MVCC.

InnoDB does implement MVCC. MVCC is not a silver bullet.

>> Do not use transactions, which introduce locks. Instead, use applicative transactions.

> Or just use a database that handle transactions more efficiently.

Easy to say, hard to implement at this scale. If you do a lot of writes and reads concurrently to a hot dataset, it's really quite hard to beat this architecture. This is why its such a popular and battle tested solution for many extremely high scale applications with workloads like this. Not to mention extremely well understood.

>> Do not normalize.

> Bullshit. Normalize as much as is practical and denormalize as necessary. It's much easier to denormalize and it greatly simplifies any transaction logic to deal with a normalized model.

But we are talking about performance... Having something in a single table that is denormalized is always going to be faster than having an elegant data model with "Everything In It's Right Place"

>> Fields only exist to be indexed. If a field is not needed for an index, store it in one blob/text field (such as JSON or XML).

> This is terrible advice.

So facebook/friendfeed, uber, dropbox, and many more or wrong then. Ok.

This is really all best practice for running something like this.

Of course it flies in the face of best practice for running a smaller system. Is there tradeoffs? Absolutely! Would it be smart to do this if the need for this scale is not obvious? Probably not.

You end up having more logic in your application and coordination layers, but this is all pretty good advice for people at this scale, and certainly not bad at all.

Re: Scaling to 100M: MySQL Is a Better NoSQL

#94

Post by philwelch »

pmontra wrote: post #47
Thaxll wrote: post #29

Earlier quoted context omitted.

MySQL is faster than pg, you should ask why Facebook is running the largest MySQL shop.

Maybe because they started with a tiny LAMP system in 2004 and got stuck there to the point they invested considerable resources to write their own PHP interpreter and optimize MySQL. From https://www.percona.com/blog/2014/03/27/a-conversation-with-... "we had the MySQL engineering talent we needed to work with the Oracle team to get 5.6 ready for production at our scale." "We all worked hard to adapt 5.6 to our scal…

Also, if you read their engineering papers it turns out Facebook uses MySQL these days as a storage backend to a graph store and not as a relational database. They also use the replication logs to power their pub-sub system. In other words, Facebook's use case is so large scale and so throughly hacked (in a good way) that it's not necessarily a credit to the idea of using MySQL on a greenfield project.

Re: Scaling to 100M: MySQL Is a Better NoSQL

#95

Post by mianos »

Buttons840 wrote: post #50
stplsd wrote: post #7

Earlier quoted context omitted.

Is it? MySQL improved a lot since 5.1 days, you know.

Has MySQL added support for Common Table Expressions (CTE) yet? CTE was added to the SQL standard in 1999. I ask because I always miss this feature when querying MySQL.

Trolling along, what about working correlated subqueries that don't degrade into a cartesian join of all rows?

Re: Scaling to 100M: MySQL Is a Better NoSQL

#96

Post by chises »

mh-cx wrote: post #80

I have not heard about "Wix" before, but maybe they should have done some more research before picking this name. To a German this sounds like "wichsen" which means, well, "wank"[1]. [1] http://dict.leo.org/ende/index_de.html#/search=wichsen

they picked the similarity to the german word "wichsen" in their campaign in germany... https://www.youtube.com/watch?v=4AKDZmsy5yo it says "everyday million of people are wanking – wanking changed my live – when my girlfriend felt asleep, i'm going to wank – i love wanking – my wife convinced me to wank – i'm wanking after my training – to be honest, we wank together most of the times – wanking is the future" and the hardest/badest part is the last sentence "make it by yourself – be a wanker"

Re: Scaling to 100M: MySQL Is a Better NoSQL

#97

Post by nerfhammer »

calpaterson wrote: post #66
koolba wrote: post #56

So much to disagree with here ... > Locks limit access to the table, so on a high throughput use case it may limit our performance. Then use a proper database that implements MVCC. > Do not use transactions, which introduce locks. Instead, use applicative transactions. Or just use a database that handle transactions more efficiently. > `site_id` varchar(50) NOT NULL, Why varchar(50)? UUIDs are 16-bytes. The best way…

This article is pretty terrible but just in point of fact: it looks like they are using MySQL's InnoDB backend - which does support transactions and MVCC. If they're even talking about avoiding transactions for speed purposes (no matter how stupidly) they must be talking about Inno because in MyISAM BEGIN and COMMIT are no-ops.

The article says "Note that a transaction is using a DB-level lock that prevents concurrent writes—and sometimes reads—from the affected tables."

In innodb locks are row-level; myisam supports table-level locks though that's not and shouldn't be confused with a transaction; I don't know what a "database-level" lock is supposed to mean, are they really saying they're locking all tables to do a write? It doesn't sound like this author understands what a transaction is.

Re: Scaling to 100M: MySQL Is a Better NoSQL

#98

Post by sriram_sun »

jjoe wrote: post #36

Scalability is like an abstract painting. It's unique to one's infrastructure. Its writing or sometimes postmortem makes good brain fertilizer. Not so much more. Beyond that I wouldn't rush to implement scalability du jour. A setup that works for a certain service won't necessarily work for another unless yours is a very close replica. Based on my experience in this area, and I'm a performance seeking nut, each platf…

That is an awesome way to describe scalability! Can you give a couple of examples?

Re: Scaling to 100M: MySQL Is a Better NoSQL

#100

Post by y0ghur7_xxx »

electrotype wrote: post #79

A little bit off topic, but I would like to hear more about using Solr [1] instead of any "real" NoSQL databases. I don't have experience with MongoDb and such, but I've always asked myself why someone wouldn't use Solr as a distributed NoSQL database... Am I wrong or, with Solr, you get that key/value scalable storage AND you get advanced search features as an extra? Why would I want to use MongoDb instead of Solr?…

We evaluated solr for a project just a week ago. It does not have authn & authz that mongo has, and that was a feature we needed. Other that that, if all you need is a kv store, solr is great.
Post reply on HN