Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sharding Pinterest: How we scaled our MySQL fleet

engineering.pinterest.com

Post reply on HN
Print view View original

31–40 of 87 posts

Re: Sharding Pinterest: How we scaled our MySQL fleet

#31

Post by taf2 »

schneems wrote: post #21
pbreit wrote: post #10

Is it surprising that Pinterest is on MySQL? Has Postgres finally usurped the position of default DB for startups?

Came here to say the same. It makes sense that older companies are locked into their database as migrating would be too difficult like Facebook. I though Pinterest was a relatively young company and I'm surprised it chose mysql. Instagram is build on top of postgres and was founded in 2010, pinterest was founded in 2009.

Other than Oracle being a part of the equation MySQL is still an open source database and too quote the article:

"Aside: I still recommend startups avoid the fancy new stuff — try really hard to just use MySQL. Trust me. I have the scars to prove it."

In many ways, Postgres is cutting edge with it's features and capabilities. I see lots of updates from Postgres that include new SQL features (json objects) - conversely I see lots of updates from MySQL that are about durability and scalability e.g. (galera cluster, percona updates etc...)

From the use case Pinterest exposed in the article, I'm not sure what the additional features of Postgres would actually buy them?

Re: Sharding Pinterest: How we scaled our MySQL fleet

#32

Post by taf2 »

thezilch wrote: post #24
z3t4 wrote: post #19

Nice article and good explanations. But I still wonder why some people store blobs in a database rather then simple files on a file-system!?

There are a lot of things your FS is not going to give you (eg. transactions), and if you're going to have to query it anyway to get that, you might as well have your blob there too. For "small" blobs, say 1MB blobs: http://research.microsoft.com/pubs/64525/tr-2006-45.pdf

also operationally it's easier... you can rely on replication to backup those files instead of separate scripts that would need to work in-conjunction with the database replication.

If say you have the database with blobs - you can restore the db and everything is back up and running. If instead, you need to also restore the individual files... well - not only do you have lots of individual disk writes, you also have to make sure you still have all the files...

Re: Sharding Pinterest: How we scaled our MySQL fleet

#33

Post by detaro »

needusername wrote: post #27
sarnowski wrote: post #4

Earlier quoted context omitted.

Hi, at Zalando, we are scaling all of our core businesses with PostgreSQL. Depending on your dataset, it can be fairly easy to shard your data for a horizontal scale-out (think of independent customer datasets). We have lots of databases that we scale horizontally to much bigger numbers. But, we also developed several tools that makes working with shards mostly transparent. Did not find a better source but one way we…

5 TB easily fits on a single Oracle instance on a single host.

It also easily fits in a Postgres instance on a single host, if you only look at "can store X amount of data".

And even if Postgres is slower, for the money you save in license costs you can buy a few beefy nodes extra.

Re: Sharding Pinterest: How we scaled our MySQL fleet

#35

Post by rimantas »

schneems wrote: post #21
pbreit wrote: post #10

Is it surprising that Pinterest is on MySQL? Has Postgres finally usurped the position of default DB for startups?

Came here to say the same. It makes sense that older companies are locked into their database as migrating would be too difficult like Facebook. I though Pinterest was a relatively young company and I'm surprised it chose mysql. Instagram is build on top of postgres and was founded in 2010, pinterest was founded in 2009.

I believe Facebook had enough talent capable to choose their DBs carefully. And I am pretty sure MySQL replication had a lot to do with it. As having people who already knew how to finetune MySQL for big loads. BTW, believe it or not, Wikipedia once run on Postgresql, but was later migrated to MySQL (and currently they use MariaDB).

Re: Sharding Pinterest: How we scaled our MySQL fleet

#36

Post by techman9 »

taf2 wrote: post #31
schneems wrote: post #21

Earlier quoted context omitted.

Came here to say the same. It makes sense that older companies are locked into their database as migrating would be too difficult like Facebook. I though Pinterest was a relatively young company and I'm surprised it chose mysql. Instagram is build on top of postgres and was founded in 2010, pinterest was founded in 2009.

Other than Oracle being a part of the equation MySQL is still an open source database and too quote the article: "Aside: I still recommend startups avoid the fancy new stuff — try really hard to just use MySQL. Trust me. I have the scars to prove it." In many ways, Postgres is cutting edge with it's features and capabilities. I see lots of updates from Postgres that include new SQL features (json objects) - conversel…

JSON object storage for one? I'm a little confused as to why their schema relies on storing JSON data as a TEXT field in MySQL. I don't believe this is technically incorrect, but seems to negate many of the features of relational database design. Why not just store each entry as either a traditional column or use a solution that enables native JSON storage.

Re: Sharding Pinterest: How we scaled our MySQL fleet

#37

Post by axiak »

halayli wrote: post #6

This looks like a big hack to compensate for using the wrong tool. Cassandra would have been a better solution IMO. With Cassandra, you can set replication factors, speed up the writes, and automatically shard the data without having to manage your own "mapping tables".

They also use HBase for a lot of their non relational data.
Post reply on HN