Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

highscalability.com

Post reply on HN
Print view View original

31–40 of 88 posts

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#31

Post by whalesalad »

irahul wrote: post #21
lionheart wrote: post #16

Earlier quoted context omitted.

Yeah, I'm actually in the process of switching to Python/Django from Rails, so its great to hear how well it scales.

At 18m visitors, I don't think your choice of Django or Rails matters. DB tends to be the limit for even the normal applications, let alone something with that kind of volume.

There's a kabillion factors to consider though. With that many users, yes the exact implementation matters less and the ancillary systems and how they interact are more critical. But, building the platform, being able to iterate quickly and deliver robust functionality along the way is important as well. Not that Ruby hasn't, but Django and Python have really proved their worth with some of the heavy hitters recently. Instagram, Pinterest, FriendFeed, Disqus... It's an impressive roster.

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#32

Post by tnash »

kitsune_ wrote: post #17
carson wrote: post #13

Earlier quoted context omitted.

You are just counting EC2 cost. The AWS cost for 410TB of S3 storage is around $39k. You would need to add in BW cost on top of that. It is also interesting that they seem to be using Akamai for a CDN instead of Cloudfront so not a completely AWS based solution. I wish they went into what they are storing in S3. 410TB is a lot of storage. My initial guess was cached images but 80M objects breaks down to 5MB per objec…

From these figures they seem to burn over 100k per month on outsourced cloud services alone. Holy shit.

Yeah...that's insane. Since they're a "pre-revenue" company, they're just burning money.

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#33

Post by PanMan »

teoruiz wrote: post #24
carson wrote: post #13

Earlier quoted context omitted.

You are just counting EC2 cost. The AWS cost for 410TB of S3 storage is around $39k. You would need to add in BW cost on top of that. It is also interesting that they seem to be using Akamai for a CDN instead of Cloudfront so not a completely AWS based solution. I wish they went into what they are storing in S3. 410TB is a lot of storage. My initial guess was cached images but 80M objects breaks down to 5MB per objec…

That's actually right, S3 costing at $39k/month seems completely crazy. No wonder 37signals decided to switch to their self-hosted storage solution[1]. [1] http://37signals.com/svn/posts/2483-nuts-bolts-storage

As they just raised 100 million, I wonder if putting (a lot of) effort in replicating S3's functions to save part of 40K/month is worth the trouble. And setting that up won't be free either. (their 100 million can pay for the current storage for 200 years...)

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#34

Post by edwinnathaniel »

wamatt wrote: post #28
edwinnathaniel wrote: post #3

Earlier quoted context omitted.

Pardon my ignorance: what is interesting about Pinterest stack? I think by now most startups, YC or not, in Silicon Valley will pretty much have a similar setup: 1) Choose the main web-stack (Rails or Django) 2) Choose the API framework (node.js, or something else) 3) Memcached for caching (or some NoSQL) 4) A message queue (ZeroMQ, RabbitMQ, or something ...) 5) nginx, HAProxy, Varnish, (or similar technology) 6) Ha…

Seems like a sensible stack. However, is it not reasonable that would someone not consider node.js, as web stack as well? I will agree the focus is more towards backend and asynchronous API's traditionally, but the amount of front-end/UX orientated modules are growing every day.

Sure why not. Some people don't even mind the lack of polished modules and instead spend time to write their own.

Not a big fan of node.js (or JavaScript for that matter) and probably never will so I can't comment much.

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#35

Post by edwinnathaniel »

blantonl wrote: post #5
rudiger wrote: post #4

It's cool that we're seeing companies reach 1 billion USD valuations with 12 employees.

It will be even cooler when we see companies reach 1 billion USD net-income with 12 employees.

I heard Microsoft Visio team probably have less than 50 people working on it (granted the name Microsoft and the Office brand provide a huge selling point).

Visio almost hit $1B according to insider... (can't verify).

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#36

Post by edwinnathaniel »

whalesalad wrote: post #31
irahul wrote: post #21

Earlier quoted context omitted.

At 18m visitors, I don't think your choice of Django or Rails matters. DB tends to be the limit for even the normal applications, let alone something with that kind of volume.

There's a kabillion factors to consider though. With that many users, yes the exact implementation matters less and the ancillary systems and how they interact are more critical. But, building the platform, being able to iterate quickly and deliver robust functionality along the way is important as well. Not that Ruby hasn't, but Django and Python have really proved their worth with some of the heavy hitters recently…

... and when we look at the um... "old" fart Java... their roster is even more impressive: LinkedIN, Google, NetFlix, Amazon.

Just commenting ... :D

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#37

Post by moe »

al_james wrote: post #27

Does anyone else feel that 410TB of user data seems quite a lot? If I have my maths right, even if all the 80 million objects are user data (as opposed to, say, logs) thats 5.3mb per object. Considering that most pinterest photos are from the web, that seems quite big.

Yes, that number is ridiculous.

For reference, that's about four times bigger than the iTunes music catalog (20mio MP3 files * 5MB average filesize = 100TB).

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#38

Post by moe »

tnash wrote: post #32
kitsune_ wrote: post #17

Earlier quoted context omitted.

From these figures they seem to burn over 100k per month on outsourced cloud services alone. Holy shit.

Yeah...that's insane. Since they're a "pre-revenue" company, they're just burning money.

"Burning" seems like an understatement.

Looks like a switch to dedicated hardware would amortize within... 3 months.

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#39

Post by swah »

edwinnathaniel wrote: post #3
pajju wrote: post #2

They have a very Interesting Stack as seen on Quora[1]: -Python + heavily-modified Django at the application layer -Tornado and (very selectively) node.js as web-servers. -Memcached and membase/redis for object and logical-caching -RabbitMQ as a message queue. -Nginx, HAproxy and Varnish for static-delivery and load-balancing. -Persistent data storage using MySQL. -MrJob on EMR for map-reduce. [1] http://www.quora.co…

Pardon my ignorance: what is interesting about Pinterest stack? I think by now most startups, YC or not, in Silicon Valley will pretty much have a similar setup: 1) Choose the main web-stack (Rails or Django) 2) Choose the API framework (node.js, or something else) 3) Memcached for caching (or some NoSQL) 4) A message queue (ZeroMQ, RabbitMQ, or something ...) 5) nginx, HAProxy, Varnish, (or similar technology) 6) Ha…

I would think that Nginx + Django + SQL, and the rest is premature optization...

Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB

#40

Post by tomwaddington »

al_james wrote: post #27

Does anyone else feel that 410TB of user data seems quite a lot? If I have my maths right, even if all the 80 million objects are user data (as opposed to, say, logs) thats 5.3mb per object. Considering that most pinterest photos are from the web, that seems quite big.

Original image plus two (or more) scaled thumbnails. Also, I don't think they're doing any reuse (upload the same pin for 2 different S3 objects of the same image). 5.3Mb / object is still really high though.
Post reply on HN