It's nice to see that two companies valued at >= $1 billion are running a Python/Django stack. #teampony
Yeah, I'm actually in the process of switching to Python/Django from Rails, so its great to hear how well it scales.
Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
21–30 of 88 posts
Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#22Earlier quoted context omitted.
Yeah, I'm actually in the process of switching to Python/Django from Rails, so its great to hear how well it scales.
I'm curious as to why you're switching. Care to share? I'm a long-time Django guy who's just been thrown into some Rails projects so I am going to be learning it shortly.
The link above is the modified, famous "blog in 10 minutes". When I was starting, I found it very pleasant as it covers the 20% of the Rails which you use 80% of the time.
Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#23Earlier quoted context omitted.
Pardon my ignorance: what is interesting about Pinterest stack? I think by now most startups, YC or not, in Silicon Valley will pretty much have a similar setup: 1) Choose the main web-stack (Rails or Django) 2) Choose the API framework (node.js, or something else) 3) Memcached for caching (or some NoSQL) 4) A message queue (ZeroMQ, RabbitMQ, or something ...) 5) nginx, HAProxy, Varnish, (or similar technology) 6) Ha…
Oh, I wish it was that simple. You can't mix and match (you can theoretically ) Let's say you began with Django, right? Or there was something already ready in Django. And then you begin to see the warts. But ok, you keep churning along. And the more you churn the more of a specialist in the deficiencies of each technology you become. Like the fact that Django's ORM runs like a dog. Or see all the "we moved to MongoD…
Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#24According to the article Pinterest is spending on AWS EC2 >$30k to support 18M visitors/month. Data: $52/h (peak time, let's say 18 out of 24 hours) and $15/h (night time, let's say 6/24). Edit: as pointed in the comments $30k/month would only be the EC2 costs.
You are just counting EC2 cost. The AWS cost for 410TB of S3 storage is around $39k. You would need to add in BW cost on top of that. It is also interesting that they seem to be using Akamai for a CDN instead of Cloudfront so not a completely AWS based solution. I wish they went into what they are storing in S3. 410TB is a lot of storage. My initial guess was cached images but 80M objects breaks down to 5MB per objec…
No wonder 37signals decided to switch to their self-hosted storage solution[1].
Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#25Earlier quoted context omitted.
Oh, I wish it was that simple. You can't mix and match (you can theoretically ) Let's say you began with Django, right? Or there was something already ready in Django. And then you begin to see the warts. But ok, you keep churning along. And the more you churn the more of a specialist in the deficiencies of each technology you become. Like the fact that Django's ORM runs like a dog. Or see all the "we moved to MongoD…
Yes, but the Django ORM helped them launch faster. There is nothing wrong using it to start with and then writing your own SQL queries when you need to scale or even changing your data model to accommodate scaling.
Use the tools to launch faster, then optimize.
Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#26Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#27Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#28They have a very Interesting Stack as seen on Quora[1]: -Python + heavily-modified Django at the application layer -Tornado and (very selectively) node.js as web-servers. -Memcached and membase/redis for object and logical-caching -RabbitMQ as a message queue. -Nginx, HAproxy and Varnish for static-delivery and load-balancing. -Persistent data storage using MySQL. -MrJob on EMR for map-reduce. [1] http://www.quora.co…
Pardon my ignorance: what is interesting about Pinterest stack? I think by now most startups, YC or not, in Silicon Valley will pretty much have a similar setup: 1) Choose the main web-stack (Rails or Django) 2) Choose the API framework (node.js, or something else) 3) Memcached for caching (or some NoSQL) 4) A message queue (ZeroMQ, RabbitMQ, or something ...) 5) nginx, HAProxy, Varnish, (or similar technology) 6) Ha…
However, is it not reasonable that would someone not consider node.js, as web stack as well?
I will agree the focus is more towards backend and asynchronous API's traditionally, but the amount of front-end/UX orientated modules are growing every day.
Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#29Those kind of articles makes me feel so stupid. Even though those technologies are ready to go, making them work smoothly without any interruption always seems hard to me. I believe, there has to be lots of tips and trick. ( in other words you have to be experienced with all them or I am very lazy) I was wondering if any common receipt exists or if there is someone who can answer my couple of high scalability questio…
Re: Pinterest Architecture Update - 18M Visitors, 10x Growth, 12 Employees, 410 TB
#30Earlier quoted context omitted.
I'm curious as to why you're switching. Care to share? I'm a long-time Django guy who's just been thrown into some Rails projects so I am going to be learning it shortly.
From a Django, Rails, Flask, Sinatra guy; Rails is going to be fun if you leave your pre-conceived notions at the door. Rails is not Django, and Ruby is not Python, and any attempts to make it behave like how it's not supposed to behave will cause pain. However, if you can play per Rails rules(and you should), that would be a very pleasant experience. How are you learning Rails? Since you are an experienced programme…