Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does it scale? Who cares (2011)

jacquesmattheij.com

Post reply on HN
Print view View original

181–190 of 285 posts

Re: Does it scale? Who cares (2011)

#181

Post by edmccard »

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

>I know of absolutely no service which failed because it couldn't scale

So what are some examples of services which failed because they tried too hard to scale too early?

For example, when your network could handle 20k users, was there another network that could have already handled 500k, but they failed because they started a month after yours?

Re: Does it scale? Who cares (2011)

#182

Post by jaimex2 »

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

You started with Java... a language designed to handle credit card transactions world wide. Your story would be very different if you started with say Rails.

Re: Does it scale? Who cares (2011)

#183

Post by petters »

hasenj wrote: post #180
qume wrote: post #154

Earlier quoted context omitted.

I built a service (~10mm users at peak) which was designed from the ground up to scale, around 2002. When the numbers did grow we just sat back and watched it pretty much. Even given this, I completely agree with you. That's why I now develop in ruby. I'll take developer productivity over performance any day. It's the old saying - 'nice problem to have'.

> That's why I now develop in ruby. I'll take developer productivity over performance any day. In my experience, dynamically typed languages don't do well for developer productivity.

They do well for single-person code bases, but velocity scales badly with the number of developers in my experience.

Re: Does it scale? Who cares (2011)

#184

Post by abritinthebay »

hasenj wrote: post #180
qume wrote: post #154

Earlier quoted context omitted.

I built a service (~10mm users at peak) which was designed from the ground up to scale, around 2002. When the numbers did grow we just sat back and watched it pretty much. Even given this, I completely agree with you. That's why I now develop in ruby. I'll take developer productivity over performance any day. It's the old saying - 'nice problem to have'.

> That's why I now develop in ruby. I'll take developer productivity over performance any day. In my experience, dynamically typed languages don't do well for developer productivity.

And in my experience the exact opposite is true.

That's why it's called an anecdote; not data.

Re: Does it scale? Who cares (2011)

#185

Post by simonh »

hasenj wrote: post #180
qume wrote: post #154

Earlier quoted context omitted.

I built a service (~10mm users at peak) which was designed from the ground up to scale, around 2002. When the numbers did grow we just sat back and watched it pretty much. Even given this, I completely agree with you. That's why I now develop in ruby. I'll take developer productivity over performance any day. It's the old saying - 'nice problem to have'.

> That's why I now develop in ruby. I'll take developer productivity over performance any day. In my experience, dynamically typed languages don't do well for developer productivity.

Having worked on corporate Python platform with tens of thousands of developers and hundreds of applications running hubdress of thousands of instances on a many million line common core code base, I beg to differ.

We were sometimes able to go from concept to deployed application in a single day.

Re: Does it scale? Who cares (2011)

#186

Post by user5994461 »

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

There is scaling and there is scaling.

A messaging service is the most trivial service one can make, it's a well known easy problem that's been solved for decades with current technologies. There is no challenge in that. For comparison, WhatsApp had people with experience and they could handle 1 billion users with 50 people.

The fact that you can handle 10M users with a single untuned MySQL database is not a demonstration that scaling is overrated. It's an expression than you are running a trivial service that doesn't do much.

Almost any problems will be more challenging than that. There are endless companies that have 1/100th the customer base and yet require 100 times the data volume and engineering.

Re: Does it scale? Who cares (2011)

#188

Post by user5994461 »

simonh wrote: post #185
hasenj wrote: post #180

Earlier quoted context omitted.

> That's why I now develop in ruby. I'll take developer productivity over performance any day. In my experience, dynamically typed languages don't do well for developer productivity.

Having worked on corporate Python platform with tens of thousands of developers and hundreds of applications running hubdress of thousands of instances on a many million line common core code base, I beg to differ. We were sometimes able to go from concept to deployed application in a single day.

Having worked on corporate Python platform with tens of thousands of developers and thousands of applications running on thousands of instances on millions of lines common core code base, I beg to differ.

It is easy to ship any proof of concept. Is it hard to maintain, it is rigged with performance issues and it is almost impossible to refactor because python has zero refactoring support and no compiler to help you catch any error.

What you save on the initial rollout, you loose ten folds on the long term.

Re: Does it scale? Who cares (2011)

#189

Post by acdha »

grey-area wrote: post #118
frik wrote: post #104

Earlier quoted context omitted.

Friendster and MySpace failed because of scaling issues - but it was not scaling per se but bad execution: They choose the wrong technology, brought in the wrong CTOs who bought very expensive hardware (wrong, few expensive high end servers with expensive storage, instead several more low cost normal servers) and software with very high license costs (wrong, ColdFusion on dotNet (experimental) on Windows (high lic co…

As a counterpoint stack overflow is hosted on windows and did just fine. If your service is printing money you can easily fix tech problems (e.g. Facebook effectively rewriting php), if it's not you may flail around trying rewrites etc but the real problem lies elsewhere. Tech people are quick to find technical reasons for failure but the reasons are usually elsewhere.

Stack Overflow seems very relevant to this conversation given how unremarkable their architecture is – their level of traffic is comfortably running on 4 MS SQL Server boxes:

https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...

I've seen too many breathless posts which would have you believe they'd need a clustered NoSQL database or it wouldn't scale.

Re: Does it scale? Who cares (2011)

#190

Post by user5994461 »

seanp2k2 wrote: post #134
grey-area wrote: post #88

Earlier quoted context omitted.

I am willing to bet that 9 out of 10 'weekend projects' have had to be thrown out completely and redeveloped from scratch when the number of users became significant. 9 out of 10 weekend projects never get to a significant number of users.

Probably more like 99/100 or 999/1000 but yeah. Also, you can get pretty damn far with PHP and MySQL. Just look at Facebook. Just don't rely on Drupal or Wordpress to get you there and it'll be fine. KISS and solve the scaling issues as they come up. Far too many projects focus way too much on infrastructure and architecture and micro services instead of building something that solves a real problem for users. Focus…

An alternative view on history: Facebook succeeded because the competitors failed to scale.

Do you remember MySpace? The big social network with hundreds of million of users that came before facebook.

They had massive scaling and performance issues. At the peak when everyone was moving to social media (almost a decade ago) the site could take an entire minute to load (if loading at all). They lost a lot of users, who went straight to facebook and never recovered.

Post reply on HN