Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does it scale? Who cares (2011)

jacquesmattheij.com

Post reply on HN
Print view View original

161–170 of 285 posts

Re: Does it scale? Who cares (2011)

#161

Post by reledi »

Trundle wrote: post #141
shadowmint wrote: post #138

I care. It's easy to brush off scaling concerns as not important, but I've had personal experience where it's mattered, and if you want a high profile example, look at twitter. Yes, premature optimization is a bad thing, and so is over engineering; but that's easy to say if you have the experience to make the right initial choices that mean you have a meaningful path forward to scale when you do need it. For example,…

The article is about choosing a business idea where technical scale isn't important. Twitter would fall under a "media play" which is what the author is telling the reader not to do. They aren't saying "try to build twitter but skimp on the tech" they're saying "don't try to build twitter, build something where people pay you money to use it and you'll be raking in millions in profit by the time technical scaling is…

Twitter is not exempt. They focused on the product before scale. They were regularly overloaded in the early days.

Re: Does it scale? Who cares (2011)

#162

Post by iveqy »

Having working on an app that we just throwed more hardware at, to the point where the azure subscription cost could be lowered by my whole anually salary, by 3 months work of optimization.

I believe performance does matter. We where a 4 person team and could have added a fifth if we had a cheaper design.

Re: Does it scale? Who cares (2011)

#163

Post by im3w1l »

pbreit wrote: post #107
milesrout wrote: post #21

Earlier quoted context omitted.

>Everything has flaws False equivalence. PHP has many, many more flaws to a much deeper and more serious level than any other mainstream programming language. It's insecure, buggy, full of broken behaviour for legacy systems, slow and easy to misuse. Its standard library is inconsistent, hard to learn, easy to misuse, full of legacy behaviour and slow. >but PHP is a reasonable choice for many problems. PHP is an unre…

When you have that sort of attitude about 2 of the most widely used languages/platforms in existence, I think it's more a reflection on you.

Well he does like python which is above them both in popularity.

Re: Does it scale? Who cares (2011)

#165

Post by nomercy400 »

Once worked at a startup where we expected 'some activity' in our webshop at product launch, and didn't think about scaling for that. Well, some activity turned out to be 450mbit/s for five hours, which our unscalable application/webshop didn't handle very well. It became overloaded in the first minute, and took us more than an hour to get remote access again. It's one of those things we did better for our next big event (major sharding, basically replicated the application 32 times of the largest VM instance we could get. It was needed and it survived).

Re: Does it scale? Who cares (2011)

#166

Post by frik »

grey-area wrote: post #118
frik wrote: post #104

Earlier quoted context omitted.

Friendster and MySpace failed because of scaling issues - but it was not scaling per se but bad execution: They choose the wrong technology, brought in the wrong CTOs who bought very expensive hardware (wrong, few expensive high end servers with expensive storage, instead several more low cost normal servers) and software with very high license costs (wrong, ColdFusion on dotNet (experimental) on Windows (high lic co…

As a counterpoint stack overflow is hosted on windows and did just fine. If your service is printing money you can easily fix tech problems (e.g. Facebook effectively rewriting php), if it's not you may flail around trying rewrites etc but the real problem lies elsewhere. Tech people are quick to find technical reasons for failure but the reasons are usually elsewhere.

1) Friendster had scaling issues. They insisted on keeping the x-degree-of-friendship-calculation which is quite resource expensive and doesn't scale. Instead making the software scalable, they brought in expensive exec who decided to through more and more very high end servers and enterprise storage on it. It eat their startup money, and they were constantly firefighting with very high page load time like 12+ sec, instead of advancing the site for at least one year. MySpace took over Friendster because of this.

2) MySpace culture and management was in trouble due Murdoch's News Corporation bought MySpace's parent company. Instead of investing News Corp choose the wrong path. Instead of improving the site, they decided to do a (in the end) very costly deal with MS to switch their ColdFusion stack from Java stack to dotNet stack incl very expensive MSSQL licenses and various other license costs. It turned out ColdFusion on dotNet was still in very experimental phase and MySpace suffered from the caused troubles a lot and bleed money like never before (which made News Corp very unhappy). MySpace website got little updates for at least one year while MySpace devs were busy with firefighting and switching backend. Facebook took over.

There are books worth reading.

3) Stackoverflow (like 2 years ago) run on less than a dozen of servers. It's several magnitudes smaller than social network services like (former) MySpace, Facebook, Twitter, etc. License costs often don't scale, you bleed through your startup money for little competitive advanced features or return. That's why successful startups often choose open source stack, look at Amazon, Google, (former) Yahoo, Facebook, Twitter, etc. - Perl, Python, PHP, Java, Ruby, MySQL, Postgres, Hadoop, etc. Stackoverflow shows it can be done with off the shelf COTS as well, if you keep the server license count low by wise decisions and performance tuning. But it's not like I could name dozends of successful startups that have a software stack like Stackoverflow. And even Hotmail, Linkedin, Bing run or used to run for the majority of their service-live on mainly open source software stack.

Re: Does it scale? Who cares (2011)

#167

Post by Michielvv »

salman89 wrote: post #112

I agree with the general premise of avoiding premature optimizations, but designing systems that scale is important for several reasons: - Startups grow exponentially, if you're playing catchup as you're growing you are focusing on keeping the lights on and hanging on for the ride. Important for a growing company to focus on vision. - Software that scales in traffic is easier to scale in engineering effort. For examp…

In the projects that I was involved in in the past years, the ones that decided to build a scalable infrastructure before gaining traction were exactly those that were burning cash on infrastructure. There is just a reasonable minimum of servers you need to run such an infrastructure and then you need something similar as a staging environment to make sure it all works together correctly.

That being said, there are of course choices that can be made early on that will help you in case you need to scale at some time. Spending a bit of time thinking through what would happen if you do need to split up and scale out your system will help you avoid pitfalls such as relying too much on the local filesystem being shared across pageviews.

In general I believe there is more benefit on spending time on performance early on. (not caching, that is just postponing the problem) as it benefits not just your ability to run on a single system for much longer, it will also make life better for your users.

Re: Does it scale? Who cares (2011)

#168

Post by robk »

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

Arguably Friendster failed due to its interminable scaling issues

Re: Does it scale? Who cares (2011)

#169

Post by logicallee »

qume wrote: post #154
technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

I built a service (~10mm users at peak) which was designed from the ground up to scale, around 2002. When the numbers did grow we just sat back and watched it pretty much. Even given this, I completely agree with you. That's why I now develop in ruby. I'll take developer productivity over performance any day. It's the old saying - 'nice problem to have'.

>I built a service (~10mm users at peak) which was designed from the ground up to scale, around 2002. When the numbers did grow we just sat back and watched it pretty much.

>Even given this, I completely agree with you. That's why I now develop in ruby.

Ouch that is a pretty big burn.

Re: Does it scale? Who cares (2011)

#170

Post by weixiyen »

qaq wrote: post #34
milesrout wrote: post #21

Earlier quoted context omitted.

>Everything has flaws False equivalence. PHP has many, many more flaws to a much deeper and more serious level than any other mainstream programming language. It's insecure, buggy, full of broken behaviour for legacy systems, slow and easy to misuse. Its standard library is inconsistent, hard to learn, easy to misuse, full of legacy behaviour and slow. >but PHP is a reasonable choice for many problems. PHP is an unre…

OK here's a list of things I miss from PHP when doing large projects in Python. 1) Much better typing support 2) interfaces 3) visibility control 4) composer As far as none would choose it for new project don't know if you ever heard of this company called Slack they chose PHP fairly recently and seem to be doing OK.

Not trying to argue about languages, but Slack is not a good example of a great technical product - backend wise. It's bloated, slow, can't support cases with too many users, and makes way too many requests just to initialize.
Post reply on HN