Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does it scale? Who cares (2011)

jacquesmattheij.com

Post reply on HN
Print view View original

111–120 of 285 posts

Re: Does it scale? Who cares (2011)

#111

Post by jv22222 »

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

> I know of absolutely no service which failed because it couldn't scale.

Well, Friendster failed for exactly that reason. But, those were the early days.

p.s Completely agree with the article.

Re: Does it scale? Who cares (2011)

#112

Post by salman89 »

I agree with the general premise of avoiding premature optimizations, but designing systems that scale is important for several reasons:

- Startups grow exponentially, if you're playing catchup as you're growing you are focusing on keeping the lights on and hanging on for the ride. Important for a growing company to focus on vision.

- Software that scales in traffic is easier to scale in engineering effort. For example, harder for a 100 engineers to work on a single monolith vs 10 services.

- Service infrastructure cost is high on the list of cash burn. Scalable systems are efficient, allow startups to live longer.

- If the product you are selling directly correlates to computing power, important to make sure you are selling something that can be done profitably. For example, if you are selling video processing as a service, you absolutely need to validate that you can do this at scale in a profitable manner.

I also don't agree with the premise that speed of development and scalable systems are always in contention. After a certain point, scalable systems go hand and hand with your ability to execute quickly.

Re: Does it scale? Who cares (2011)

#114

Post by jssmith »

carbocation wrote: post #46
technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

This is a delightful retelling; thanks for sharing. > I know of absolutely no service which failed because it couldn't scale. Genuine question: what about Friendster?

Friendster definitely failed because they couldn't pull off scale. I was CTO of the Tagged social network at the time and the issue loomed large during our 2005 fundraising. Investors had seen $50m go down the drain when Friendster got slow. Despite network effects, users moved over to products that were up and running (mostly MySpace but also hi5, etc.).

I seem to recall that the Friendster team had more than one aborted rewrite before they got it right, and by then it was too late. Scaling social networks required a different bag of tricks than was popular at the time (e.g., database replication was well accepted as a best practice, but sharding worked much better). Talented people could still get it wrong.

Still, social networking illustrates how failing to scale is rare, and should practically never be the concern early on. Almost every social network that gained traction struggled to scale at times, but the vast majority of them overcame their challenges.

We poured sweat and tears and many sleepless nights into scaling the Tagged back-end from 10 machines to over 1,000, over time serving hundreds of millions of users. It was a tremendous amount of work, much more than building the initial product, and I don't think there's much we could have done early on to make later scaling easier.

More on the story: http://highscalability.com/blog/2011/8/8/tagged-architecture...

Re: Does it scale? Who cares (2011)

#115

Post by yummyfajitas »

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

I can name one: Darcs. It was my favorite certain control system, apart from the lack of scalability.

Re: Does it scale? Who cares (2011)

#116

Post by emmelaich »

devduderino wrote: post #11

I care because it usually goes like this: Product manager > "Niche sass app {x} will never need to support more than 10-20 users" Two weeks after launch > "We have 10k users and counting, why didn't you architect this for scale?" Always assume you underestimated the scope of the project.

It's a good things to overestimate in a Big Corp, because the managers always penny pinch and will put the blame on you if things go to shit.

If you're a startup, go the other way.

Re: Does it scale? Who cares (2011)

#117

Post by ajdlinux »

pbreit wrote: post #66
Alex3917 wrote: post #33

Earlier quoted context omitted.

> I know of absolutely no service which failed because it couldn't scale. If by scaling you mean increasing the number of page views that a given version of a web app can serve, then this is mostly true. But things like conceptual simplicity, unit economics, codebase readability, strategy, etc. are also dimensions of scaleability. E.g. the only reason that startups can even exist at all is because large companies hav…

"scale" here, as with all conversations on HN about "scale", refers to the technology required to service a large number of users or usage.

I think you'll find there are plenty of conversations here where "scale" is used in reference to stuff like the business models of "gig economy" apps, or at what point your startup has enough scale to need an HR department, and so on and so forth.

Re: Does it scale? Who cares (2011)

#118

Post by grey-area »

frik wrote: post #104
cyberferret wrote: post #52

Earlier quoted context omitted.

> I know of absolutely no service which failed because it couldn't scale. I would say this is because of the simple fact of visibility and adoption. You've probably never heard of these services probably because they ground to a halt with a mere 1000 users, so they never got mainstream enough to be recognised as a viable service. It is a bit like how no one remembers the dozens of people who failed to achieve sustain…

Friendster and MySpace failed because of scaling issues - but it was not scaling per se but bad execution: They choose the wrong technology, brought in the wrong CTOs who bought very expensive hardware (wrong, few expensive high end servers with expensive storage, instead several more low cost normal servers) and software with very high license costs (wrong, ColdFusion on dotNet (experimental) on Windows (high lic co…

As a counterpoint stack overflow is hosted on windows and did just fine. If your service is printing money you can easily fix tech problems (e.g. Facebook effectively rewriting php), if it's not you may flail around trying rewrites etc but the real problem lies elsewhere.

Tech people are quick to find technical reasons for failure but the reasons are usually elsewhere.

Re: Does it scale? Who cares (2011)

#120

Post by stephenr »

yummyfajitas wrote: post #115
technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

I can name one: Darcs. It was my favorite certain control system, apart from the lack of scalability.

Darcs is a distributed version control system. It is inherently not a hosted solution that "needs to scale" the way the article is talking about.
Post reply on HN