Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does it scale? Who cares (2011)

jacquesmattheij.com

Post reply on HN
Print view View original

151–160 of 285 posts

Re: Does it scale? Who cares (2011)

#151

Post by std_throwaway »

shadowmint wrote: post #138

I care. It's easy to brush off scaling concerns as not important, but I've had personal experience where it's mattered, and if you want a high profile example, look at twitter. Yes, premature optimization is a bad thing, and so is over engineering; but that's easy to say if you have the experience to make the right initial choices that mean you have a meaningful path forward to scale when you do need it. For example,…

I don't like opinionated pieces like the one presented here because while they are right about some things they miss other things and present half-truths as full-truths.

In my experience (and I have little of that) it's important to know the upgrade path and adjust your planning accordingly.

How many users can you serve with your solution?

How big do you expect the market to be in that stage?

What technology would be the next step?

How do you get there? How much more work would it be?

Always be one step ahead with technology, but not two. Most markets are surprisingly small. Most use cases scale surprisingly well. You can probably push your solution by an order of one magnitude if you need it quickly.

In order to answer the questions you need people who know the product, the (potential) technologies and the market. When you start you probably won't know any of that. See the first prototype you deploy to the customers as a means of collecting data for the first production version. Your first product is not your first product. Your funding should respect that. Get it done quickly with the aim of answering the critical questions. Then go back and design the next version "good-enough" for the second scaling step with the upgrade path in mind.

Re: Does it scale? Who cares (2011)

#152

Post by tajen »

iamd3vil wrote: post #99
technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

Since I am an Indian I know of a service which is hated by most people - IRCTC because of it's scaling issues. It's a lot better now but I remember those horrible days you have to do a tatkal ticket. I also know of some government services (my state has paperless administration so everything is digital) which are used in day to day basis (but only hundreds of users) by government employees goes down for almost 2 hour…

In case of government, it's often another issue. While the social network of the parent was probably a lean stack, the government one was probably over-engineered. Some webapps are front-ends that communicate with a back-end in XML, with massive flaws like serialization and lack of transactions, and the backend syncs with a business solution backed by dBase, all of that for 11 web pages (I know, I've been on govt apps before – France). Scaling was a major requirement from the beginning, but those engineers just don't know what they're doing and follow the XML frontend/ESB bus patterns, assuming that's what gives performance. And committee design (12 to 24 ppl) does the rest of the bloat. 11 screens, $2m, sluggish result.

Had they started with only $300 like the parent comment, their webapp would have often performed much better. Big projects are a difficult curse, our most difficult question in IT engineering.

Re: Does it scale? Who cares (2011)

#154

Post by qume »

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

I built a service (~10mm users at peak) which was designed from the ground up to scale, around 2002. When the numbers did grow we just sat back and watched it pretty much.

Even given this, I completely agree with you. That's why I now develop in ruby. I'll take developer productivity over performance any day.

It's the old saying - 'nice problem to have'.

Re: Does it scale? Who cares (2011)

#155

Post by [deleted] »

Twisell wrote: post #147
cyberferret wrote: post #52

Earlier quoted context omitted.

> I know of absolutely no service which failed because it couldn't scale. I would say this is because of the simple fact of visibility and adoption. You've probably never heard of these services probably because they ground to a halt with a mere 1000 users, so they never got mainstream enough to be recognised as a viable service. It is a bit like how no one remembers the dozens of people who failed to achieve sustain…

What you describe is more of a design flaw than a scaling issue. Thinking ahead that custom fields could be needed in the future is a design choice and can apply to 5k users or 500 000k users. It's not the scaling itself that causing you pain, it just exacerbate the difficulty of a bad design choice. So advice of OP still apply, at start you should take most of your time develloping the best design for your applicati…

[deleted]

Re: Does it scale? Who cares (2011)

#156

Post by shadowmint »

Trundle wrote: post #141
shadowmint wrote: post #138

I care. It's easy to brush off scaling concerns as not important, but I've had personal experience where it's mattered, and if you want a high profile example, look at twitter. Yes, premature optimization is a bad thing, and so is over engineering; but that's easy to say if you have the experience to make the right initial choices that mean you have a meaningful path forward to scale when you do need it. For example,…

The article is about choosing a business idea where technical scale isn't important. Twitter would fall under a "media play" which is what the author is telling the reader not to do. They aren't saying "try to build twitter but skimp on the tech" they're saying "don't try to build twitter, build something where people pay you money to use it and you'll be raking in millions in profit by the time technical scaling is…

    The article is about choosing a business idea where technical scale isn't important.
Those sorts of business ideas don't exist, unless you don't have any customers. :)

Re: Does it scale? Who cares (2011)

#158

Post by kiriakasis »

bedros wrote: post #109
cyberferret wrote: post #52

Earlier quoted context omitted.

> I know of absolutely no service which failed because it couldn't scale. I would say this is because of the simple fact of visibility and adoption. You've probably never heard of these services probably because they ground to a halt with a mere 1000 users, so they never got mainstream enough to be recognised as a viable service. It is a bit like how no one remembers the dozens of people who failed to achieve sustain…

myspace.com failed because it could not scale I personally switched to facebook for two reasons: got sick of repeated errors every time I browse myspace facebook has better album permissions, (myspace has none)

as another comment said these look like bad design not bad scaling.

Re: Does it scale? Who cares (2011)

#159

Post by andreapaiola »

seanp2k2 wrote: post #134
grey-area wrote: post #88

Earlier quoted context omitted.

I am willing to bet that 9 out of 10 'weekend projects' have had to be thrown out completely and redeveloped from scratch when the number of users became significant. 9 out of 10 weekend projects never get to a significant number of users.

Probably more like 99/100 or 999/1000 but yeah. Also, you can get pretty damn far with PHP and MySQL. Just look at Facebook. Just don't rely on Drupal or Wordpress to get you there and it'll be fine. KISS and solve the scaling issues as they come up. Far too many projects focus way too much on infrastructure and architecture and micro services instead of building something that solves a real problem for users. Focus…

Drupal is for scaling, learn the architecture and you see it.

Re: Does it scale? Who cares (2011)

#160

Post by etnos »

Mhee, We built a rest API with Laravel, after a heated conversation with a junior dev about PHP-Laravel being slow.

+20k petitions average a day no problem at all. Digital ocean bill is like 20dlls/month ; )

Kids these days need to stop watching "The Social Network", chill out Mr. next Zuckerberg, you ain't scaling anything but a headache

Post reply on HN