Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does it scale? Who cares (2011)

jacquesmattheij.com

Post reply on HN
Print view View original

201–210 of 285 posts

Re: Does it scale? Who cares (2011)

#201

Post by MusaTheRedGuard »

timewarrior wrote: post #123
jssmith wrote: post #114

Earlier quoted context omitted.

Friendster definitely failed because they couldn't pull off scale. I was CTO of the Tagged social network at the time and the issue loomed large during our 2005 fundraising. Investors had seen $50m go down the drain when Friendster got slow. Despite network effects, users moved over to products that were up and running (mostly MySpace but also hi5, etc.). I seem to recall that the Friendster team had more than one ab…

Thanks for sharing your experience. There is always scope for a botched execution or just pure bad luck. I wish someone shared more candid details of what happened at Friendster.

The Startup podcast[1] has an upcoming series about Friendster

1 - https://gimletmedia.com/startup/

Re: Does it scale? Who cares (2011)

#202

Post by user5994461 »

addicted wrote: post #132

Healthcare.gov was a site that failed and suffered due to scaling issues. However, it was an anomaly because unlike a product someone this article is intended towards would be building, that site had an immediate audience of millions of users from the get go. Also, the fact that it took a few weeks to be rewritten to handle the load at which point it became extremely successful, strengthens the original article's poi…

This is not an anomaly.

It's the norm for all government projects, and most projects started in established companies that already have millions of users.

To be working in these domains, the "doesn't need to scale" mentality" is very inappropriate and I find it to be doing a lot of damages.

The guys at healthcare handled most the scaling problems, accounting for the means at their disposal and the time frame they had, judging by what was published in the news. That it took a few weeks to smooth showed that they handled scaling beforehand. A launch from 10 users to 10 million is always a bit bumpy for the initial weeks.

Re: Does it scale? Who cares (2011)

#203

Post by PakG1 »

user5994461 wrote: post #187
kiriakasis wrote: post #157

Earlier quoted context omitted.

How people perceive it is not relevant if it actually is a side business

How people perceive it may be impacting your main business.

Only they would have the data to be able to determine that, if anyone would have that data at all.

Re: Does it scale? Who cares (2011)

#204

Post by nemothekid »

acdha wrote: post #189
grey-area wrote: post #118

Earlier quoted context omitted.

As a counterpoint stack overflow is hosted on windows and did just fine. If your service is printing money you can easily fix tech problems (e.g. Facebook effectively rewriting php), if it's not you may flail around trying rewrites etc but the real problem lies elsewhere. Tech people are quick to find technical reasons for failure but the reasons are usually elsewhere.

Stack Overflow seems very relevant to this conversation given how unremarkable their architecture is – their level of traffic is comfortably running on 4 MS SQL Server boxes: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... I've seen too many breathless posts which would have you believe they'd need a clustered NoSQL database or it wouldn't scale.

>believe they'd need a clustered NoSQL database or it wouldn't scale.

While for many developers, NoSQL might be overkill - Stackoverflow is a bad example. If you were any fast growing startup in the cloud, and you wanted to go the SO route it would have meant going CoLo. SO has set of machines with a nearly a terabyte of RAM - GCE doesn't even offer cloud machines with the same specs.

And even then their setup is far more fiscally expensive than something you could get done with 20 cloud nodes on some NoSQL solution.

Re: Does it scale? Who cares (2011)

#205

Post by franciscop »

user5994461 wrote: post #188
simonh wrote: post #185

Earlier quoted context omitted.

Having worked on corporate Python platform with tens of thousands of developers and hundreds of applications running hubdress of thousands of instances on a many million line common core code base, I beg to differ. We were sometimes able to go from concept to deployed application in a single day.

Having worked on corporate Python platform with tens of thousands of developers and thousands of applications running on thousands of instances on millions of lines common core code base, I beg to differ. It is easy to ship any proof of concept. Is it hard to maintain, it is rigged with performance issues and it is almost impossible to refactor because python has zero refactoring support and no compiler to help you c…

But that is basically the point of the whole article and this thread, that it's great for proof of concept that morphs into production when needed. If you can deploy in one day that means you can test 5 ideas/week until one catches on (hopefully not that long). Once one catches on, you can actually put more effort into it and optimize it as needed and not the other way around.

Re: Does it scale? Who cares (2011)

#206

Post by bpicolo »

developer2 wrote: post #194
technosamay20 wrote: post #59

Earlier quoted context omitted.

I mentioned something similar in a separate comment. My initial DB schema was pretty bad. We did at 2 schema rewrites and migrations from the launch to 5M users. Each time it took 2 weeks of sleep less nights. The machines today are really powerful. You can do a lot with 244 GB RAM machines backed by SSD. Someone who doesn't have the skill set to be able to scale once they get traction - it's likely they will not hav…

>> You can do a lot with 244 GB RAM machines Is this a typo of "244 GB" instead of "24 GB"? Nearly any company that has a single machine provisioned with 244 GB of RAM is doing something severely wrong, likely putting the company's ability to grow at risk. Such a machine screams of trying to vertically scale a poorly performing legacy product instead of figuring out to horizontally scale out with 16-64 GB servers. Th…

Pretty sure it's not a typo. Amazon i3.16xlarge is 488 GB of Ram, 64 vCPUs, and a 20 Gigabit connection. Postgres/Mysql can absolutely scream perf-wise on that.

Re: Does it scale? Who cares (2011)

#207

Post by acdha »

user5994461 wrote: post #195
acdha wrote: post #189

Earlier quoted context omitted.

Stack Overflow seems very relevant to this conversation given how unremarkable their architecture is – their level of traffic is comfortably running on 4 MS SQL Server boxes: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... I've seen too many breathless posts which would have you believe they'd need a clustered NoSQL database or it wouldn't scale.

>>> how unremarkable their architecture is – their level of traffic is comfortably running on 4 MS SQL Server boxes: Wrong, and wrong. First, their architecture is extremely remarkable. They are doing and mastering vertical scaling, down to every little details. Terabytes of RAM, C# instead of Ruby/Python, FusionIO drives, MS SQL instead of MySQL/PostGre, etc... Second, that's at least 6 database servers, each one be…

I think you misunderstand my use of unremarkable. It's not wrong or bad but key parts are the same stack you'd have picked 1-2 decades ago – well implemented, values scaled up, yes, but something you could have shown people in 2000 and not had to explain more than that the hardware costs so much less.

I think that's good: SQL databases are very mature and you don't want to be exciting for your core business data if you don't get some major benefit to defray the cost. Boring is a delightful characteristic for data storage.

Re: Does it scale? Who cares (2011)

#208

Post by acdha »

nemothekid wrote: post #204
acdha wrote: post #189

Earlier quoted context omitted.

Stack Overflow seems very relevant to this conversation given how unremarkable their architecture is – their level of traffic is comfortably running on 4 MS SQL Server boxes: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar... I've seen too many breathless posts which would have you believe they'd need a clustered NoSQL database or it wouldn't scale.

>believe they'd need a clustered NoSQL database or it wouldn't scale. While for many developers, NoSQL might be overkill - Stackoverflow is a bad example. If you were any fast growing startup in the cloud, and you wanted to go the SO route it would have meant going CoLo. SO has set of machines with a nearly a terabyte of RAM - GCE doesn't even offer cloud machines with the same specs. And even then their setup is far…

One: run the numbers on the dozens of cloud nodes you mentioned. Is that really cheaper than renting some servers in a colo?

Second: how much time would they have spent rolling all of the data integrity, reporting, etc. features they'd have needed to add. I'm inclined to take them at their word when they say this was safer and cheaper given their resources.

Re: Does it scale? Who cares (2011)

#209

Post by stephenhuey »

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

I love this example. Thanks for sharing! It's very encouraging to remind me to just make something and get it out there.

One prominent counterexample that comes to mind is Friendster which was a big social network before MySpace and Facebook. They had terrible performance issues but to be fair to your point, their decline was more mismanagement than a true inability to scale.

https://www.inc.com/magazine/20070601/features-how-to-kill-a...

https://en.wikipedia.org/wiki/Friendster

Re: Does it scale? Who cares (2011)

#210

Post by user5994461 »

acdha wrote: post #207
user5994461 wrote: post #195

Earlier quoted context omitted.

>>> how unremarkable their architecture is – their level of traffic is comfortably running on 4 MS SQL Server boxes: Wrong, and wrong. First, their architecture is extremely remarkable. They are doing and mastering vertical scaling, down to every little details. Terabytes of RAM, C# instead of Ruby/Python, FusionIO drives, MS SQL instead of MySQL/PostGre, etc... Second, that's at least 6 database servers, each one be…

I think you misunderstand my use of unremarkable. It's not wrong or bad but key parts are the same stack you'd have picked 1-2 decades ago – well implemented, values scaled up, yes, but something you could have shown people in 2000 and not had to explain more than that the hardware costs so much less. I think that's good: SQL databases are very mature and you don't want to be exciting for your core business data if y…

My bad then. I think that using a proven technology perfectly tuned and the way it's meant to be done, is remarkable. This should be promoted more, instead of the new and shiny.

I'm not sure you could get 1TB of memory and multi TB SSD drives anywhere in the 2000's, even for a million dollar. That makes a major difference in the ability to scale up. Data didn't grow, storing 1M user account always took the same space.

Post reply on HN