Live data from Hacker News

Does it scale? Who cares (2011)

jacquesmattheij.com

131–140 of 285 posts

Re: Does it scale? Who cares (2011)

#131
Fundamentally disagree with the ethos of this article. What about ambition? What happened to that?

> You won’t make the cover of Time Magazine and you won’t be ordering a private jet but that Ferrari is a definite possibility, if that’s the thing you are hurting for. (college education for your kids is probably a better idea ;) ).

I'd like to be on the cover of fortune or as Russ Hanneman might say, "I wanna make a fuckton of money all at once".

I don't see any problem with being ambitious or wanting that private jet.

Re: Does it scale? Who cares (2011)

#132
Healthcare.gov was a site that failed and suffered due to scaling issues.

However, it was an anomaly because unlike a product someone this article is intended towards would be building, that site had an immediate audience of millions of users from the get go.

Also, the fact that it took a few weeks to be rewritten to handle the load at which point it became extremely successful, strengthens the original article's point.

By the time scalability becomes a problem, you will have enough resources to tackle the scalability problem.

Re: Does it scale? Who cares (2011)

#133

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

> I know of absolutely no service which failed because it couldn't scale. Bullshit. What about the site (voat something?) that tried to be an alternative to reddit when there was some scandal there but couldn't keep users because they kept crashing?

Bullshit. You can go on voat right now just fine and there are people using it just fine. It hasn't failed, it's just not a big deal because it's an almost exact replica of a site that already exists and has traction. The difference being one allows you to hate on fat people all the time and the other only lets you hate on fat people most of the time. It's just too niche of a USP for most people.

Re: Does it scale? Who cares (2011)

#134

Earlier quoted context omitted.

> I know of absolutely no service which failed because it couldn't scale. I would say this is because of the simple fact of visibility and adoption. You've probably never heard of these services probably because they ground to a halt with a mere 1000 users, so they never got mainstream enough to be recognised as a viable service. It is a bit like how no one remembers the dozens of people who failed to achieve sustain…

I am willing to bet that 9 out of 10 'weekend projects' have had to be thrown out completely and redeveloped from scratch when the number of users became significant. 9 out of 10 weekend projects never get to a significant number of users.

Probably more like 99/100 or 999/1000 but yeah. Also, you can get pretty damn far with PHP and MySQL. Just look at Facebook. Just don't rely on Drupal or Wordpress to get you there and it'll be fine. KISS and solve the scaling issues as they come up. Far too many projects focus way too much on infrastructure and architecture and micro services instead of building something that solves a real problem for users. Focus on that first. It doesn't matter if the tech behind it is a bunch of bash scripts if it does something that is really useful to a lot of people.

Re: Does it scale? Who cares (2011)

#135
post #122

I'm in the unfortunate position where this question actually matters from day 1. I learnt the hard way a few days ago when I hit the bottleneck (about 50-100 concurrent users) and I'm not sure how to proceed. It's a multiplayer drawing site built with Node.js/socket.io. I'm already on the biggest Heroku dyno my budget can allow and it's too big of a task to rewrite the back end to support load balancing (and I wouldn…

Heroku is fairly expensive. Could you not port this to a more dedicated server? Yes, it's likely complicates deployment but for a few hours of work in initial deployment, you can save hundreds of hours rewriting your codebase.

Re: Does it scale? Who cares (2011)

#136
post #122

I'm in the unfortunate position where this question actually matters from day 1. I learnt the hard way a few days ago when I hit the bottleneck (about 50-100 concurrent users) and I'm not sure how to proceed. It's a multiplayer drawing site built with Node.js/socket.io. I'm already on the biggest Heroku dyno my budget can allow and it's too big of a task to rewrite the back end to support load balancing (and I wouldn…

Maybe you could rely on Firebase for the real-time db stuff and do the drawing there.

Re: Does it scale? Who cares (2011)

#137
post #38

Earlier quoted context omitted.

Can you go into a bit more detail? I've seen amplification referenced before. My estimation is that the database will be pretty much read heavy. I will need to split a given database across multiple instances.

https://blog.2ndquadrant.com/on-the-impact-of-full-page-writ... this is fairly detailed analysis of this issue

How about simply using non-random UUIDs?

Re: Does it scale? Who cares (2011)

#138
I care.

It's easy to brush off scaling concerns as not important, but I've had personal experience where it's mattered, and if you want a high profile example, look at twitter.

Yes, premature optimization is a bad thing, and so is over engineering; but that's easy to say if you have the experience to make the right initial choices that mean you have a meaningful path forward to scale when you do need it.

For example, lets say you build a typical business app and push something out quickly that doesn't say, log when it fails, or provide an auto-update mechanism, or have any remote access. Now you have it deployed at 50 locations and its 'not working' for some reason. Not only do you physically have to go out to see whats wrong, you have to organize a reinstall at 50 locations. Bad right? yes. It's very bad. (Or, you do a similar ruby or python app when your domain is something that involves bulk processing massive loads of data. It works fine and you have a great 'platform' until you have 3 users, and then it starts to slow down for everyone; and it turns out, you need a dedicated server for each customer because doing your business logic in a slow language works when you only need to do 10 items a second, not 10000. Bad right? yes. Very. Bad. (It's not premature optimization to not pick stupid technology choices for your domain, or ship prototypes.

...but sometimes you don't have someone on the team with the experience to realize that, and the push from management is to just get it out, and not worry about the details; but trust me, if you have someone who is sticking their neck out and go, hey wait, this isn't going to scale...

Maybe you should listen to what they have to say, not quote platitudes.

Ecommerce is probably one of those things where the domain is well enough known you can get away with it; heck, just throw away all your rubbish and use an off-the-shelf solution if you hit a problem; but I'm going to suggest that the majority of people aren't building that sort of platform, because its largely a solved problem.

Re: Does it scale? Who cares (2011)

#139
While a point the article tries to make (fix your leaky funnel before acquiring users) is true, I disagree with the article. If your application is converting well, scalability problems are not acceptable.

I have seen applications that convert very well, but were limited by scalability problems. That meant that the business had to hold on on marketing and user acquisition, missed their financial targets, and that cascaded into breaching contracts. The phrase that nobody wants to hear in that situation is "who cares about scalability".

Now, if you did not have a lot of problems scaling in your particular case, that just means it was not an obstacle for you. e.g: you had good intuition around performance/scalability, or the problem was coincidentally a good fit for your technological choices.

Unfortunately not everyone has a good intuition about scalability, not everyone is risk averse and not everyone is good at picking a good technology for their use case. So I disagree with this article in the sense that it is not in the best interest of a random reader to not care about scalability.

Re: Does it scale? Who cares (2011)

#140
post #122

I'm in the unfortunate position where this question actually matters from day 1. I learnt the hard way a few days ago when I hit the bottleneck (about 50-100 concurrent users) and I'm not sure how to proceed. It's a multiplayer drawing site built with Node.js/socket.io. I'm already on the biggest Heroku dyno my budget can allow and it's too big of a task to rewrite the back end to support load balancing (and I wouldn…

I think what the author of the original article would suggest that you need to convert some of the concurrent users to paying customers and use the money to purchase additional Heroku dynos.
Post reply on HN