Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does it scale? Who cares (2011)

jacquesmattheij.com

Post reply on HN
Print view View original

261–270 of 285 posts

Re: Does it scale? Who cares (2011)

#261

Post by acdha »

user5994461 wrote: post #210
acdha wrote: post #207

Earlier quoted context omitted.

I think you misunderstand my use of unremarkable. It's not wrong or bad but key parts are the same stack you'd have picked 1-2 decades ago – well implemented, values scaled up, yes, but something you could have shown people in 2000 and not had to explain more than that the hardware costs so much less. I think that's good: SQL databases are very mature and you don't want to be exciting for your core business data if y…

My bad then. I think that using a proven technology perfectly tuned and the way it's meant to be done, is remarkable. This should be promoted more, instead of the new and shiny. I'm not sure you could get 1TB of memory and multi TB SSD drives anywhere in the 2000's, even for a million dollar. That makes a major difference in the ability to scale up. Data didn't grow, storing 1M user account always took the same space…

Agreed that data sizes would have been far more of a challenge – far more people used clustered services just because a single box only supported so much RAM, RAID arrays for IOPs and size as well as redundancy, etc. – so it's arguably become much easier for a growing chunk of the industry.

Re: Does it scale? Who cares (2011)

#262

Post by bedros »

kiriakasis wrote: post #158
bedros wrote: post #109

Earlier quoted context omitted.

myspace.com failed because it could not scale I personally switched to facebook for two reasons: got sick of repeated errors every time I browse myspace facebook has better album permissions, (myspace has none)

as another comment said these look like bad design not bad scaling.

Many problems including not filtering user input and comments for CSS, JS, etc.

But main problem, they crumbled under heavy traffic

Re: Does it scale? Who cares (2011)

#263

Post by nickpsecurity »

inetknght wrote: post #247
nickpsecurity wrote: post #243

Earlier quoted context omitted.

Facebook deploys thousands a week: https://swreflections.blogspot.com/2013/09/this-is-how-faceb...

If you're looking to Facebook for your user experience guidelines, you're going to end up angering a lot of actual users.

While making millions to billions dominating a market I own. So, sure. Other circumstances it would just teach you can experiment more than a few things a week.

Re: Does it scale? Who cares (2011)

#264

Post by nickpsecurity »

user5994461 wrote: post #256
nickpsecurity wrote: post #243

Earlier quoted context omitted.

Facebook deploys thousands a week: https://swreflections.blogspot.com/2013/09/this-is-how-faceb...

Facebook have thousands of products and billions of users. You do not ;)

Now that's a better point. Obviously, the amount of changes are tied to company's circumstances.

Re: Does it scale? Who cares (2011)

#265

Post by luord »

user5994461 wrote: post #251
luord wrote: post #218

Earlier quoted context omitted.

> You can always push a python/ruby app an order of magnitude by putting an order of magnitude more AWS instances. Or a Java app or a Go app. Really, if one's working in a domain where the language would become the bottleneck, one deliberately screwed up by going against the grain because that language is little used in that domain. For almost everything else or with very specific exceptions, something else is the bo…

Java or Go are easily 10 times faster than Python/Rail on the same hardware. (Potentially much more, especially when you have non trivial logic to process in the code). That's a recurrent issue with API code (that usually has to be somewhat fast). Not so much for frontend generation. P.S. Sorry but hundreds of customer is nothing. That's served by a pair of boxes (DB + webserver) irrelevant of the language.

> Java or Go are easily 10 times faster than Python/Rail on the same hardware. (Potentially much more, especially when you have non trivial logic to process in the code).

... I know that, everyone here knows that; saying it is little more than stating the obvious. Evidently, you missed my point so I fold; but once more, JIC: if an application is slow, one of the last things one should look into to improve performance is the language; it rarely is the bottleneck.

P.S.: I know it's nothing, all anecdotal evidence is nothing. For every anecdote one could give, anyone could give counterexamples. That was another point I tried to make. Cheers.

Re: Does it scale? Who cares (2011)

#266

Post by shadowmint »

luord wrote: post #265
user5994461 wrote: post #251

Earlier quoted context omitted.

Java or Go are easily 10 times faster than Python/Rail on the same hardware. (Potentially much more, especially when you have non trivial logic to process in the code). That's a recurrent issue with API code (that usually has to be somewhat fast). Not so much for frontend generation. P.S. Sorry but hundreds of customer is nothing. That's served by a pair of boxes (DB + webserver) irrelevant of the language.

> Java or Go are easily 10 times faster than Python/Rail on the same hardware. (Potentially much more, especially when you have non trivial logic to process in the code). ... I know that, everyone here knows that ; saying it is little more than stating the obvious. Evidently, you missed my point so I fold; but once more, JIC: if an application is slow, one of the last things one should look into to improve performanc…

> it rarely is the bottleneck.

If you're using python, it can be the bottle neck quite easily.

I don't endorse change language as an optimisation; that's just ridiculous.

...but you might look at splitting your application up into parts, and doing some service in a more suitable language; or, up front realizing that you have a heavy data processing workload you need to do in parallel, and python isn't a good choice for it.

Maybe you're right; you can look at optimisations that patch over the problem with queries and so forth as a first pass; but in some cases your choice of language (specifically node and python in my experience) are actually fundamentally the performance problem (but to be fair, not always).

...but basically, if you don't address the root cause of your perf issues (whatever they are), you're going to be patching and firefighting forever.

Re: Does it scale? Who cares (2011)

#267

Post by luord »

shadowmint wrote: post #266
luord wrote: post #265

Earlier quoted context omitted.

> Java or Go are easily 10 times faster than Python/Rail on the same hardware. (Potentially much more, especially when you have non trivial logic to process in the code). ... I know that, everyone here knows that ; saying it is little more than stating the obvious. Evidently, you missed my point so I fold; but once more, JIC: if an application is slow, one of the last things one should look into to improve performanc…

> it rarely is the bottleneck. If you're using python, it can be the bottle neck quite easily. I don't endorse change language as an optimisation; that's just ridiculous. ...but you might look at splitting your application up into parts, and doing some service in a more suitable language; or, up front realizing that you have a heavy data processing workload you need to do in parallel, and python isn't a good choice f…

> If you're using python, it can be the bottle neck

> data processing workload you need to do in parallel

It's funny how often those two go together, you're of course correct (for now). I never wrote that the language couldn't be the bottleneck, though.

If you are doing something that requires heavy parallelization then, by all means, don't use python or replace it if you're already using it... But that kind of workloads isn't actually a common (i.e. 50% or more of all applications) scenario.

> ...but basically, if you don't address the root cause of your perf issues (whatever they are), you're going to be patching and firefighting forever.

Yes, that's what I'm saying: Find the bottleneck and solve that. I was only addressing the, in my opinion, undue focusing on languages of the comment I replied to.

> quite easily

Only if one is working with poor developers. And this is true for all languages.

Re: Does it scale? Who cares (2011)

#268

Post by shadowmint »

luord wrote: post #267
shadowmint wrote: post #266

Earlier quoted context omitted.

> it rarely is the bottleneck. If you're using python, it can be the bottle neck quite easily. I don't endorse change language as an optimisation; that's just ridiculous. ...but you might look at splitting your application up into parts, and doing some service in a more suitable language; or, up front realizing that you have a heavy data processing workload you need to do in parallel, and python isn't a good choice f…

> If you're using python, it can be the bottle neck > data processing workload you need to do in parallel It's funny how often those two go together, you're of course correct (for now). I never wrote that the language couldn't be the bottleneck, though. If you are doing something that requires heavy parallelization then, by all means, don't use python or replace it if you're already using it... But that kind of workl…

> Only if one is working with poor developers

It's not about the developers, it's about the workload.

Objectively, you can't write high performance multi-threaded python. No one can; it's not possible; it's just slow.

If you're rewriting your code in C++ so its not slow and pretending its python, you should just rewrite your code in C++. That's not writing fast python, it's writing C++.

/shrug

So python can easily be a bottle neck, regardless of how good your developers are, if you've picked it for a poor purpose:

That's my point: Don't pick the wrong language for your task in the first place. ...and specifically python is the wrong choice for certain types of heavy lifting.

Re: Does it scale? Who cares (2011)

#269

Post by flukus »

shadowmint wrote: post #138

I care. It's easy to brush off scaling concerns as not important, but I've had personal experience where it's mattered, and if you want a high profile example, look at twitter. Yes, premature optimization is a bad thing, and so is over engineering; but that's easy to say if you have the experience to make the right initial choices that mean you have a meaningful path forward to scale when you do need it. For example,…

Wouldn't twitter be a counter-example? They had some issues but resolved them later when the had the resources to and they are now a multi-billion dollar company. That may not have happened had the devoted resources into scaling from the outset.

Re: Does it scale? Who cares (2011)

#270

Post by flukus »

devduderino wrote: post #11

I care because it usually goes like this: Product manager > "Niche sass app {x} will never need to support more than 10-20 users" Two weeks after launch > "We have 10k users and counting, why didn't you architect this for scale?" Always assume you underestimated the scope of the project.

Everywhere I've worked that tried to architect for massive scales at the beginning has ended up with an architecture that inhibits scaling. Using correct indexes and transactions can go a long way.
Post reply on HN