Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A lot of complex “scalable” systems can be done with a simple, single C++ server

twitter.com

Post reply on HN
Print view View original

111–120 of 376 posts

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#111

Post by lmm »

Horizontal scalability carries a lot of overhead. Probably a factor of 10, easily. But the clue is in the name: eventually you'll get to a point where you have to scale.

Back in 2010 I worked for a company whose system, in Java, ran on a single web server (with one identical machine for failover). We laughed at our nearest rivals, who were using Ruby, and apparently needed 60(!) machines to run their system, which had about 5x the average request latency of ours.

Then traffic doubled, and suddenly we were having to buy six-figure servers and carefully performance-optimize our code, and our rivals with the 120 Ruby servers didn't look so funny any more. And then traffic doubled again.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#112

Post by steev »

lmm wrote: post #111

Horizontal scalability carries a lot of overhead. Probably a factor of 10, easily. But the clue is in the name: eventually you'll get to a point where you have to scale. Back in 2010 I worked for a company whose system, in Java, ran on a single web server (with one identical machine for failover). We laughed at our nearest rivals, who were using Ruby, and apparently needed 60(!) machines to run their system, which ha…

This sounds like an architecture problem, not a language problem. Can you elaborate?

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#113

Post by pnako »

8fingerlouie wrote: post #28

As someone who has implemented a complex system in C++ in this decade, I’d say he’s not wrong, but you need to carefully weight the pros and cons. In our case latency and real time demands mattered a lot (NASDAQ feed parser), to the point of the (potential) slowdown of a garbage collector kicking in was enough to rule out Java and .NET. It runs entirely in memory and on 64+ cores. We implemented our own reference cou…

I've seen state-of-the-art low-latency trading software written in C# or Java.

You should not be doing any memory allocation in the critical path anyway, so garbage collection is hardly a relevant factor.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#114

Post by wffurr »

stevenwoo wrote: post #101
sumeno wrote: post #78

Earlier quoted context omitted.

Memory management in modern C++ is considerably easier than it used to be. Bespoke memory managers aren't really needed, you can do almost anything you need to without ever using new or delete.

Out of curiosity, what is the field for which you find this to be true?

If you are writing C++, any field! Just use std::shared_ptr and std::unique_ptr from the standard library, along with std::make_shared and std::make_unique.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#115

Post by bcrosby95 »

lmm wrote: post #111

Horizontal scalability carries a lot of overhead. Probably a factor of 10, easily. But the clue is in the name: eventually you'll get to a point where you have to scale. Back in 2010 I worked for a company whose system, in Java, ran on a single web server (with one identical machine for failover). We laughed at our nearest rivals, who were using Ruby, and apparently needed 60(!) machines to run their system, which ha…

Horizontally scaling a monolith web app has basically zero overhead. Both in app design and production complexity. But it can go decently far.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#116

Post by Silhouette »

lmm wrote: post #111

Horizontal scalability carries a lot of overhead. Probably a factor of 10, easily. But the clue is in the name: eventually you'll get to a point where you have to scale. Back in 2010 I worked for a company whose system, in Java, ran on a single web server (with one identical machine for failover). We laughed at our nearest rivals, who were using Ruby, and apparently needed 60(!) machines to run their system, which ha…

But the clue is in the name: eventually you'll get to a point where you have to scale.

Isn't that the conceit underlying this whole argument, though? Many systems won't ever get to the point where you have to scale in that way, if you build them efficiently in the first place.

Perhaps more importantly, for many applications, you'll be able to see the limit coming some way ahead, and if you've reached a size where you do need a fundamental restructuring in order to start scaling horizontally, that's going to be a nice problem to have and you'll also have the resources to do it.

I know several online systems that are handling significant traffic volumes perfectly well on a simple, single-server basis. They don't get bogged down in infrastructure and tooling issues, ever. They don't get confused by complicated cloud hosting issues, ever. They are free to spend almost their entire development budget on actually developing useful functionality, which is like a breath of fresh air in today's dev culture.

Obviously in the more serious cases they probably also have some redundancy for backup/failover purposes, but even that is simple and if necessary can probably be handled manually when you only have a handful of servers to manage. Here I do slightly disagree with one of Carmack's later tweets, in that I would argue 100 servers instead of 1000 is just accounting, but 100 down to 10 or fewer is also more of a qualitative change (albeit not exactly the same qualitative change as 10 down to 1).

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#117

Post by lmm »

steev wrote: post #112
lmm wrote: post #111

Horizontal scalability carries a lot of overhead. Probably a factor of 10, easily. But the clue is in the name: eventually you'll get to a point where you have to scale. Back in 2010 I worked for a company whose system, in Java, ran on a single web server (with one identical machine for failover). We laughed at our nearest rivals, who were using Ruby, and apparently needed 60(!) machines to run their system, which ha…

This sounds like an architecture problem, not a language problem. Can you elaborate?

Of course it's possible to write a horizontally scaled application in Java or C++. But once you have to deal with horizontal scaling anyway, language performance is much less of an advantage: as Carmack says, the difference between 100 servers and 10 is just accounting.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#118

Post by pnako »

avip wrote: post #79
ginko wrote: post #73

Earlier quoted context omitted.

That’s not exactly what I would call good speedup.

The point of the code is not to speedup the execution of summing a list of random number, but rather to speedup the acknowledgement of N random python developers that they have some misconceptions about the GIL. I think it does that pretty well but, well, that's just like my opinion.

I'm a casual Python programmer (just bit of scripting) and I had no particular misconception about the GIL; I don't care because I write single-threaded cookie cutter scripts.

Your example I think is demonstrating the opposite of what you want to show. Those figures are atrocious.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#119

Post by pnako »

postpawl wrote: post #61
ummonk wrote: post #50

Earlier quoted context omitted.

From what I’ve seen in production environments, Python has an uncanny ability to take tasks that should be IO- / server- bound and make them CPU-bound.

For sure, but often that’s a sign that you’re doing it wrong. Maybe those aggregations should be done in the database, maybe it should be cached, maybe you should do that in bulk with one request, maybe it should run in the background anyway, etc.

i.e. "maybe do it anywhere else than in Python"

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#120

Post by Silhouette »

twic wrote: post #59

What's that, Frank McSherry? For horizontally scaled systems, we should ask what the Configuration that Outperforms a Single Thread is? https://blog.acolyer.org/2015/06/05/scalability-but-at-what-...

IMO, the linked article is much more insightful than the flippant comment here might suggest. It's a solid argument, backed by real world data, about how easy it is to make bad assumptions equating better scalability with better performance.
Post reply on HN