Live data from Hacker News

A lot of complex “scalable” systems can be done with a simple, single C++ server

twitter.com

141–150 of 376 posts

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#141
This is precisely the point made by McSherry, Isard and Murray in their lovely paper, "Scalability! But at what COST?" (Usenix HotOS '15). They demonstrate how much performance headroom there is in modern CPU and memory, and show how simple cache-sensitive batch algorithms running on a single core can outperform hundreds of cores running distributed map-reduce style jobs.

https://www.usenix.org/system/files/conference/hotos15/hotos...

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#142
post #105
post #96

Earlier quoted context omitted.

There is a significant amount of question submission, commenting and voting going on on SO.

A slim subset of humans typing things is not what I would label "write heavy". Write heavy is more like 100k+ devices out in the field sending their current position every 10s. That's still very manageable, but requires some thoughtful design.

The Raspberry Pi 3 in my cupboard without a heat sink on can do 80000 Redis write ops/sec.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#143
post #114

Earlier quoted context omitted.

If you are writing C++, any field! Just use std::shared_ptr and std::unique_ptr from the standard library, along with std::make_shared and std::make_unique.

This is the quickest way to kill your performance in C++. I guarantee it wasn’t what Carmack was talking about. I used sharedptr extensively in a game engine. Whoops: suddenly 20% of the frame time was gone, never to be recovered. Once that performance is gone it’s almost impossible to get it back, short of rewriting every system.

[deleted]

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#144
post #117
post #112

Earlier quoted context omitted.

This sounds like an architecture problem, not a language problem. Can you elaborate?

Of course it's possible to write a horizontally scaled application in Java or C++. But once you have to deal with horizontal scaling anyway , language performance is much less of an advantage: as Carmack says, the difference between 100 servers and 10 is just accounting.

The Internet uses on the order of 10% of the world's electricity. A 10x difference is huge.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#145
post #84
post #2

I've been programming C++ and assembly for 23 years. Few years ago I became a huge fan of Python. In my opinion Python is amazingly well suited for rapid first revision and can then be swapped out for C++ / asm.

This is fine as long as you can convince management to spend the money to rewrite your software. That's usually a hard sell though. In my experience this plan usually ends up with a python monstrosity that everyone hates but is forced to deal with forever.

I'm speaking in terms of the reality of what is effective in development. Not in terms of what management at particular companies will approve.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#146
post #92

Earlier quoted context omitted.

At the risk of exposing my ignorance - look at all those "Peak 5%-20%" labels. Doesn't that mean they have a lot more than they need?

Stack overflow hosts all (most?) Of their own baremetal servers in their own data center. Looking at the specs of the machines, they are actually pretty basic as far as servers go. A server isn't barely worth the cost of it's chassis and motherboard if you put less than 64 G ram and 24 CPUs in it. In other words, these are about the lowest specd proper servers you can get. So yeah, even their modest hardware is still…

"Their own data center" implies (to me at least) that they built their own data center. That doesn't sound right, so I looked it up, and it seems like they're colocating. That might be what you meant, but the data centers they use certainly weren't built or owned by SO.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#147
post #114

Earlier quoted context omitted.

Out of curiosity, what is the field for which you find this to be true?

If you are writing C++, any field! Just use std::shared_ptr and std::unique_ptr from the standard library, along with std::make_shared and std::make_unique.

And, use unique pointers where possible. I see a lot of code using shared ptr where unique would suffice, even in my own code.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#148

This is precisely the point made by McSherry, Isard and Murray in their lovely paper, "Scalability! But at what COST?" (Usenix HotOS '15). They demonstrate how much performance headroom there is in modern CPU and memory, and show how simple cache-sensitive batch algorithms running on a single core can outperform hundreds of cores running distributed map-reduce style jobs. https://www.usenix.org/system/files/conferenc…

But nobody in BigCo(tm) would do that, because everyone (including the executives) involved want to have Hadoop/BigData(tm) in their resume. /sarcasm

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#149
post #58
post #52

vibe.d is well worth a look: https://vibed.org/

Isn't D dead practically? https://news.ycombinator.com/item?id=21902953

I have watched the ocaml community go through a renaissance when for a while it looked like it was moribund, and the D community looks like it is developing the same undercurrent of momentum. given that D is a great language and the implementation looks solid, I don't think it is in danger of dying.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#150

Earlier quoted context omitted.

One thing to keep in mind that their work-load is very ready-heavy which eases things a lot when scaling the system. The same is true for Wikipedia. Scaling a write-heavy workload is way more complex than scaling a read-heavy workload.

SO content changes all the time. Votes, comments, moderation, edits, tagging, search and recommendations, etc. There are also real-time community features. It's not as simple as it seems.

They said read heavy, not read only.

I guarantee you the bulk of traffic to SO is hitting a page and performing zero writes.

Post reply on HN