Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A lot of complex “scalable” systems can be done with a simple, single C++ server

twitter.com

Post reply on HN
Print view View original

281–290 of 376 posts

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#281

Post by otabdeveloper4 »

filleokus wrote: post #276
otabdeveloper4 wrote: post #193

Earlier quoted context omitted.

No, it's not significant. Dev time depends on programmer skill, not the toolset. A good C++ programmer will develop your MVP many times faster than an average Python programmer. Python programmers are much easier to hire, though - you already need a good C++ programmer on the team to hire another one, because HR and corporate management can't into proper hiring process. This last factor is the overarching most import…

> Dev time depends on programmer skill, not the toolset. This is obviously not strictly true, always. A skilled programmer will use the proper tools for the job. If you for example is tasked with writing a backend service exposing a GraphQL API, I think it would be foolish to do this in C++, and would bet that the average Python programmer would do it quicker than even a top-tier C++ programmer (if the latter would b…

Two points:

a) Writing a schema parser is not rocket science. In fact, for a good programmer implementing their own GraphQL library would be quicker than integrating some third-party library. So your first point ("average Python programmer would do it quicker than even a top-tier C++ programmer") is absolutely wrong.

b) There's no value in an MVP that does something generic that is already available in off-the-shelf libraries. Your GraphQL example is pretty pointless because it doesn't actually do anything.

> ...the Python/Node/Go/etc developer would have a working GraphQL server up and running connected to a database of choice within an afternoon

Well, no. By the end of the week they'll still be arguing about which package manager to use, whether TDD is a good idea, what makes a microservice 'micro' and how to configure Kubernetes.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#282

Post by mannykannot »

bane wrote: post #275
jpz wrote: post #181

Earlier quoted context omitted.

Big data is about I/O not CPU I’m a C++ veteran btw and understand the point but big data is about how to process petabytes of I/O not how to consume CPU.

This is true in a "water is wet" kind of way. The point is that a great many problems that can fit neatly into a single machine are being turned into I/O problems by being distributed onto clusters. There's an incredible number of gigabyte and even terabyte scale problems that are consuming racks of blades when a little thinking and understanding of the problem being solved can be done pretty nicely on far fewer reso…

>...and end up shifting the complexity into cluster administration rather than programmer time.

Sometimes, that is the right thing to do. The problem is when the cluster solution also adds to the programmer-time complexity.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#283

Post by otabdeveloper4 »

stefano wrote: post #249
otabdeveloper4 wrote: post #193

Earlier quoted context omitted.

No, it's not significant. Dev time depends on programmer skill, not the toolset. A good C++ programmer will develop your MVP many times faster than an average Python programmer. Python programmers are much easier to hire, though - you already need a good C++ programmer on the team to hire another one, because HR and corporate management can't into proper hiring process. This last factor is the overarching most import…

The same argument can be made for Python (or for most non-trivial jobs, for that matter). You always need a good technical person to hire another one.

Not really; hiring Python and Java devs is very amenable to keyword-driven recruitment. Hiring C++ devs in the same manner is a clusterfuck waiting to happen.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#284

Post by bsenftner »

cthalupa wrote: post #225
bsenftner wrote: post #56

I'm 15 years in writing high performance Internet servers in C++, and I can confirm higher level languages provide an illusion of capability, but once you're talking high performance with high compute requirements and scaling your service, the cost efficiency of C++ is exponential better than any other language. The higher level language ecosystems are bloated beyond repair. I was able to use one 32-core physical ser…

The thing that worries me about stories like this is that there is frequently (as is the case here) no mention any sort of HA or backups. No details on what disaster recovery looks like. Those are business critical considerations that cost money, and just disappear from the discussion when people say "hey i saved all this money dropping everything down to a single server!"

Well, in my case described above, single server solutions include an automated backup sub-system, and my servers expect multiple instances of itself to running on the client network, and these multiple instances synchronize with one another via additional endpoints specific for the purpose. The whole issue of HA and backups is critical and one of the areas my approach shines.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#285

Post by 0x445442 »

tlb wrote: post #155
wiradikusuma wrote: post #148

Earlier quoted context omitted.

But nobody in BigCo(tm) would do that, because everyone (including the executives) involved want to have Hadoop/BigData(tm) in their resume. /sarcasm

No doubt there are people who do it for cynical reasons. But at least some people do it sincerely thinking it’s the right choice. It’d be more interesting to talk about them, and how they came to make the wrong decision for what they thought were the right reasona.

> It’d be more interesting to talk about them, and how they came to make the wrong decision for what they thought were the right reason

My guess, essentially holding their finger to the wind. Most orgs can't afford to do real engineering.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#286

Post by fock »

RhodesianHunter wrote: post #150
manigandham wrote: post #103

Earlier quoted context omitted.

SO content changes all the time. Votes, comments, moderation, edits, tagging, search and recommendations, etc. There are also real-time community features. It's not as simple as it seems.

They said read heavy, not read only. I guarantee you the bulk of traffic to SO is hitting a page and performing zero writes.

so, would you mind showing any non-FANG, but write heavy site? Besides that, even the operation of most social networks should be very easy to be split into town-sized instances with batch-updates in-between instances (see mastodon specs https://github.com/tootsuite/documentation/blob/master/Runni...). What actually costs a lot is constantly surveilling all user interactions, data-warehousing that and playing out ads. None of which is necessary, and a 20ct/per user/per month should cover the actual service nicely...

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#287

Post by 0x445442 »

FpUser wrote: post #243
IanCal wrote: post #227

Earlier quoted context omitted.

If you can set it up so it solves my problems, yes. I could manage my own server and fine tune everything, or I can throw it on snowflake. Snowflake means I've spent almost no time managing anything and it was costing less than an aws box running postgres but absolutely blew it out of the water performance wise. Depends on your workload but it's been perfect for one of my use cases. If they were just using postgres u…

I know nothing about Snowflake so can not really comment on particular case. However the generic statement that I hear often goes something like this: it is so much trouble to manage your own infrastructure and on cloud everything is done for you. Well I saw with my own eyes that having ones's infrastructure deployed on Azure keeps them quite busy anyways.

After a year of AWS I can tell you it's as much a PITA or more to manage all the components as it would have been to manage a few Spring Boot and MySQL Droplets.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#288

Post by OGWhales »

angrais wrote: post #239
buzzkillington wrote: post #216

Earlier quoted context omitted.

You have not had to interview the python devs I've had to interview. Last round, for the second best candidate who we hired at 130% the salary we initially offered, after the best candidate was snatched under our nose for what we were told was twice the salary we were offering: >"Last question, I saw that you wrote a 100 line function here, is this because you ran out of time?" >>"No. I don't like to break up my func…

That sounds more like someone unfamiliar with good practices generally than python-specific issues. Also ... Why'd you hire this person? Is there such a lack of "good" developers in your area?

maybe a lack of good developers for the salary they are offering...

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#289

Post by 0x445442 »

latch wrote: post #146
dahfizz wrote: post #92

Earlier quoted context omitted.

Stack overflow hosts all (most?) Of their own baremetal servers in their own data center. Looking at the specs of the machines, they are actually pretty basic as far as servers go. A server isn't barely worth the cost of it's chassis and motherboard if you put less than 64 G ram and 24 CPUs in it. In other words, these are about the lowest specd proper servers you can get. So yeah, even their modest hardware is still…

"Their own data center" implies (to me at least) that they built their own data center. That doesn't sound right, so I looked it up, and it seems like they're colocating. That might be what you meant, but the data centers they use certainly weren't built or owned by SO.

Yeah, I recall an article that said their physical hardware was in jeopardy during hurricane Sandy.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#290

Post by zozbot234 »

z0mbie42 wrote: post #230
jhgg wrote: post #186

Earlier quoted context omitted.

SO is really not a great example of a high traffic site. 5500 req/sec of mostly read only traffic is not that crazy at all, and their hardware footprint is incredibly over-provisioned for the workload. I don't really think their example stands well here. For example, at work, our entire analytics ingest workload (HTTP) for a few hundred million users runs on 8 core VMs on GCP, written in Rust/Go, each node doing ~40k…

Hi, I'm curious because I plan to rewrite a Rust service to Go (development velocity is too slow) Which part of your service is in Rust amd which in Go? Do you think that if it would be only in Go it could sustain such a load?

What specific ergonomics pitfalls did you experience in Rust? "Development velocity is too slow" is a bit vague; beyond the use of GC, which only really matters in specialized domains, there's not much reason to think that rewriting your service in Go would give you better 'development velocity'.
Post reply on HN