Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A lot of complex “scalable” systems can be done with a simple, single C++ server

twitter.com

Post reply on HN
Print view View original

301–310 of 376 posts

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#301

Post by vchak1 »

hnews_account_1 wrote: post #132
jandrewrogers wrote: post #47

Many developers severely underestimate how much workload can be served by a single modern server and high-quality C++ systems code. I've scaled distributed workloads 10x by moving them to a single server and a different software architecture more suited for scale-up, dramatically reducing system complexity as a bonus. The number of compute workloads I see that actually need scale-out is vanishingly small even in indu…

Can you expand on this? I have some pretty massive compute loads that need to be scaled onto a cluster with 100+ workers for most computations. This is after I use a library called dask that graphically does its own mapreduce optimisation inside its modules. This is all for a relatively small 250GB raw data file that I keep in a csv (and need to convert to SQL at some point). Are you saying this can be optimised to f…

Need to understand your domain better, but in many cases, the 250GB csv can be compressed down quite effectively using a columnar representation. And the columns can (potentially) be processed using simd/gpu based approaches to where a single server would outrun a cluster. Food for thought..

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#302

Post by paulddraper »

ses1984 wrote: post #38
nine_k wrote: post #32

Earlier quoted context omitted.

The only point of using Python threads is to wait for I/O. Which, while limiting, is a huge use case, for network servers in particular.

Can't you also defer work to c extensions in threads?

Yes. Numpy releases the GIL for its array operations.

Parallelism is possible, but only for computations in native code.

FWIW Node.js is essentially the same situation.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#303

Post by paulddraper »

lmm wrote: post #242
paulddraper wrote: post #205

Earlier quoted context omitted.

Web servers are usually trivially horizontally scalable. You must have had significant in-memory shared state to encounter that problem. Right? Had you adopted a less stateful model, you'd have looked rather pretty with two Java servers.

> You must have had significant in-memory shared state to encounter that problem. Right? "Significant" is in the eye of the beholder. The core of the system was easy to make shardable. But you'd be surprised how many implicit assumptions creep in, how easy it is for ancillary parts to end up sharing state when it's easy. Also note that just because your state's in a database doesn't mean having two instances of the t…

> At the point where we're running 4 or 8 servers we'd have been facing much the same ops problems that they were.

Yes and no.

You'd need some ops work, but you'd need to worry a lot less about managing your infrastructure provisioning to keep costs low, e.g. reserved instances, dynamic scaling, etc. and putting out fires when you inevitable exceed your tight perf margins.

You could overprovision 24/7 by 50% and write it off. Your competitors couldn't.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#304

Post by anon73044 »

dboreham wrote: post #82
pleasecalllater wrote: post #49

Earlier quoted context omitted.

Two. The number is two. You always need a backup server :)

Three. So you can do maintenance on one while still having HA.

I was expecting a "two is one and one is none" reference. Three keeps the investors happy in my case.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#305

Post by paulddraper »

aspaceman wrote: post #34
edgyquant wrote: post #33

Earlier quoted context omitted.

I'd like to see these 10 lines that supposedly avoid the problems with the GIL.

It still hits the problems, but it can still be better than single-threaded. That's the point of his comment. People say the GIL is bad so they throw the baby out with the bathwater and no longer use threads, which isn't very well-reasoned. Been a while since I've used Python, but as far as I remember the GIL only affects Python objects. So if you use Numpy for operations, you can avoid the GIL.

> as far as I remember the GIL only affects Python objects. So if you use Numpy for operations, you can avoid the GIL.

Correct.

Threads cannot help Python-heavy code; they can help I/O-heavy or native-heavy code (assuming native code releases the GIL).

Even without multithreading, numpy programming can give amazing speedups. It's the reason why data science Python isn't dead in the water.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#306

Post by paulddraper »

BalinKing wrote: post #8

From the next tweet in the thread: “JAVA [sic] or C# would also be close, and there are good reasons to prefer those over C++ for servers.”

Yep. C++ is the nuclear option, when you know you need to keep memory reasonable, or capture the last bit of compute performance.

Neither of these are usually the case for web application servers.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#307

Post by ww520 »

filleduchaos wrote: post #260
ww520 wrote: post #241

Earlier quoted context omitted.

Backend web development doesn't change much once developed. How many ways can one do CRUD on the backend?

Out of curiosity, have you ever actually been employed as a backend web developer?

Not specifically just backend but full stack projects that included backend.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#308

Post by vladf »

IanCal wrote: post #227
pnako wrote: post #210

Earlier quoted context omitted.

I'm starting to wonder if there is market to do "technology laundering", use things like PostgreSQL, SQLite, standard Unix tools, put it under some cloud marketing and charge a x10 premium. Or perhaps not only there is market, but that's more or less what everyone is already doing.

If you can set it up so it solves my problems, yes. I could manage my own server and fine tune everything, or I can throw it on snowflake. Snowflake means I've spent almost no time managing anything and it was costing less than an aws box running postgres but absolutely blew it out of the water performance wise. Depends on your workload but it's been perfect for one of my use cases. If they were just using postgres u…

You use Snowflake for OLTP? Can you comment more on how/why?

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#309

Post by ww520 »

filleokus wrote: post #271
ww520 wrote: post #241

Earlier quoted context omitted.

Backend web development doesn't change much once developed. How many ways can one do CRUD on the backend?

In my experience this is only true iff the system never gets any more user facing features. Every new non-insignificant feature requires a new REST-API route, or database table or modification of the GraphQL schema. And depending on how you designed the backend, even small redesigns of the frontend might require changes on the backend. Consider a simple app showing car rentals, where you initially have something like…

But all those are just extensions using the established frameworks and development practices for that backend application. Once those are developed, it’s pretty easy to add new features. I would argue using Ruby and Python like script languages brings no significant benefit in development time. In fact they make it worse when working on an existing app, with the extra unit tests, difficulty in refactoring, and extra performance work.

Re: A lot of complex “scalable” systems can be done with a simple, single C++ server

#310

Post by giantrobot »

inetknght wrote: post #259
giantrobot wrote: post #224

Earlier quoted context omitted.

You have 250GB of "raw" data stored in CSV format. The parsed version of this data in memory is likely to be a fraction of the on-disk size. A `long` or `double` only take up eight bytes in memory but 10-20 bytes on disk stored as ASCII in a CSV file. Even if your raw data was 250GB you could store it in memory mapped files. A fast SSD can easily hit a gigabyte per second sequential read speed, far faster than your t…

> A fast SSD can easily hit a gigabyte per second sequential read speed, far faster than your typical network. It's important to note that often your disks aren't directly attached to your compute. That's frequently the case in (particularly cheap) cloud instances.

That's the thing, you don't necessarily need a bunch of cloud instances to process data. If you must be doing everything in "the cloud" every service has dedicated instances with fast attached storage available. You can spin up one long enough to rip through your data instead of trying to distribute it over hundreds of workers.

It's also a domain where you can buy an off-the-shelf desktop for a few hundred dollars to do the work. That's the thrust of this whole thread, because scalable "cloud" systems exist and look cheap people obsess about throwing more instances at problems.

Modern commodity systems are ridiculously powerful and far more capable than people tend to assume. Even "the cloud" gets underestimated because people look at the low end cheap instances and assume they need to spin up hundreds of those when one beefy image for a short duration could do the same work.

Post reply on HN