This is why Google developed Go. Python like productivity with C++ like performance...
A lot of complex “scalable” systems can be done with a simple, single C++ server
351–360 of 376 posts
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#352Earlier quoted context omitted.
A big push in .net core 2.x and 3.x was what we call the Span-ification of the base class library and the runtime. This means there are many new APIs for dealing with slices of memory in a non-allocating manner, and this combined with memory pooling has contributed greatly to an overall performance boost to the runtime by reducing copying and GC time. These same APIs are available to the developer so I'd imagine that…
There exist libraries for native memory pooling in Java as well, and we're using them. I'm not saying low allocation code can't be done in C# or Java. But these languages don't give some nice tools that are present in C++ and Rust - in particular RAII and automatic reference counting.
> I'm not saying low allocation code can't be done in C# or Java.
With modern tools and APIs it should even be possible to write alloc-free work loops in C# (though you're probably writing pretty alien C# at this point). AFAIK it'll remain impossible in Java until the "value types" effort bears fruits.
A few months back a study of sort made the rounds implementing a network driver in multiple languages (Ixy), C# did extremely well in it (better throughput than Go and near competitive with C and Rust at higher batch size though it was way behind on latency) while Java was pretty much in the dumps.
In one of the reaction threads (don't remember if it was on HN or Reddit) one of the people involved explained the discrepancy between Java and C# by not being able to go under ~20 bytes of allocation per packet forwarded in Java.
There were other odd / interesting results from the effort e.g. Rust was slightly slower than C, in investigating that they found out Rust executed way more instructions (especially significantly more stores) but had significantly higher IPC and much higher cache hit rates.
ixy also tried just converting the C code to Rust (using C2Rust) then compiling it and that turned out to be very slightly faster than the C code, which was funny: https://github.com/ixy-languages/ixy-languages/blob/master/R...
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#353Earlier quoted context omitted.
Can you expand on this? I have some pretty massive compute loads that need to be scaled onto a cluster with 100+ workers for most computations. This is after I use a library called dask that graphically does its own mapreduce optimisation inside its modules. This is all for a relatively small 250GB raw data file that I keep in a csv (and need to convert to SQL at some point). Are you saying this can be optimised to f…
You have 250GB of "raw" data stored in CSV format. The parsed version of this data in memory is likely to be a fraction of the on-disk size. A `long` or `double` only take up eight bytes in memory but 10-20 bytes on disk stored as ASCII in a CSV file. Even if your raw data was 250GB you could store it in memory mapped files. A fast SSD can easily hit a gigabyte per second sequential read speed, far faster than your t…
> A fast SSD can easily hit a gigabyte per second sequential read speed, far faster than your typical network.
Nit: 1GB/s is ok, not even solid let alone fast.
A fast SSD pretty much saturates 4x3.0 links (which explains why they universally tend to cap out at 3.5GB/s). In fact there are now a few PCIe4 SSDs (e.g. Corsair's MP600) which close in on 5GB/s.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#354Earlier quoted context omitted.
One thing to keep in mind that their work-load is very ready-heavy which eases things a lot when scaling the system. The same is true for Wikipedia. Scaling a write-heavy workload is way more complex than scaling a read-heavy workload.
SO content changes all the time. Votes, comments, moderation, edits, tagging, search and recommendations, etc. There are also real-time community features. It's not as simple as it seems.
All that can be implemented as separate services. Doing that also enables graceful degradation, as a failure of, say, voting doesn't prevent me from reading the question.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#355Earlier quoted context omitted.
Can you expand on this? I have some pretty massive compute loads that need to be scaled onto a cluster with 100+ workers for most computations. This is after I use a library called dask that graphically does its own mapreduce optimisation inside its modules. This is all for a relatively small 250GB raw data file that I keep in a csv (and need to convert to SQL at some point). Are you saying this can be optimised to f…
Don't know why you're being downvoted but I'll assume your question is genuine. You use a cluster when your data and compute requirements are large and parallel enough that the tax paid on network latency trumps the 10-20X speedup you get on SSD and 1000X speedup you get from just keeping data in RAM. 250 Gigs is tiny enough that you could probably much get better performance running on high memory instance in AWS or…
> You use a cluster when your data and compute requirements are large and parallel enough that the tax paid on network latency trumps the 10-20X speedup you get on SSD and 1000X speedup you get from just keeping data in RAM.
Nowadays there's another reason to use clusters: to autoscale your expenditure wrt workload. A little inefficiency might be acceptable if you don't have to pay for a huge beefed up server idling at, say, 30%.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#356Earlier quoted context omitted.
The idea that GCed languages in general have Python-like performance is a dangerous myth. Languages that are managed but not interpreted (e.g. Java, OCaml, Haskell, C#, Swift) have performance characteristics that are much closer to C than to Python.
Actually, Python is compiled to its bytecode and then interpreted by the Python VM, which is also how Java works. Python is slow because the lack of funds and people's focus, just take look at how fast JavaScript(V8) is nowadays.
I think you entirely ignored the performance cost of dynamism (not having types known, not having value types, dynamic binding of methods etc..) that is handy 1% of the time but imposes type safety & performance cost other 99% of the time..
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#357Earlier quoted context omitted.
Can you expand on this? I have some pretty massive compute loads that need to be scaled onto a cluster with 100+ workers for most computations. This is after I use a library called dask that graphically does its own mapreduce optimisation inside its modules. This is all for a relatively small 250GB raw data file that I keep in a csv (and need to convert to SQL at some point). Are you saying this can be optimised to f…
Don't know why you're being downvoted but I'll assume your question is genuine. You use a cluster when your data and compute requirements are large and parallel enough that the tax paid on network latency trumps the 10-20X speedup you get on SSD and 1000X speedup you get from just keeping data in RAM. 250 Gigs is tiny enough that you could probably much get better performance running on high memory instance in AWS or…
note: doing anything on a 250gb file in python will require a lot more ram than 250gb. generally my expectation is I will need 10x the ram as the size of the file when using pandas, for when I accidentally do something that triggers pathological behavior.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#358Earlier quoted context omitted.
Newer Java GCs are very low latency (microsecond). You trade performance and memory for that low latency though. AFAIK, they are still compacting. Still though, probably makes sense to do it in a lower level language. It's just far easier in C++ to decide that "Hey, you know what, I just want a big memory block that I control". I've even heard of game devs doing things like having per frame allocators. They get super…
The point is that a one millisecond pause is unacceptable. Low latency Java GCs have average latencies of one millisecond, 99th percentile latencies of 10 milliseconds, and 99.9th percentile latencies are neither measured nor optimized for. I don't consider it realistic to think that garbage collected languages might ever be usable in the context of game engines or HFT.
> game engines
Game engines are already written in GCed languages. Java in particular.
You may be right that a 10ms pause is unacceptable for HFT. However, for a FPS, 10ms is more than acceptable. It translates to 1 or 2 missed frames in the worst case.
A bigger issue with using Java in particular for games is it's lack of value types. Writing high performance code for java is just that much harder because the language gets in the way.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#359Earlier quoted context omitted.
The thing that worries me about stories like this is that there is frequently (as is the case here) no mention any sort of HA or backups. No details on what disaster recovery looks like. Those are business critical considerations that cost money, and just disappear from the discussion when people say "hey i saved all this money dropping everything down to a single server!"
Well, in my case described above, single server solutions include an automated backup sub-system, and my servers expect multiple instances of itself to running on the client network, and these multiple instances synchronize with one another via additional endpoints specific for the purpose. The whole issue of HA and backups is critical and one of the areas my approach shines.
You're still not actually answering the question of how you are HA and backing things up with only a single physical server and nothing else.
If you're backing things up to the same server, that's not enough. If the HA instances are running in the same server, that's not enough.
If there are other things besides that one physical server and it's power/network, you didn't include them in the cost, so the comparison is disingenuous there.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#360Earlier quoted context omitted.
The thing that worries me about stories like this is that there is frequently (as is the case here) no mention any sort of HA or backups. No details on what disaster recovery looks like. Those are business critical considerations that cost money, and just disappear from the discussion when people say "hey i saved all this money dropping everything down to a single server!"
I am in the same boat (writing native servers). I will also "disappear" if you start asking me about HA/backups/etc. A particular solutions are very much case specific and can depend on business conducting rules just as much as on pure tech factors. Properly answering your question requires way too much writing and hardly a subject of a single post. I have HA solutions for the products I built but this post is the ex…
I'm not asking about the specifics, and I don't really care about them.
But the fact of the matter is quite simply is that any single physical server solution will never be satisfactory for backups or HA purposes.
You can't store your backups in the same place as your data and call it good - what do you do when you have multiple disks fail and your RAID can't be rebuilt? This happens. What do you do when operator error accidentally destroys the array? This happens. What do you do when there's a datacenter fire and the server burns up? This happens. None of these things should be a business ending event, but if you only have a single server handling quintuple duty, that's what it has a real chance of being.
If you need HA, a single server isn't good enough even if you've got multiple VMs running the service. What do you do when the utility power is out and the generator fails to kick on properly or they run out of fuel? Both of those happen pretty frequently. What do you do when there's a network outage at the DC? This happens. When someone fucks up BGP somewhere and now the prefix you used is being routed to god knows where? This happens. When you have any sort of physical server failure that brings your single box down? This happens. Any of those situations will take you offline and render your HA meaningless.
I'm not saying that any of these things are impossible to do when colocating your hardware - but they're not free, and they're not mentioned even at a high level in this story. And since we're not talking about running benchmarks on price/performance and instead talking about a service that a business needs to keep available to their customers to make money, these are important aspects to talk about. Everywhere I've worked, keeping our services available and being able to recover from hardware failure are far more important priorities than being able to optimize performance. And doing those things properly takes more than one physical server.