Yes, I’m always shocked by just how much performance overhead most languages have compared to C and similar lower level languages. It is a price worth paying for better language ergonomics, but I do wonder whether Rust might be able to give us the best of both worlds here.
A lot of complex “scalable” systems can be done with a simple, single C++ server
41–50 of 376 posts
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#42Earlier quoted context omitted.
Many other people "know" about the GIL, to the extent of believing there's no point using threads in python "because of the GIL". I had a funny such experience lately in a job interview. I told the interviewer his misconception could be falsified with ~10 LOC summing a list with 2 threads.
I'd like to see these 10 lines that supposedly avoid the problems with the GIL.
> I'd like to see these 10 lines that supposedly avoid the problems with the GIL.
Spawn a thread and handle disk or socket IO requests there? You can get fancy and use a thread pool so maybe 15 lines of code.
Not sure why you used the word “supposedly”. Threads have been in the standard library for a very long time.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#43Earlier quoted context omitted.
The only point of using Python threads is to wait for I/O. Which, while limiting, is a huge use case, for network servers in particular.
Can't you also defer work to c extensions in threads?
Some extensions do release GIL while doing its own parallel processing, so, to a degree, yes.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#44Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#45Yes, I’m always shocked by just how much performance overhead most languages have compared to C and similar lower level languages. It is a price worth paying for better language ergonomics, but I do wonder whether Rust might be able to give us the best of both worlds here.
He did say that Java/C# are also up there, and Go is in that family, so it probably remains the best balance for lots of cases. I do think there's also territory to be explored writing hot paths in Rust and interoping from Python/JS.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#46Carmack is moving to AI and inevitably has to deal with a lot of Python, which still bottlenecks process here an there despite all the effort to move computation to C extensions. I have a really high hope that he detours a bit and creates very-very good non-python tooling for ML.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#47Many developers severely underestimate how much workload can be served by a single modern server and high-quality C++ systems code. I've scaled distributed workloads 10x by moving them to a single server and a different software architecture more suited for scale-up, dramatically reducing system complexity as a bonus. The number of compute workloads I see that actually need scale-out is vanishingly small even in industries known for their data intensity. You can often serve millions of requests per second from a single server even when most of your data model resides on disk.
We've become so accustomed to extremely inefficient software systems that we've lost all perspective on what is possible.
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#48As someone who has implemented a complex system in C++ in this decade, I’d say he’s not wrong, but you need to carefully weight the pros and cons. In our case latency and real time demands mattered a lot (NASDAQ feed parser), to the point of the (potential) slowdown of a garbage collector kicking in was enough to rule out Java and .NET. It runs entirely in memory and on 64+ cores. We implemented our own reference cou…
If you're parsing market data then you shouldn't really be allocating at all once the initial setup is complete. So there shouldn't be a need for ref counts. This is because the allocator itself can have an unbounded runtime that takes milliseconds, causing you to drop. In the past I've replaced malloc with an implementation that asserts if called on certain threads after init time.
Milliseconds??! What causes that to occur? Though I suppose if they can take that long, avoiding them is an even better idea than it is normally...
For constant time allocations, try TLSF: http://www.gii.upv.es/tlsf/
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#49Many developers severely underestimate how much workload can be served by a single modern server and high-quality C++ systems code. I've scaled distributed workloads 10x by moving them to a single server and a different software architecture more suited for scale-up, dramatically reducing system complexity as a bonus. The number of compute workloads I see that actually need scale-out is vanishingly small even in indu…
Two. The number is two. You always need a backup server :)
Re: A lot of complex “scalable” systems can be done with a simple, single C++ server
#50Earlier quoted context omitted.
Not really IO is always going to be slow compared to the CPU. The Python interpreter will be blocked 90% of that time waiting for IO anyway.
Exactly, and I’m surprised none of the other comments mentioned this so far. Often web app endpoints are bottlenecked by IO because they’re spending most of their time talking to a database or cache server. Python is probably not the right tool for a CPU intensive endpoint that needs to serve up hundreds of thousands of requests per minute and can’t be cached. There are a lot of ways to handle the IO intensive scenar…
From what I’ve seen in production environments, Python has an uncanny ability to take tasks that should be IO- / server- bound and make them CPU-bound.