Python performance myths and fairy tales

As a matter of practice: the python community disagrees strongly. And the python community ate the world.

It's fine to have an opinion, but you're not going to change python.

For 99.99% of the programs that people write, the modern M.2 NVME hard drives are plenty fast, and thats the laziest way to load data into a C extension or process.

Then there is unix pipes which are sufficiently fast.

Then there is shared memory, which basically involves no loading.

As with Python, all depends on the setup.

Typically from a user perspective, the initial starting time is either manageable or imperceptible in the cases of long running services, although there are other costs.

If you look at examples that make the above claim, they are almost always tiny toy programs where the cost of producing byte/machine code isn't easily amortized.

This quote from the post is an oversimplification too:

> But the program will then run into Amdahl's law, which says that the improvement for optimizing one part of the code is limited by the time spent in the now-optimized code

I am a huge fan of Amdahl's law, but also realize it is pessimistic and most realistic with parallelization.

It runs into serious issues when you are multiprocessing vs parallel processing due to preemption, etc .

Yes you still have the costs of abstractions etc...but in today's world, zero pages on AMD, 16k pages and a large number of mapped registers on arm, barrel shifters etc... make that much more complicated especially with C being forced into trampolines etc...

If you actually trace the CPU operations, the actual operations for 'math' are very similar.

That said modern compilers are a true wonder.

Interpreted language are often all that is necessary and sufficient. Especially when you have Internet, database and other aspects of the system that also restrict the benefits of the speedups due to...Amdahl's law.

Except it does. The key is to figure out which part you actually need to go fast, and write it in C. If most of your use case is dominated by network latency.

Overall, people seem to miss the point of Python. The best way to develop software is "make it work, make it good, make it fast" - the first part gets you to an end to end prototype that gives you a testable environment, the second part establishes the robustness and consistency, and the third part lets you focus on optimizing the performance with a robust framework that lets you ensure that your changes are not breaking anything.

Pythons focus is on the first part. The idea is that you spend less time making it work. Once you have it working, then its much easier to do the second part (adding tests, type checking, whatever else), and then the third part. Now with LLMs, its actually pretty straightforward to take a python file and translate it to .c/.h files, especially with agents that do additional "thinking" loops.

However, even given all of that, in practice you often don't need to move away from Python. For example, I have a project that datamines Strava Heatmaps (i.e I download png tiles for entire US). The amount of time that it took me to write it in Python in addition to running it (which takes about a day) is much shorter than it would have taken me to write it in C++/Rust and then run it with speedup in processing.