Live data from Hacker News

Python numbers every programmer should know

mkennedy.codes

101–110 of 191 posts

Re: Python numbers every programmer should know

#101
There are lots of discussions about relatedness of these numbers for a regular software engineer.

Firstly, I want to start with the fact that the base system is a macOS/M4Pro, hence;

- Memory related access is possibly much faster than a x86 server. - Disk access is possibly much slower than a x86 server.

*) I took x86 server as the basis as most of the applications run on x86 Linux boxes nowadays, although a good amount of fingerprint is also on other ARM CPUs.

Although it probably does not change the memory footprint much, the libraries loaded and their architecture (ie. being Rosetta or not) will change the overall footprint of the process.

As it was mentioned on one of the sibling comments -> Always inspect/trace your own workflow/performance before making assumptions. It all depends on specific use-cases for higher-level performance optimizations.

Re: Python numbers every programmer should know

#102
post #29

Earlier quoted context omitted.

That's an "all or nothing" fallacy. Just because you use Python and are OK with some slowdown, doesn't mean you're OK with each and every slowdown when you can do better. To use a trivial example, using a set instead of a list to check membership is a very basic replacement, and can dramatically improve your running time in Python. Just because you use Python doesn't mean anything goes regarding performance.

That's an example of an algorithmic improvement (log n vs n), not a micro benchmark, Mr. Fallacy.

"Mr. Fallacy."? Got any better juvenile name-calling?

The case is among the example numbers given in TFA:

"Dict lookup by key", "List membership check"

Does it have to spell out the difference is algorithmic in this case for the comparison to be useful?

Or, inversely, is the difference between e.g. memory and disk access times insignificant, because it's not algorithmic?

Re: Python numbers every programmer should know

#103
post #77

The point of the original list was that the numbers were simple enough to memorize: https://gist.github.com/jboner/2841832 Nobody is going to remember any of the numbers on this new list.

That's a fair point @esafak. I updated the article with something akin to the doubling chart of numbers in the original article from 2012.

Re: Python numbers every programmer should know

#104

Author here. Thanks for the feedback everyone. I appreciate your posting it @woodenchair and @aurornis for pointing out the intent of the article. The idea of the article is NOT to suggest you should shave 0.5ns off by choosing some dramatically different algorithm or that you really need to optimize the heck out of everything. In fact, I think a lot of what the numbers show is that over thinking the optimizations of…

Then you should have written that. Instead you have given more fodder for the premature optimization crowd.

I didn't tell anyone to optimize anything. I just posted numbers. It's not my fault some people are wired that way. Anytime I suggested some sort of recommendation it was to NOT optimize.

For example, from the post "Maybe we don’t have to optimize it out of the test condition on a while loop looping 100 times after all."

Re: Python numbers every programmer should know

#105
post #52

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

> A lot of important and large codebases were grown and maintained in Python How does this happen? Is it just inertia that cause people to write large systems in a essentially type free, interpreted scripting language?

Most large things begin life as small things.

Re: Python numbers every programmer should know

#106
post #52

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

> A lot of important and large codebases were grown and maintained in Python How does this happen? Is it just inertia that cause people to write large systems in a essentially type free, interpreted scripting language?

Python has types, now even gradual static typing if you want to go further. It's irrelevant whether language is interpreted scripting if it solves your problem.

Re: Python numbers every programmer should know

#107
post #82

Earlier quoted context omitted.

These are the metrics underneath it all. Profiles tell you what parts are slow relative to others and time your specific implementation. How long should it take to sum together a million integers?

It literally doesn’t matter unless it impacts users. I don’t know why you would waste time on non problems.

No one is suggesting “wasting time on non problems.” You’re tilting at windmills.

Re: Python numbers every programmer should know

#108
post #85

It’s missing the time taken to instantiate a class. I remember refactoring some code to improve readability, then observing something that was previously a few microseconds take tens of seconds. The original code created a large list of lists. Each child list had 4 fields each field was a different thing, some were ints and one was a string. I created a new class with the names of each field and helper methods to pro…

Instantiating classes is in general not a performance issue in Python. Your issue here strongly sounds like you're abusing OO to pass a list of instances into every method and downstream call (not just the usual reference to self, the instance at hand). Don't do that, it shouldn't be necessary. It sounds like you're trying to get a poor-man's imitation of classmethods, without identifying and refactoring whatever it is that methods might need to access from other instances.

Please post your code snippet on StackOverflow ([python] tag) or CodeReview.SE so people can help you fix it.

> created a new class with the names of each field and helper methods to process the data. The new code created a list of instances of my class. Downstream consumers of the list could look at the class to see what data they were getting.

Re: Python numbers every programmer should know

#109

That's a long list of numbers that seem oddly specific. Apart from learning that f-strings are way faster than the alternatives, and certain other comparisons, I'm not sure what I would use this for day-to-day. After skimming over all of them, it seems like most "simple" operations take on the order of 20ns. I will leave with that rule of thumb in mind.

If you're interested, fstrings are faster because they directly become bytecode at compile time rather than being a function call at runtime

Thanks for the that bit of info! I was surprised by the speed difference. I have always assumed that most variations of basic string formatting would compile to the same bytecode.

I usually prefer classic %-formatting for readability when the arguments are longer and f-strings when the arguments are shorter. Knowing there is a material performance difference at scale, might shift the balance in favour of f-strings for some situations.

Re: Python numbers every programmer should know

#110
post #55

Earlier quoted context omitted.

It’s very natural. Python is fantastic for going from 0 to 1 because it’s easy and forgiving. So lots of projects start with it. Especially anything ML focused. And it’s much harder to change tools once a project is underway.

this is absolutely true, but there's an additional nuance: yes, python is fantastic, yes, it's easy and forgiving, but there are other languages like that too. ...except there really aren't. other than ruby and maybe go, every other popular language sacrifices ease of use for things that simply do not matter for the overwhelming majority of programs. much of python's popularity doesn't come from being easy and forgiv…

Omg, switching to C++ for pupils programming beginners ... "How to turn off the most students from computer programming?" 101. Really can't get much worse than C++ for beginners.
Post reply on HN