Earlier quoted context omitted.
It literally doesn’t matter unless it impacts users. I don’t know why you would waste time on non problems.
No one is suggesting “wasting time on non problems.” You’re tilting at windmills.
Read more carefully
Python numbers every programmer should know
121–130 of 191 posts
Re: Python numbers every programmer should know
#122> small int (0-256) cached It's -5 to 256, and these have very tricky behavior for programmers that confuse identity and equality. >>> a = -5 >>> b = -5 >>> a is b True >>> a = -6 >>> b = -6 >>> a is b False
Java does similar. Confusing for beginners who run into it for the first time for sure.
Re: Python numbers every programmer should know
#123A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…
Agreed, and on top of that:
I think these kind of numbers are everywhere and not just specific to Python.
In zig, I sometimes take a brief look to the amount of cpu cycles of various operations to avoid the amount of cache misses. While I need to aware of the alignment and the size of the data type to debloat a data structure. If their logic applies, too bad, I should quit programming since all languages have their own latency on certain operations we should aware of.
There are reasons to not use Python, but that particular reason is not the one.
Re: Python numbers every programmer should know
#124I wonder why an empty set takes so much more memory than an empty dict
Re: Python numbers every programmer should know
#125My god, the memory bloat is out of this world compared to platforms like the JVM or .NET, let alone C++ or Rust!
Re: Python numbers every programmer should know
#126Earlier quoted context omitted.
I agree. I've been living off Python for 20 years and have never needed to know any of these numbers, nor do I need them now, for my work, contrary to the title. I also regularly use profiling for performance optimization and opt for Cython, SWIG, JIT libraries, or other tools as needed. None of these numbers would ever factor into my decision-making.
..... You don't see any value in knowing that numbers?
That's what I just said. There is zero value to me knowing these numbers. I assume that all python built in methods are pretty much the same speed. I concentrate on IO being slow, minimizing these operations. I think about CPU intensive loops that process large data, and I try to use libraries like numpy, DuckDB, or other tools to do the processing. If I have a more complicated system, I profile its methods, and optimize tight loops based on PROFILING. I don't care what the numbers in the article are, because I PROFILE, and I optimize the procedures that are the slowest, for example, using cython. Which part of what I am saying does not make sense?
Re: Python numbers every programmer should know
#127Earlier quoted context omitted.
Knowing that an empty string is 41 bytes or how many ns it takes to do arithmetic operations is not general knowledge.
How is it not general knowledge? How do you otherwise gauge if your program is taking a reasonable amount of time, and, if not, how do you figure out how to fix it?
In my experience, which is series A or earlier data intensive SaaS, you can gauge whether a program is taking a reasonable amount of time just by running it and using your common sense.
P50 latency for a fastapi service’s endpoint is 30+ seconds. Your ingestion pipeline, which has a data ops person on your team waiting for it to complete, takes more than one business day to run.
Your program is obviously unacceptable. And, your problems are most likely completely unrelated to these heuristics. You either have an inefficient algorithm or more likely you are using the wrong tool (ex OLTP for OLAP) or the right tool the wrong way (bad relational modeling or an outdated LLM model).
If you are interested in shaving off milliseconds in this context then you are wasting your time on the wrong thing.
All that being said, I’m sure that there’s a very good reason to know this stuff in the context of some other domains, organizations, company size/moment. I suspect these metrics are irrelevant to disproportionately more people reading this.
At any rate, for those of us who like to learn, I still found this valuable but by no means common knowledge
Re: Python numbers every programmer should know
#128Earlier quoted context omitted.
Someone says "let's write a prototype in Python" and someone else says "are you sure we shouldn't use a a better language that is just as productive but isn't going to lock us into abysmal performance down the line?" but everyone else says "nah we don't need to worry about performance yet, and anyway it's just a prototype - we'll write a proper version when we need to"... 10 years later "ok it's too slow; our options…
What language is “just as productive but isn't going to lock us into abysmal performance down the line”? What makes that language not strictly superior to Python?
Loose typing makes you really fast at writing code, as long as you can keep all the details in your head. Python is great for smaller stuff. But crossed some threshold, the lack of a mechanism that has your back starts slowing you down.
Re: Python numbers every programmer should know
#129It is always a good idea to have at least a rough understanding of how much operations in your code cost, but sometimes very expensive mistakes end up in non-obvious places.
If I have only plain Python installed and a .py file that I want to test, then what's the easiest way to get a visualization of the call tree (or something similar) and the computational cost of each item?
Re: Python numbers every programmer should know
#130Earlier quoted context omitted.
How is it not general knowledge? How do you otherwise gauge if your program is taking a reasonable amount of time, and, if not, how do you figure out how to fix it?
In my experience, which is series A or earlier data intensive SaaS, you can gauge whether a program is taking a reasonable amount of time just by running it and using your common sense. P50 latency for a fastapi service’s endpoint is 30+ seconds. Your ingestion pipeline, which has a data ops person on your team waiting for it to complete, takes more than one business day to run. Your program is obviously unacceptable…
I'm not sure it's common knowledge, but it is general knowledge. Not all HNers are writing web apps. Many may be writing truly compute bound applications.
In my experience writing computer vision software, people really struggle with the common sense of how fast computers really are. Some knowledge like how many nanoseconds an add takes can be very illuminating to understand whether their algorithm's runtime makes any sense. That may push loose the bit of common sense that their algorithm is somehow wrong. Often I see people fail to put bounds on their expectations. Numbers like these help set those bounds.