LLMs can improve Python code performance. I used it myself on a few projects.
Python numbers every programmer should know
61–70 of 191 posts
Re: Python numbers every programmer should know
#62Earlier quoted context omitted.
Having general knowledge about the tools you're working with is not a distraction, it's an intellectual enrichment in any case, and can be a valuable asset in specific cases.
Knowing that an empty string is 41 bytes or how many ns it takes to do arithmetic operations is not general knowledge.
How is it not general knowledge? How do you otherwise gauge if your program is taking a reasonable amount of time, and, if not, how do you figure out how to fix it?
Re: Python numbers every programmer should know
#63Why? If those micro benchmarks mattered in your domain, you wouldn't be using python.
...and other hilarious jokes you can tell yourself!
Re: Python numbers every programmer should know
#64A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…
I think both points are fair. Python is slow - you should avoid it if speed is critical, but sometimes you can’t easily avoid it. I think the list itself is super long winded and not very informative. A lot of operations take about the same amount of time. Does it matter that adding two ints is very slightly slower than adding two floats? (If you even believe this is true, which I don’t.) No. A better summary would s…
I think the list itself is super long winded and not very informative.
I agree. I have to complement the author for the effort put in. However it misses the point of the original Latency numbers every programmer should know, which is to build an intuition for making good ballpark estimations of the latency of operations and that e.g. A is two orders of magnitude more expensive than B.
Re: Python numbers every programmer should know
#65Every Python programmer should be thinking about far more important things than low level performance minutiae. Great reference but practically irrelevant except in rare cases where optimization is warranted. If your workload grows to the point where this stuff actually matters, great! Until then it’s a distraction.
Yeah, if you hit limits just look for a module that implements the thing in C (or write it). This is how it was always done in Python.
I am currently (as we type actually LOL) doing this exact thing in a hobby GIS project: Python got me a prototype and proof of concept, but now that I am scaling the data processing to worldwide, it is obviously too slow so I'm rewriting it (with LLM assistance) in C. The huge benefit of Python is that I have a known working (but slow) "reference implementation" to test against. So I know the C version works when it produces identical output. If I had a known-good Python version of past C, C++, Rust, etc. projects I worked on, it would have been most beneficial when it came time to test and verify.
Re: Python numbers every programmer should know
#66I'm confused why they repeatedly call a slots class larger than a regular dict class, but don't count the size of the dict
Re: Python numbers every programmer should know
#67Earlier quoted context omitted.
> you need to know these numbers on Python to know when to actually build something in C People usually approach this the other way, use something like pandas or numpy from the beginning if it solves your problem. Do not write matrix multiplications or joins in python at all. If there is no library that solves your problem, it's a great indication that you should avoid python. Unless you are willing to spend 5 man-ye…
People generally aren’t rolling their own matmuls or joins or whatever in production code. There are tons of tools like Numba, Jax, Triton, etc that you can use to write very fast code for new, novel, and unsolved problems. The idea that “if you need fast code, don’t write Python” has been totally obsolete for over a decade.
Yes, that's what I said.
If you are writing performance sensitive code that is not covered by a popular Python library, don't do it unless you are a megacorp that can put a team to write and maintain a library.
Re: Python numbers every programmer should know
#68Earlier quoted context omitted.
> A lot of important and large codebases were grown and maintained in Python How does this happen? Is it just inertia that cause people to write large systems in a essentially type free, interpreted scripting language?
It’s very natural. Python is fantastic for going from 0 to 1 because it’s easy and forgiving. So lots of projects start with it. Especially anything ML focused. And it’s much harder to change tools once a project is underway.
this is absolutely true, but there's an additional nuance: yes, python is fantastic, yes, it's easy and forgiving, but there are other languages like that too.
...except there really aren't. other than ruby and maybe go, every other popular language sacrifices ease of use for things that simply do not matter for the overwhelming majority of programs. much of python's popularity doesn't come from being easy and forgiving, it's that everything else isn't. for normal programming why would we subject ourselves to anything but python unless we had no choice?
while I'm on the soapbox I'll give java a special mention: a couple years ago I'd have said java was easy even though it's tedious and annoying, but I've become reacquainted with it for a high school program (python wouldn't work for what they're doing and the school's comp sci class already uses java.)
this year we're switching to c++.
Re: Python numbers every programmer should know
#69Counterintuitively: program in python only if you can get away without knowing these numbers. When this starts to matter, python stops being the right tool for the job.
Why? I've build some massive analytic data flows in Python with turbodbc + pandas which are basically C++ fast. It uses more memory which supports your point, but on the flip-side we're talking $5-10 extra cost a year. It could frankly be $20k a year and still be cheaper than staffing more people like me to maintain these things, rather than having a couple of us and then letting the BI people use the tools we provid…
From the complete opposite side, I've built some tiny bits of near irrelevant code where python has been unacceptable, e.g. in shell startup / in bash's PROMPT_COMMAND, etc. It ends up having a very painfully obvious startup time, even if the code is nearing the equivalent of Hello World
time python -I -c 'print("Hello World")'
real 0m0.014s
time bash --noprofile -c 'echo "Hello World"'
real 0m0.001sRe: Python numbers every programmer should know
#70> small int (0-256) cached
It's -5 to 256, and these have very tricky behavior for programmers that confuse identity and equality.
>>> a = -5
>>> b = -5
>>> a is b
True
>>> a = -6
>>> b = -6
>>> a is b
False