Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python numbers every programmer should know

mkennedy.codes

Post reply on HN
Print view View original

151–160 of 191 posts

Re: Python numbers every programmer should know

#151

Post by aragilar »

Scubabear68 wrote: post #113
thundergolfer wrote: post #48

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

No. Python’s issue is that it is incredibly slow in use cases that surprise average developers. It is incredibly slow at very basic stuff, like calling a function or accessing a dictionary. If Python didn’t have such an enormous number of popular C and C++ based libraries it would not be here. It was saved by Numpy etc etc.

I'm not sure how Python can be described as "saved" by numpy et al., when the numerical Python ecosystem was there near the beginning, and the language and ecosystem have co-evolved? Why didn't Perl (with PDL), R or Ruby (or even php) succeed in the same way?

Re: Python numbers every programmer should know

#152

Post by TacticalCoder »

thundergolfer wrote: post #48

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

> ... a function O(10_000) times in a hot loop

O(10_000) is a really weird notation.

Re: Python numbers every programmer should know

#153

Post by KeplerBoy »

Demiurge wrote: post #126
AtlasBarfed wrote: post #75

Earlier quoted context omitted.

..... You don't see any value in knowing that numbers?

That's what I just said. There is zero value to me knowing these numbers. I assume that all python built in methods are pretty much the same speed. I concentrate on IO being slow, minimizing these operations. I think about CPU intensive loops that process large data, and I try to use libraries like numpy, DuckDB, or other tools to do the processing. If I have a more complicated system, I profile its methods, and opti…

That makes perfect sense. Especially since those numbers can change with new python versions.

Re: Python numbers every programmer should know

#154

Post by srean »

saagarjha wrote: post #143
thundergolfer wrote: post #48

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

I do performance optimization for a system written in Python. Most of these numbers are useless to me, because they’re completely irrelevant until they become a problem, then I measure them myself. If you are writing your code trying to save on method calls, you’re not getting any benefit from using the language and probably should pick something else.

It's always a balance.

Good designs do not happen in a vacuum but informed with knowledge of at least the outlines of the environment.

One can have a breakfast pursuing an idea -- let me spill some sticky milk on the dining table, who cares, I will clean up if it becomes a problem later.

Another is, it's not much of an overbearing constraint not to make a mess with spilt milk in the first place, maybe it will not be a big bother later, but it's not hurting me much now, to be not be sloppy, so let me be a little hygienic.

There's a balance between making a mess and cleaning up and not making a mess in the first place. The other extreme is to be so defensive about the possibility of creating a mess that it paralyses progress.

The sweet spot is somewhere between the extremes and having the ball-park numbers in the back of one's mind helps with that. It informs about the environment.

Re: Python numbers every programmer should know

#155

Post by tialaramex »

TacticalCoder wrote: post #152
thundergolfer wrote: post #48

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

> ... a function O(10_000) times in a hot loop O(10_000) is a really weird notation.

Generously we could say they probably mean ~10_000 rather than O(10_000)

Re: Python numbers every programmer should know

#156

Post by HenriTEL »

Scubabear68 wrote: post #113
thundergolfer wrote: post #48

A lot of people here are commenting that if you have to care about specific latency numbers in Python you should just use another language. I disagree. A lot of important and large codebases were grown and maintained in Python (Instagram, Dropbox, OpenAI) and it's damn useful to know how to reason your way out of a Python performance problem when you inevitably hit one without dropping out into another language, whic…

No. Python’s issue is that it is incredibly slow in use cases that surprise average developers. It is incredibly slow at very basic stuff, like calling a function or accessing a dictionary. If Python didn’t have such an enormous number of popular C and C++ based libraries it would not be here. It was saved by Numpy etc etc.

22ns for a function call and dictionary key lookup, that's actually surprisingly fast.

Re: Python numbers every programmer should know

#157

Post by dirtbag__dad »

oivey wrote: post #130
dirtbag__dad wrote: post #127

Earlier quoted context omitted.

In my experience, which is series A or earlier data intensive SaaS, you can gauge whether a program is taking a reasonable amount of time just by running it and using your common sense. P50 latency for a fastapi service’s endpoint is 30+ seconds. Your ingestion pipeline, which has a data ops person on your team waiting for it to complete, takes more than one business day to run. Your program is obviously unacceptable…

I'm not sure it's common knowledge, but it is general knowledge. Not all HNers are writing web apps. Many may be writing truly compute bound applications. In my experience writing computer vision software, people really struggle with the common sense of how fast computers really are. Some knowledge like how many nanoseconds an add takes can be very illuminating to understand whether their algorithm's runtime makes an…

Thanks this is helpful framing!

Re: Python numbers every programmer should know

#158

Post by Izkata »

nightfly wrote: post #131
zelphirkalt wrote: post #110

Earlier quoted context omitted.

Omg, switching to C++ for pupils programming beginners ... "How to turn off the most students from computer programming?" 101. Really can't get much worse than C++ for beginners.

PSU (Oregon) uses C++ as just "c with classes" and ignores the rest of C++ for intro to programming courses. It frustrates people who already use C++ but otherwise works pretty well.

This was how we learned it in an intro class in highschool ages ago, worked pretty well there too.

Re: Python numbers every programmer should know

#159

Post by jgalt212 »

nightfly wrote: post #131
zelphirkalt wrote: post #110

Earlier quoted context omitted.

Omg, switching to C++ for pupils programming beginners ... "How to turn off the most students from computer programming?" 101. Really can't get much worse than C++ for beginners.

PSU (Oregon) uses C++ as just "c with classes" and ignores the rest of C++ for intro to programming courses. It frustrates people who already use C++ but otherwise works pretty well.

C++, The Good Parts

Re: Python numbers every programmer should know

#160

Post by Izkata »

CmdrKrool wrote: post #100

I'm confused by this: String operations in Python are fast as well. f-strings are the fastest formatting style, while even the slowest style is still measured in just nano-seconds. Concatenation (+) 39.1 ns (25.6M ops/sec) f-string 64.9 ns (15.4M ops/sec) It says f-strings are fastest but the numbers show concatenation taking less time? I thought it might be a typo but the bars on the graph reflect this too?

String concatenation isn't usually considered a "formatting style", that refers to the other three rows of the table which use a template string and have specialized syntax inside it to format the values.
Post reply on HN