Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Profiling.sampling – Statistical Profiler

docs.python.org

Post reply on HN
Print view View original

21–26 of 26 posts

Re: Profiling.sampling – Statistical Profiler

#21

Post by Intralexical »

add-sub-mul-div wrote: post #19
thomasmg wrote: post #5

I'm not trying to criticize, but Python is known to be much slower than eg. Java or Go etc. So for performance-critcal code, why use Python? I find Python to be very good because it is concise and simple, but I have not used it for production so far.

In production what's making your application slow is extremely unlikely to be the python code. It's going to be I/O, the threading/concurrency architecture, other mistakes or inefficiency that can be cleared up without leaving the ecosystem. The question of fast vs. slow languages doesn't make a lot of sense to entertain before you have any context of the specific needs of the application or use case. On its own it's…

With the obvious caveat that low-level game engine, image/video processing, numerical code etc. isn't really viable in Python. But outside of that, it's fast enough for gluing together other code that's doing the heavy lifting.

Re: Profiling.sampling – Statistical Profiler

#22

Post by thomasmg »

gwking wrote: post #15
thomasmg wrote: post #5

I'm not trying to criticize, but Python is known to be much slower than eg. Java or Go etc. So for performance-critcal code, why use Python? I find Python to be very good because it is concise and simple, but I have not used it for production so far.

The simple answer is, I choose to use Python because I am productive with it. I get a lot done compared to the other languages I have tried. Performance is almost never the limiting factor in my work, nor has it been for the vast majority of the work I've ever been witness to. When it is, it comes up in very particular circumstances, and can usually be fixed algorithmically. Indeed, that is the situation where I have…

> I choose to use Python because I am productive with it.

That, I fully understand. I think many developers are productive in one language: in the one that they know best. Which is probably the one they use most. It might happen that this is is a "fast" language by accident (like Java or Go), or a language like Python. And then there is never enough reason to switch.

> "Most engineers would kill for a 5% speedup"

I think this is very rare - maybe a heavily used app in Facebook or Google, where 5% could mean a lot of money. But a factor of 10 speedup is much more common (and possible sometimes).

> there is an allure to performance optimization due to the fact that it can be so easily quantified.

That's true. I also think simplicity is quantifiable, and so my personal hobby is to write something impressive in few lines. Like a chess engine, QR code reader, editor, data compression tool, compiler, in 500 lines. But this is mostly for hobbies I guess. For work, it's mostly about features, and then performance, I guess.

Re: Profiling.sampling – Statistical Profiler

#23

Post by add-sub-mul-div »

Intralexical wrote: post #21
add-sub-mul-div wrote: post #19

Earlier quoted context omitted.

In production what's making your application slow is extremely unlikely to be the python code. It's going to be I/O, the threading/concurrency architecture, other mistakes or inefficiency that can be cleared up without leaving the ecosystem. The question of fast vs. slow languages doesn't make a lot of sense to entertain before you have any context of the specific needs of the application or use case. On its own it's…

With the obvious caveat that low-level game engine, image/video processing, numerical code etc. isn't really viable in Python. But outside of that, it's fast enough for gluing together other code that's doing the heavy lifting.

Right, if you're writing an Xbox game for example you wouldn't go with Python but if you're in the range of use cases where a language isn't fully disqualified then the pure speed of the language itself rarely needs to guide the decision.

Re: Profiling.sampling – Statistical Profiler

#24

Post by nomel »

thomasmg wrote: post #5

I'm not trying to criticize, but Python is known to be much slower than eg. Java or Go etc. So for performance-critcal code, why use Python? I find Python to be very good because it is concise and simple, but I have not used it for production so far.

It's super easy. If you collaborate with non-software people (say, in hardware engineering), there's not many good alternatives. I've seen attempts at DSLs, Lua, and Perl, C (disaster) but Python lets anyone dig down and understand/debug.

Re: Profiling.sampling – Statistical Profiler

#25

Post by zephen »

thomasmg wrote: post #22
gwking wrote: post #15

Earlier quoted context omitted.

The simple answer is, I choose to use Python because I am productive with it. I get a lot done compared to the other languages I have tried. Performance is almost never the limiting factor in my work, nor has it been for the vast majority of the work I've ever been witness to. When it is, it comes up in very particular circumstances, and can usually be fixed algorithmically. Indeed, that is the situation where I have…

> I choose to use Python because I am productive with it. That, I fully understand. I think many developers are productive in one language: in the one that they know best. Which is probably the one they use most. It might happen that this is is a "fast" language by accident (like Java or Go), or a language like Python. And then there is never enough reason to switch. > "Most engineers would kill for a 5% speedup" I t…

> I think many developers are productive in one language: in the one that they know best.

While superficially true, this conflates cause and effect. I am, or was at one time, extremely proficient in multiple assembly languages, Fortran, C, Pascal, Modula-2, Verilog, and Python, marginally proficient in multiple other assembly languages, Perl, Tcl, etc., and have toyed with various lisps and functional programming languages.

I viscerally recoiled from a few languages, like Java, Javascript, and C++, and found other languages, like Go, Lua and Julia, not compelling enough to bother switching to for my use cases.

And while I can certainly believe that many people are most productive in the language they know best, that most emphatically doesn't mean that they learned a language and became proficient in it to the exclusion of others. It most certainly meant in my case that I spent more time with, and learned ever more about, a language that was already productive and useful for me.

Re: Profiling.sampling – Statistical Profiler

#26

Post by tirutiru »

thomasmg wrote: post #5

I'm not trying to criticize, but Python is known to be much slower than eg. Java or Go etc. So for performance-critcal code, why use Python? I find Python to be very good because it is concise and simple, but I have not used it for production so far.

I wrote a logistics optimization program (think travelling-salesman++) in Python and it gradually took over from the v1 version written in Java.

People generally compare "speed of execution" to "speed of coding".

I'll add "speed of troubleshooting". When something goes wrong and the clock is ticking, you need tools to identify quickly if the data is inconsistent, a config set wrongly or the algorithm stuck at a local optimum. And then decide if things can be mitigated with a patch or we should we grovel and buy time from the client... All of this is much easier in Python (bar the grovelling).

LLMs are changing things though. I recently ported a complex part of the kernel to C++. 90% of it was creating the scaffolding before the actual algorithms were ported. Very tedious work but Claude can just chug through. Wouldn't like to maintain it just yet.

Post reply on HN