Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python’s Weak Performance Matters

metarabbit.wordpress.com

Post reply on HN
Print view View original

21–30 of 336 posts

Re: Python’s Weak Performance Matters

#21

Post by nightski »

gameswithgo wrote: post #18
thinkpad20 wrote: post #4

Earlier quoted context omitted.

How so? Haskell has a very high performance ceiling.

it does have a high perf ceiling, but the code isn't easy to write when you approach it. Things like C#, F#, Java, Kotlin, Nim, Lua would be more natural things to turn to when you want something "Easy" like python but faster, I think.

I think you could replace "isn't easy" with "isn't familiar". Haskell is very likely to be the first language a developer encounters which is lazy by default instead of strict by default (for many good reasons).

So no, it's not easy in a similar fashion that pointers or double pointers in C/C++ are not easy. Or understanding call by value vs. call by reference semantics are not easy. The list goes on. It's probably the largest barrier to learning the language.

But once you get a handle on the evaluation model it becomes a lot more natural. At least that was my experience, maybe it is not typical.

Re: Python’s Weak Performance Matters

#22

Post by gh02t »

pletnes wrote: post #10

There’s a project to plug different JIT compilers into CPython, so there’s hope. https://github.com/Microsoft/Pyjion/blob/master/README.md Also, I’ve more than once seen cpython beat C++/Fortran since it’s easier to do the right algo/datastructure things, plus numpy is more optimized than most «amateur» C loop-over-arrays. That being said, faster python is always welcome.

Honestly, NumPy is gonna be hard to beat even for someone knowledgeable in certain use cases, especially ones where the overhead in Python is trumped by time spent in library calls. It's the same reason that it's hard to beat MATLAB or Mathematica in cases they are optimized for despite being relatively slow languages. They are calling some of the most heavily optimized libraries in existence (e.g., BLAS) and using heuristics to help choose the smartest evaluation strategy.

Edit: More speed on the Python side is good though, because it gives you flexibility. Sometimes it's hard to figure out how to do stuff optimally in NumPy, versus just banging things out in a for loop. I've definitely done that when I wanted something to just work, versus spending an hour figuring out what arcane incantation I need to pass to np.einsum to get the operation I want.

Re: Python’s Weak Performance Matters

#23

Post by Pxtl »

woolvalley wrote: post #15
indubitable wrote: post #5

Something I often wonder in these sort of discussions is why C# is generally omitted. Its performance is comparable to C++, with none of the trappings. It also does an excellent job of integrating some of the most useful features of functional programming into an imperative language. And multi-processor programming with the language is also incredibly simple. But I think the best part is in programmer time. An anecdo…

It has been for the longest time been a closed source MSFT only thing. It wasn't open source and running on linux was a second class citizen. Not sure if it is still a second class citizen. It's also seen as something fairly heavyweight to write things in, such as Java. You probably don't see it used much for the same reasons why Java isn't used.

Honestly, I love C#, but if I was looking for a sane language with a wide library to draw from that matches as many use-cases as possible, I'd probably do Kotlin. Java has a solid community for just about everything, and Kotlin is close-enough to c# for my taste. But Kotlin doesn't have much name recognition outside of Android.

Re: Python’s Weak Performance Matters

#25

Post by joshuamorton »

civility wrote: post #14
hyperion2010 wrote: post #7

I recently discovered that pypy3 can run all my day to day Python code. It has some issues with slightly different behavior from cpython when using threads but other than that I see a 4x speedup on most of my slowest pure python workloads (parsing large rdf files and reserializing them after computing a total order on all their nodes). Huge win for productivity.

> I see a 4x speedup on most of my slowest pure python workloads Heh, only 25..250 X to go. We did a direct line for line translation of some numerically intensive code from Python to C++ and saw a literal 1000X speedup. On other projects, it's been more like 100X slower. That says two things: first Python can be really slow, second, for some programs, Python doesn't really save on lines of code over modern C++. I've…

While I'm more of a pythonist than a C-ist, hearing "1000x speedup" and "line for line" to me implies that you aren't writing idiomatic python. Idiomatic python is (often) faster than not, and (often) more difficult to translate to lower level languages.

As a simple example, list-comprehensions are faster than loops, and can't be line for line translated into C++.

Re: Python’s Weak Performance Matters

#27

Post by dfinninger »

Insanity wrote: post #3

does this not come down to using the right tool for the job? I love writing in Python but would not use it for something where performance matters.

I'm with you here. I tend to use Python, but if I need raw speed/concurrency I'll reach for something else. Currently Go.

Re: Python’s Weak Performance Matters

#28

Post by icey »

pletnes wrote: post #12
indubitable wrote: post #5

Something I often wonder in these sort of discussions is why C# is generally omitted. Its performance is comparable to C++, with none of the trappings. It also does an excellent job of integrating some of the most useful features of functional programming into an imperative language. And multi-processor programming with the language is also incredibly simple. But I think the best part is in programmer time. An anecdo…

The author is a scientist analyzing his data. I never met anyone in that crowd using C#. Are there even any good data science/numerics libs out there? C++ has a lot of number crunching libs, python even more.

It's probably more common for data scientists in the .NET world to use F#. FsLab (https://fslab.org/) seems cool from some limited experimentation (disclaimer: I'm not a data scientist, but am data curious).

Re: Python’s Weak Performance Matters

#29

Post by hidenotslide »

I don't find this a very compelling argument. The author doesn't mention any attempts to profile or speed up the code.

Specifically with pandas I've found if you aren't careful you can do a lot of unnecessary copying. Not sure if that's what is going on here, but cProfile can help find the bottlenecks.

Re: Python’s Weak Performance Matters

#30

Post by psyklic »

civility wrote: post #14
hyperion2010 wrote: post #7

I recently discovered that pypy3 can run all my day to day Python code. It has some issues with slightly different behavior from cpython when using threads but other than that I see a 4x speedup on most of my slowest pure python workloads (parsing large rdf files and reserializing them after computing a total order on all their nodes). Huge win for productivity.

> I see a 4x speedup on most of my slowest pure python workloads Heh, only 25..250 X to go. We did a direct line for line translation of some numerically intensive code from Python to C++ and saw a literal 1000X speedup. On other projects, it's been more like 100X slower. That says two things: first Python can be really slow, second, for some programs, Python doesn't really save on lines of code over modern C++. I've…

Confirmed. Earlier today I rewrote some (non-NumPy) Python numerical sampling code in C and it was a 100x speedup.
Post reply on HN