Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python’s Weak Performance Matters

metarabbit.wordpress.com

Post reply on HN
Print view View original

281–290 of 336 posts

Re: Python’s Weak Performance Matters

#281

Post by dragonwriter »

apendleton wrote: post #278
jakear wrote: post #238

Earlier quoted context omitted.

To me the last sentence nullifies the entire argument. People doing a lot of numeric computations aren't (typically) doing it all in Python. They write critical sections in C (or some other "fast" language), or use existing libraries that already have done so. If they aren't writing those sections in C already, perhaps because they don't know how, or it would take too long to do right, then why would they ever choose…

Firstly, not everyone doing performance-sensitive work is doing numeric work (that seems to have been a motivator for writing this article), so numpy isn't always practical. Secondly, I think the "if it's slow just rewrite the hard parts in C" is generally out of step with more modern options. Python gets you a really nice environment that's super pleasant to work with... until suddenly it doesn't and you're backed i…

> Secondly, I think the "if it's slow just rewrite the hard parts in C" is generally out of step with more modern options.

Right, including modern options like “use Cython”, which opens up C-like power and performance within the Python ecosystem and while maintaining Python ergonomics (because Cython is both a Python language superset and had tooling integrated with Python's distutils, etc., tooling.)

Re: Python’s Weak Performance Matters

#282

Post by digikata »

cbcoutinho wrote: post #245
dahart wrote: post #221

> It is true that programmer time is more valuable than computer time, but waiting for results to finish computing is also a waste of my time (I suppose I could do something else in the meanwhile, but context switches are such a killer of my performance that I often just wait). In film CG production, we had a rule of thumb. If running an interactive program takes longer than about ten seconds, the artist (user) becom…

Somewhat tangential to this, but I remember reading an article on HN about a group creating some web app that was constrained by some size limit (50 KB or so). Groups ended up putting 'dead code' in their projects to guard against other groups taking their space while they were working on their feature. I think this made it so the app never got less than 50... Did you ever see something similar where devs put in some…

It's typical in mil & aviation software to get specs with a required 50% cpu & memory reserve in anticipation of updates in the maintenance window (which could be shown as decades for many products).

Re: Python’s Weak Performance Matters

#283

Post by eesmith »

indubitable wrote: post #253
eesmith wrote: post #197

Earlier quoted context omitted.

You asked for possible reasons for why you are being downvoted for your reply to the following comment by woolvalley: "It has been for the longest time been a closed source MSFT only thing. It wasn't open source and running on linux was a second class citizen. Not sure if it is still a second class citizen." Your reply was: "Why do you think it's been closed source with minimal cross platform effort?" I don't think y…

In hindsight, something I think I should have emphasized in our discussion earlier is the exact state of Mono. Everything else is mostly tangential as 'the Mono question' essentially closes the discussion of whether C# was a 'mostly closed source Microsoft only thing.' And I think the evidence is bountiful and evident that Mono has long since been a very well developed production ready environment. Just listing a few…

Yes, that would have been appropriate as woolvalley specifically brought up Microsoft, and your reply talked a lot about Microsoft's source available release as if it were meaningful counter-example to woolvalley's reference to "closed source."

You wondered why you got downvotes? That's why.

I have no dog in this race.

If you "personally do agree" then why do you bring up an argument that you disagree with, in order to justify your views? It comes across as if you are making the argument to win some sort of rhetorical point, where the end - your advocacy of C# - justifies any tactic.

I do not care to follow up on this discussion any longer.

Re: Python’s Weak Performance Matters

#284

Post by indubitable »

eesmith wrote: post #283
indubitable wrote: post #253

Earlier quoted context omitted.

In hindsight, something I think I should have emphasized in our discussion earlier is the exact state of Mono. Everything else is mostly tangential as 'the Mono question' essentially closes the discussion of whether C# was a 'mostly closed source Microsoft only thing.' And I think the evidence is bountiful and evident that Mono has long since been a very well developed production ready environment. Just listing a few…

Yes, that would have been appropriate as woolvalley specifically brought up Microsoft, and your reply talked a lot about Microsoft's source available release as if it were meaningful counter-example to woolvalley's reference to "closed source." You wondered why you got downvotes? That's why. I have no dog in this race. If you "personally do agree" then why do you bring up an argument that you disagree with, in order…

Mono is a product of the open source nature of C#, not the cause. In general, I think its wise to go as close to the 'first party reason' as possible. But the nuance there makes it surprisingly intricate, and nuance is often lost in online discussion...

As for '4 years being a long time in software', there's a difference between considering the industry at large, and individual experience. I, like everybody, have anecdotal experience and opinions that run contrary to the norm. And in this case my personal view is that 4 years is not really a long time in terms of software, yet for the industry and people as a whole, I think that couldn't be further from the average truth.

Sure, been a fun discussion!

Re: Python’s Weak Performance Matters

#285

Post by pletnes »

VHRanger wrote: post #171
fxmc wrote: post #103

Earlier quoted context omitted.

Have you given dask a try? It gives you out-of-core arrays with numpy semantics and distributed computing.

Dask doesn't solve that problem since it's a wrapper around pandas functions. If you can't make the core pandas code decently fast, dask won't save you.

There’s dask.array which works on numpy arrays instead of dataframes. Otherwise, your argument holds.

Re: Python’s Weak Performance Matters

#286

Post by fafhrd91 »

deathanatos wrote: post #262
fafhrd91 wrote: post #176

Earlier quoted context omitted.

Python extension doesn’t mean C. Rust works perfectly for extensions, it covers a lot of low level c-api integration and it is fast. You can write whole application in rust and use python as a glue language https://github.com/PyO3/pyo3 Pyo3 library gives you ability to work both diractions. Call python code from rust and call rust code from python.

PyO3 is a fork of rust-cpython, which has a nasty abort issue[1] which is unfortunately a show-stopper for me. It isn't clear to me if PyO3 is also affected by this issue. [1]: https://github.com/dgrunwald/rust-cpython/issues/59

Pyo3 is not affected by this issue. pyo3 compiles in c-api interface, it doesn't use separate libs for that (python27-sys)

Re: Python’s Weak Performance Matters

#287

Post by stuaxo »

smoe wrote: post #231
skywhopper wrote: post #198

The quoted argument that "easy to write but slow languages are better because programmer time is far more costly than CPU speed" was pretty common, and I honestly think correct, 10-15 years ago. But things have changed. CPU performance long ago hit physical limits, and more and more we are scaling out applications across hundreds, thousands, or millions of servers. We've passed the inflection point where CPU speed re…

How many people are there who actually need to scale an application across hundreds, thousands, or millions of servers vs the ones that think they are going to need it? I agree that at some point CPU speed is more expensive than programming time, but how many applications in the wild are actually at or beyond that point? I would be surprised if it is more than a couple of percent. Sure if I know that a product has to…

I'd quite like to be able to write some plain python (not PIL or numpy or Cython or whatever) to change all the pixels in an image at a similar speed to say Java.

Re: Python’s Weak Performance Matters

#288

Post by ChrisRackauckas »

ptype wrote: post #106

I love Python. The language is a joy, the eco system is fantastic. But yes, let’s be honest, if you can not vectorise your code it is slow, and I think that will be its downfall eventually. I’m excited about Julia, I hope it gains popularity and the eco system grows. Until then, and in particular until the data frames story can compete with pandas, it Python with Cython for me, but I’d rather skip the Cython if it wa…

I'm using Julia and loving it. I've built a bunch of differential equation solvers which routinely outperform the classic C++/Fortran codes. I started out without "software development" experience but Julia and its community got me up to speed and helped me build something quite unique. Now Julia is the only language that has the numerical libraries I need to do my research.

In fact, the whole library story in Python/MATLAB is quite overblown. If you're doing something which is actually new, like PhD methods research, you need to be writing a lot of stuff from scratch. And in that case, you usually cannot get by with vectorizing everything... and vectorization always has the issue with temporary arrays too. Meanwhile, Julia's type system makes everything fast (which is a plus when trying to publish a paper on it!) but also get cool extra features for free like GPU support and arbitrary precision. For people developing and testing new methods, Julia is the best tool right now.

Re: Python’s Weak Performance Matters

#289

Post by ChrisRackauckas »

Certhas wrote: post #127
ivanb wrote: post #47

I have an impression that there are features in Python that give very little programmer productivity but make the language slow. It should be possible to implement a hypothetical FastPython without such features but with great performance gains. Of course it wouldn't be compatible with most of the libraries. I can imagine though that porting most of the libraries to FastPython still would be a manageable task. I wond…

Julia goes in that direction.

I'd go even further and say Julia is the instantiation of that idea.

Re: Python’s Weak Performance Matters

#290

Post by narimiran »

lmm wrote: post #213
narimiran wrote: post #102

If you would like 10-100x faster performance than Python, but would like to keep the easy-to-read code, give Nim [0] a try. I do all my work in Python, and I've been using Nim in last couple of months - it took me a week or two until I was able to be productive in Nim. Don't expect Python's large ecosystem, nor some Python goodies, but if you're looking for a readable, writable, high-performance post-Python language…

Why Nim rather than e.g. Haskell (mentioned in the article) or OCaml, which are much more mature and have much bigger, more established tool/library ecosystems?

> Why Nim rather than e.g. Haskell?

Because Nim syntax will be familiar to Python developer. Sometimes all you need to change is add variable declarations and rename `def` to `proc`.

Haskell has a much steeper learning curve. Been there, struggled with that. If I would recommend a functional language to Python developer, I would go with F#.

Post reply on HN