Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python’s Weak Performance Matters

metarabbit.wordpress.com

Post reply on HN
Print view View original

251–260 of 336 posts

Re: Python’s Weak Performance Matters

#251

Post by ziftface »

gameswithgo wrote: post #196
passive wrote: post #137

This is a weird article at this point in time. The question it addresses: "Does Python's performance matter?" Has always had the answer: "Sometimes, and you have options for those cases." The OP found a "sometimes", and he's using one of those options. In this case, he's got Python for prototyping and glue, with Haskell improving performance. This is as it should be. I don't know of any Python advocates who say it's…

I would argue that performance always matters, and that Python is never the right tool for the job in an absolute sense. Python may be the right tool for the job given the options we have today but there is no reason we cannot have a language exactly as nice to use as Python is, but that also provides good performance. Languages like Nim or F# approximate that ideal, for instance. And while I realize there are high p…

I agree that we can do better. But I don't think you'll ever be able to avoid some trade-off between CPU performance and ease-of-use. We might shrink the gap, but there will always be a market for languages that allow easy prototyping and new developer onboarding, and those languages will always be slower than C++.

Re: Python’s Weak Performance Matters

#252

Post by abuckenheimer »

skywhopper wrote: post #198

The quoted argument that "easy to write but slow languages are better because programmer time is far more costly than CPU speed" was pretty common, and I honestly think correct, 10-15 years ago. But things have changed. CPU performance long ago hit physical limits, and more and more we are scaling out applications across hundreds, thousands, or millions of servers. We've passed the inflection point where CPU speed re…

When people say cpython is slow they are generally pointing to two things

1) The interpreter is _slow_

2) You can't achieve real thread based parallelism

I think in general people have a high level concept of (1) as static vs. dynamic and interpreted vs compiled is an understandable trade off. CPython as an implementation generally gets type casted as slow for its default dynamism which have understandable negative performance implications[1]. However CPython also gives you lots of ergonomic ways to push your program towards the static/compiled end of the spectrum with things like pandas/numpy/numba/extensions. In general using these correctly put you within the ballpark of _faster_ languages, could you write faster assembly by hand? Sure! Is optimizing in another language worth your time? I don't know.

I've never really understood (2) the lack of threading as a problem, multi process parallelism can be accomplished fairly easily if you are CPU bound or projects like uvloop[2] make async tasks fast enough to compete with any web framework out there. Furthermore even though your cpu cycle cost may be levered to a considerable point while operating over hundreds/thousands of servers doing distributed computing right is still hard and developing something like celery/airflow/luigi/dask from scratch is not cheap either. Leaning on CPython's massive ecosystem can massively lower the barrier to entry in a lot of big problems.

I think there are plenty of examples of re-writes in go[3]/rust[4] that have worked great for people, I have no doubt that python is _not_ the end all be all language, but I think the "python is slow" worry is generally overplayed.

Specific problems require specific solutions, I'm glad Haskell seems to work for the author in genetic analysis but think this could have been a more interesting article with some specific python-haskell comparisons rather than the generic python is slow argument.

[1] http://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/

[2] https://magic.io/blog/uvloop-blazing-fast-python-networking/

[3] https://web.archive.org/web/20170101002625/http://blog.parse...

[4] https://blogs.dropbox.com/tech/2016/05/inside-the-magic-pock...

Re: Python’s Weak Performance Matters

#253

Post by indubitable »

eesmith wrote: post #197
indubitable wrote: post #184

Earlier quoted context omitted.

From my perspective here, you're shifting the goal posts. In particular this conversation began with you apparently thinking that Mono lacked full implementations of ASP/ADO/etc. And I think that might justify the original comment that Linux implementations have been treated as second class citizens. I say might since ADO/ASP are not really part of .NET, but rather independent projects built on top of .NET. But in an…

You asked for possible reasons for why you are being downvoted for your reply to the following comment by woolvalley: "It has been for the longest time been a closed source MSFT only thing. It wasn't open source and running on linux was a second class citizen. Not sure if it is still a second class citizen." Your reply was: "Why do you think it's been closed source with minimal cross platform effort?" I don't think y…

In hindsight, something I think I should have emphasized in our discussion earlier is the exact state of Mono. Everything else is mostly tangential as 'the Mono question' essentially closes the discussion of whether C# was a 'mostly closed source Microsoft only thing.' And I think the evidence is bountiful and evident that Mono has long since been a very well developed production ready environment.

Just listing a few projects built on it should really suffice to make the point. The Sims 3 was built using Mono and launched in 2009, Second Life went live with it in early 2008, Unity swapped to it after their initial 'front end' language began, in their words, "proving to be too slow and unwieldy." Their original language, quite appropriate for this topic, was Python! There's really an infinite room for discussion about this, but I think it's all a subset of the 'The Mono Question.' For instance your issue on exactly what open source license is used is completely redundant given Mono.

As for your divergence on 4 years not being a long time. I personally do agree. But our agreement stands in contradiction to the standards of software today. Entire languages are born and die in this span, libraries that didn't exist become ubiquitous, and in general 4 years is certainly far longer than necessary to expect some reasonable evolution of adaptation and opinion. The fact it has not is a peculiarity I find bemusing, and fun to consider. If nothing else, it leads to enjoyable discussion - which I suppose is the ultimate point of these forums.

Re: Python’s Weak Performance Matters

#254

Post by kevin_thibedeau »

_dps wrote: post #234
smitherfield wrote: post #229

Earlier quoted context omitted.

>in Python it's quite difficult even to reason about what's going to be in L1 The interpreter's stack?

I haven't looked at it in a while, so I could be wrong, but I think with small enough programs you can still squeeze some payload into L1 in long tight loops where you're not jumping up and down the Python stack a lot. But your overall point stands: if you're writing non-trivial Python programs your L1 is usually spent on language/runtime overhead.

When such things matter you drop to Cython and avoid interacting with PyObjects. Then you get native performance for tight loops.

Re: Python’s Weak Performance Matters

#255

Post by pfranz »

cbcoutinho wrote: post #245
dahart wrote: post #221

> It is true that programmer time is more valuable than computer time, but waiting for results to finish computing is also a waste of my time (I suppose I could do something else in the meanwhile, but context switches are such a killer of my performance that I often just wait). In film CG production, we had a rule of thumb. If running an interactive program takes longer than about ten seconds, the artist (user) becom…

Somewhat tangential to this, but I remember reading an article on HN about a group creating some web app that was constrained by some size limit (50 KB or so). Groups ended up putting 'dead code' in their projects to guard against other groups taking their space while they were working on their feature. I think this made it so the app never got less than 50... Did you ever see something similar where devs put in some…

In my experience of vfx usually the same author has ownership over the whole tool, so there's no incentive to pad time. We have joked about adding in sleeps and removing them look impressive to users, though.

Games probably have a situation more similar to what you're describing. Many games target 30fps, which is 33.3 milliseconds per frame. Each department usually gets a "budget" of how long they can spend. I've heard similar stories about padding memory usage and time, but it's hard to tell if they were serious and it's not common.

Re: Python’s Weak Performance Matters

#256

Post by cjalmeida »

lmm wrote: post #213
narimiran wrote: post #102

If you would like 10-100x faster performance than Python, but would like to keep the easy-to-read code, give Nim [0] a try. I do all my work in Python, and I've been using Nim in last couple of months - it took me a week or two until I was able to be productive in Nim. Don't expect Python's large ecosystem, nor some Python goodies, but if you're looking for a readable, writable, high-performance post-Python language…

Why Nim rather than e.g. Haskell (mentioned in the article) or OCaml, which are much more mature and have much bigger, more established tool/library ecosystems?

Their syntax is weird for someone coming from C-like langs (Java, C, C++, C#)

Re: Python’s Weak Performance Matters

#257

Post by lmm »

moocowtruck wrote: post #225
lmm wrote: post #203

Earlier quoted context omitted.

> It is a failure of our community that we allow languages to proliferate while remaining slow. This is bad because allowing slow tools to become popular means people create slow things, which wastes other peoples time and energy. Make it work, then make it work right, then make it work fast. I mean yes, a lot of things are slower than they should be, but the level of outright correctness bugs in software today is mi…

i'd like to know how we make correct software

Type systems and languages that make them easy to use. I mean, other approaches are possible, but type systems are something we already all use and understand, and they seem to be enough.

Re: Python’s Weak Performance Matters

#258

Post by Chris2048 »

jakear wrote: post #238
uluyol wrote: post #220

Earlier quoted context omitted.

> If you're a startup and your 100x more performant Go code takes an extra 3-6 months to write I can write Go code nearly as fast as Python, sometimes faster if I need to refactor. Obviously this depends on familiarity with languages, but I think most of difference is probably experience with the language more than anything else. > 100x difference in CPU time is nothing compared to the 1000x loss in a cache hit or a…

To me the last sentence nullifies the entire argument. People doing a lot of numeric computations aren't (typically) doing it all in Python. They write critical sections in C (or some other "fast" language), or use existing libraries that already have done so. If they aren't writing those sections in C already, perhaps because they don't know how, or it would take too long to do right, then why would they ever choose…

Maybe C-devs are easier to find than Python devs with competence in C?

Re: Python’s Weak Performance Matters

#259

Post by 52-6F-62 »

whyever wrote: post #90
inciampati wrote: post #68

Earlier quoted context omitted.

If you write more C++ than python, it will have a lower TimeToWriteCode. Despite having spent years writing python I don't find it any more productive than C++. C++11 has all the nice features you might expect from python with the only drawback being the lack of a REPL.

There is a C++ REPL: https://root.cern.ch/cling

https://repl.it/ also provides C++ (and many more) support in an online version.

Re: Python’s Weak Performance Matters

#260

Post by snowwrestler »

commandlinefan wrote: post #201
skywhopper wrote: post #198

The quoted argument that "easy to write but slow languages are better because programmer time is far more costly than CPU speed" was pretty common, and I honestly think correct, 10-15 years ago. But things have changed. CPU performance long ago hit physical limits, and more and more we are scaling out applications across hundreds, thousands, or millions of servers. We've passed the inflection point where CPU speed re…

I've been "premature optimization is the root of all evil"ed in more code reviews than I can count - yet when people complain about software, they first complain that it's ugly and then they complain that it's slow. Users don't care what it's written in.

If your users are complaining that your software is slow, then optimization would not be premature.

Optimization is premature when the cost of creating or managing it outweighs the business benefits it provides.

Post reply on HN