Live data from Hacker News

Python’s Weak Performance Matters

metarabbit.wordpress.com

81–90 of 336 posts

Re: Python’s Weak Performance Matters

#81

Earlier quoted context omitted.

OP here. Speed is the main motivation, but total time is TimeToWriteCode + TimeToRunCode. Python has the lowest TimeToWriteCode, but very high TimeToRunCode. C++ has lowest TimeToRunCode, but high TimeTowWriteCode. Haskell is often a good compromise for me. Also, with Haskell, it can be very easy to take advantage of 20 CPU cores, while I don't have as much familiarity with high-level C++ threading libraries.

If you write more C++ than python, it will have a lower TimeToWriteCode. Despite having spent years writing python I don't find it any more productive than C++. C++11 has all the nice features you might expect from python with the only drawback being the lack of a REPL.

The lack of REPL compounds with long compilation times, which is practically a feature of C++ and not going to go away anytime soon. The effect is that, when you explore a new API or need to tune parameters to some function call deep in the call stack, you're an order of magnitude slower than with Python (or Lisp, Scala, F#, Haskell, or even Nim or plain C (b/c compilation times)).

If you know exactly what you need to write, you're just as quick in C++ as in Python, that's true. Programming is mostly about learning what to write, though, and here C++ loses.

Re: Python’s Weak Performance Matters

#82
post #52

Earlier quoted context omitted.

OP here. Speed is the main motivation, but total time is TimeToWriteCode + TimeToRunCode. Python has the lowest TimeToWriteCode, but very high TimeToRunCode. C++ has lowest TimeToRunCode, but high TimeTowWriteCode. Haskell is often a good compromise for me. Also, with Haskell, it can be very easy to take advantage of 20 CPU cores, while I don't have as much familiarity with high-level C++ threading libraries.

That’s easy with python, too, in a lot of number crunching cases. Numpy with MKL will use all your cores, as will e.g dask and other libraries built on numpy. Farming out embarassingly parallel work to threads or processes is also easy.

Or use languages where you don't have to these extreme workarounds for what should happen by default.

Re: Python’s Weak Performance Matters

#83

Earlier quoted context omitted.

If you write more C++ than python, it will have a lower TimeToWriteCode. Despite having spent years writing python I don't find it any more productive than C++. C++11 has all the nice features you might expect from python with the only drawback being the lack of a REPL.

The lack of REPL compounds with long compilation times, which is practically a feature of C++ and not going to go away anytime soon. The effect is that, when you explore a new API or need to tune parameters to some function call deep in the call stack, you're an order of magnitude slower than with Python (or Lisp, Scala, F#, Haskell, or even Nim or plain C (b/c compilation times)). If you know exactly what you need t…

If you are developing your code as a small tiny functions getting stitched later. Then writing unit test cases will solve this problem too.

Re: Python’s Weak Performance Matters

#84
post #3

does this not come down to using the right tool for the job? I love writing in Python but would not use it for something where performance matters.

Exactly. For web dev stuff, the database is usually the bottleneck. Caching is the next lace to put your efforts. I think I have had one problem where Python was too slow.

Re: Python’s Weak Performance Matters

#85

> The result is that I find myself doing more and more things in Haskell, which lets me write high-level code with decent performance (still slower than what I get if I go all the way down to C++, but with very good libraries). This strikes me as an odd conclusion to come to if speed was the main motivator.

OP here. Speed is the main motivation, but total time is TimeToWriteCode + TimeToRunCode. Python has the lowest TimeToWriteCode, but very high TimeToRunCode. C++ has lowest TimeToRunCode, but high TimeTowWriteCode. Haskell is often a good compromise for me. Also, with Haskell, it can be very easy to take advantage of 20 CPU cores, while I don't have as much familiarity with high-level C++ threading libraries.

@ the OP - not to sound hostile, but you write code (like in the example here [1]) that is bound to be slow, just from a glance at it. vstacking, munging with pandas indices (and pandas in general), etc; in order for it to be fast, you want pure numpy, with as little allocations happening as possible. I help my coworkers “make things faster” with snippets like this all the time.

If you provide me with a self-contained code example (with data required to run it) that is “too slow”, I’d be willing to try and optimise it to support my point above.

Also, have you tried Numba? It maybe a matter of just applying a “@jit” decorator and restructuring your code a bit in which case it may get magically boosted a few hundred times in speed.

[1] https://git.embl.de/costea/metaSNV/blob/master/metaSNV_post....

Re: Python’s Weak Performance Matters

#87
post #61

Earlier quoted context omitted.

Why do you think it's been closed source with minimal cross platform effort? Most of everything from Microsoft related to C# is open source. This includes their compiler, the runtime, and the framework and libraries. And it's been this way since 2014. The .NET standard itself has always been open and cross platform implementations like Mono go back to 2004! I'm not sure what you mean by 'heavyweight.' Its performance…

I know little on this topic, but as you were looking for some feedback I thought I would response. Because the Microsoft version was closed source for a long time? It was released in 2000. You give 2014 as the open source date, which was only 4 years ago. Woolvalley was referring to the implementation, so bringing up the specification is not that relevant. The ECMA specification does (did?) not include ASP.NET, ADO.N…

Much appreciated!

One clarification, since my own language was sloppy. C# did not get open sourced in 2014. That was the date that Microsoft open sourced just about everything (and that's a weasel word there - to my knowledge, it is everything) that wasn't already open source. Things like ADO/ASP/Windows Forms/etc were open sourced back in 2008, along with Microsoft's implementation of their framework libraries, which Mono rapidly integrated into their implementation.

And so on that note, much of that information on the Wiki page is outdated by at least a decade. Mono, especially since their 2.0 release (back in 2008), has been a fully fleshed out, cross platform, production ready alternative to Microsoft's implementation. So I certainly wouldn't call C# a second class citizen on Linux by any means. The one and only reason Linux is not my primary development platform is the lack of Visual Studio.

Ultimately C# started out pretty awful. The language was a mediocre java clone, performance was abysmal, and the language itself was lacking in features. But that changed relatively rapidly, and certainly today it bears little resemblance to where it started. And I think that is perhaps the problem, people seem to think C# of today is the C# of 2004, but I don't understand why that degree of misinformation is so strongly exemplified in this particular language. I think it's particularly a shame because of what a successful tool the language has evolved into.

Re: Python’s Weak Performance Matters

#88
post #59
post #11

I don't think it's true to say that Python's core developers are uninterested in performance. Speeding up Python is a hard problem. He mentions PyPy but even that has only managed modest performance gains in some areas (and not without tradeoffs). He suggests JavaScript as a comparison but doesn't elaborate on how they're comparable beyond the superficial (they're both dynamic scripting languages). I get that he's fr…

Common Lisp, Dylan and Smalltalk are as dynamic as Python, yet they all enjoy of good quality native code compilers (AOT/JIT).

To be honest, Dylan is not exactly alive anymore. CL and Smalltalk still have commercial vendors providing implementations, Dylan has currently two implementations, but one of those (Gwydion) is completely neglected and the other (OpenDylan) has maybe 5 developers working on it in their spare time.

It's a shame because even with its verbose syntax Dylan is a nice language, with a module system, with object system based on multimethods, with hygienic macros and, as you noted, an AOT compiler. It could be huge. We could have ended with Dylan in the place of Java and we'd be better off with that. I consider Dylan one of the biggest missed opportunities in PL space.

Re: Python’s Weak Performance Matters

#89
post #74

Earlier quoted context omitted.

Anytime someone brings up Julia, I think of Dan Luu's review of the language: https://danluu.com/julialang/

From 4 years ago... a lot has changed since. Consider this recent post for example: https://tk3369.wordpress.com/2018/02/04/an-updated-analysis-...

The linked article doesn't attempt to refute the significant claims from Dan's article (which has an update from a year later, so 2015, at the bottom):

1. That the language is (was) undertested, and as a result, full of easy to run into bugs

2. The language makes it easy to ignore errors

3. APIs are inconsistent

4. The head branch isn't kept build-clean (i.e., often it fails to build)

5. Code is often undocumented, with non-descriptive naming. The combination makes it difficult to understand what code is doing.

Dan also writes that he expects the performance issues would be fixed and those aren't what concerns him:

> The purely technical problems, like slow load times or the package manager, are being fixed or will be fixed, so there’s not much to say there.

And bonus community problem:

> Update: this post was edited a bit to remove a sentence about how friendly the Julia community is since that no longer seemed appropriate in light of recent private and semi-private communications from one of the co-creators of Julia. They were, by far, the nastiest and most dishonest responses I’ve ever gotten to any blog post. Some of those responses were on a private discussion channel; multiple people later talked to me about how shocked they were at the sheer meanness and dishonesty of the responses. Oh, and there’s also the public mailing list. The responses there weren’t in the same league, but even so, I didn’t stick around long since I unsubscribed when one the Julia co-creators responded with something bad enough that it prompted someone else to to suggest sticking to the facts and avoiding attacks.

Now, I don't know if any or how many of those issues have been fixed since 2014-2015. But the 2018 blog post about microbenchmark improvements doesn't really address these concerns.

Re: Python’s Weak Performance Matters

#90

Earlier quoted context omitted.

OP here. Speed is the main motivation, but total time is TimeToWriteCode + TimeToRunCode. Python has the lowest TimeToWriteCode, but very high TimeToRunCode. C++ has lowest TimeToRunCode, but high TimeTowWriteCode. Haskell is often a good compromise for me. Also, with Haskell, it can be very easy to take advantage of 20 CPU cores, while I don't have as much familiarity with high-level C++ threading libraries.

If you write more C++ than python, it will have a lower TimeToWriteCode. Despite having spent years writing python I don't find it any more productive than C++. C++11 has all the nice features you might expect from python with the only drawback being the lack of a REPL.

There is a C++ REPL: https://root.cern.ch/cling
Post reply on HN