Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python’s Weak Performance Matters

metarabbit.wordpress.com

Post reply on HN
Print view View original

61–70 of 336 posts

Re: Python’s Weak Performance Matters

#61

Post by eesmith »

indubitable wrote: post #31
woolvalley wrote: post #15

Earlier quoted context omitted.

It has been for the longest time been a closed source MSFT only thing. It wasn't open source and running on linux was a second class citizen. Not sure if it is still a second class citizen. It's also seen as something fairly heavyweight to write things in, such as Java. You probably don't see it used much for the same reasons why Java isn't used.

Why do you think it's been closed source with minimal cross platform effort? Most of everything from Microsoft related to C# is open source. This includes their compiler, the runtime, and the framework and libraries. And it's been this way since 2014. The .NET standard itself has always been open and cross platform implementations like Mono go back to 2004! I'm not sure what you mean by 'heavyweight.' Its performance…

I know little on this topic, but as you were looking for some feedback I thought I would response.

Because the Microsoft version was closed source for a long time?

It was released in 2000. You give 2014 as the open source date, which was only 4 years ago.

Woolvalley was referring to the implementation, so bringing up the specification is not that relevant.

The ECMA specification does (did?) not include ASP.NET, ADO.NET, and Windows Forms. The implementation of those APIs were potentially covered under patent, as the Microsoft Community Promise did not apply.

As Wikipedia points out, "These technologies are today[when?] not fully implemented in Mono and not required for developing Mono-applications, they are simply there for developers and users who need full compatibility with the Windows system." https://en.wikipedia.org/wiki/Mono_(software)#Mono_and_Micro... .

While that point is moot today, that is part of the history which guides current views.

If by C# you mean the ECMA specification, then that's different than C# as available for Windows by Microsoft.

Is C# a first-class citizen on Linux comparable to how Visual C# is a first-class citizen on Microsoft Windows, or how C++ is a first-class citizen on Linux? I don't have the experience with that, but it doesn't seem to be the case.

What languages do you consider to be "first-class" and "second-class" on Linux?

Re: Python’s Weak Performance Matters

#63

Post by pjmlp »

pletnes wrote: post #12
indubitable wrote: post #5

Something I often wonder in these sort of discussions is why C# is generally omitted. Its performance is comparable to C++, with none of the trappings. It also does an excellent job of integrating some of the most useful features of functional programming into an imperative language. And multi-processor programming with the language is also incredibly simple. But I think the best part is in programmer time. An anecdo…

The author is a scientist analyzing his data. I never met anyone in that crowd using C#. Are there even any good data science/numerics libs out there? C++ has a lot of number crunching libs, python even more.

Yes there are, IMSL and AleaGPU for starters.

https://www.roguewave.com/products-services/imsl-numerical-l...

http://www.quantalea.com/

.NET is used a lot for data analysis in life sciences, as most laboratory devices only expose COM and .dll interfaces.

Specially for stuff like DNA sequencing, chemical structure manipulation, reaction curves analysis and such.

Re: Python’s Weak Performance Matters

#64

Post by luispedrocoelho »

pletnes wrote: post #52
luispedrocoelho wrote: post #36

Earlier quoted context omitted.

OP here. Speed is the main motivation, but total time is TimeToWriteCode + TimeToRunCode. Python has the lowest TimeToWriteCode, but very high TimeToRunCode. C++ has lowest TimeToRunCode, but high TimeTowWriteCode. Haskell is often a good compromise for me. Also, with Haskell, it can be very easy to take advantage of 20 CPU cores, while I don't have as much familiarity with high-level C++ threading libraries.

That’s easy with python, too, in a lot of number crunching cases. Numpy with MKL will use all your cores, as will e.g dask and other libraries built on numpy. Farming out embarassingly parallel work to threads or processes is also easy.

If I can fit the code into numpy-like structure, then Python is typically fine.

The issue is when I cannot.

Re: Python’s Weak Performance Matters

#65

Post by blub »

indubitable wrote: post #5

Something I often wonder in these sort of discussions is why C# is generally omitted. Its performance is comparable to C++, with none of the trappings. It also does an excellent job of integrating some of the most useful features of functional programming into an imperative language. And multi-processor programming with the language is also incredibly simple. But I think the best part is in programmer time. An anecdo…

C# started out as a boring Java clone enterprise language from the company famous for its "embrace, extend, extinguish" strategy. In fact they invented C# because they got sued for trying to "extend" Java.

And until now no amount of cool features and Microsoft PR were able to remove that stench.

Re: Python’s Weak Performance Matters

#66

Post by munro »

> At the same time, data keeps getting bigger and computers come with more and more cores (which Python cannot easily take advantage of), while single-core performance is only slowly getting better. Thus, Python is a worse and worse solution, performance-wise.

PySpark is makes it really easy to take advantage of multiple cores & machines. Most operations I want to do to my data I can find in PySpark's pyspark.sql.functions, so I get all the benefits of the JVM. In the cases I need something from Python, I can just UDF, it's a little slower than JVM but still extremely fast when distributed--I find all problems come down to time or memory complexity, which is independent to whatever your programming in. Also, it's very easy to take advantage of spot instances with Spark... I'm usually working with 2-20 spot instances, and sometimes go up to 60 depending on what I'm doing.

Re: Python’s Weak Performance Matters

#67

Post by ggm »

To me, the critical quality exposed is the abstraction/synthesis moment of Haskell/types/FP thinking. Python is what I use, but I rely on insights from a Haskell person to get solutions of merit. Left to my own devices I frequently derive python solutions with bad scaling, few and weak opportunitistic parallelism moments and heaps of errors.

When driven to think in types and simple function composition the solutions seem to run better

Re: Python’s Weak Performance Matters

#68

Post by inciampati »

luispedrocoelho wrote: post #36
matt_wulfeck wrote: post #2

> The result is that I find myself doing more and more things in Haskell, which lets me write high-level code with decent performance (still slower than what I get if I go all the way down to C++, but with very good libraries). This strikes me as an odd conclusion to come to if speed was the main motivator.

OP here. Speed is the main motivation, but total time is TimeToWriteCode + TimeToRunCode. Python has the lowest TimeToWriteCode, but very high TimeToRunCode. C++ has lowest TimeToRunCode, but high TimeTowWriteCode. Haskell is often a good compromise for me. Also, with Haskell, it can be very easy to take advantage of 20 CPU cores, while I don't have as much familiarity with high-level C++ threading libraries.

If you write more C++ than python, it will have a lower TimeToWriteCode. Despite having spent years writing python I don't find it any more productive than C++.

C++11 has all the nice features you might expect from python with the only drawback being the lack of a REPL.

Re: Python’s Weak Performance Matters

#69

Post by e12e »

pjmlp wrote: post #57
acqq wrote: post #40

Earlier quoted context omitted.

> Python has a shallower learning curve and an easy way to get “good enough” performance (a bit slower than C). The article we all reply to exactly claims that as soon as you don't use e.g. NumPy, it's not "good enough" anymore, and I agree with that. The article also argues that e.g. JavaScript isn't more in the same category with Python, but much faster, even if it's not less dynamic. I think the reason for JavaScr…

Every time this comes up, there are also the good examples of Lisp, Dylan and Smalltalk as languages that as dynamic as Python, while enjoying relatively good JIT compilers.

Which of course python has too (pypy). Although it depends on what your bar for "relatively good" is.

Re: Python’s Weak Performance Matters

#70

Post by inciampati »

luispedrocoelho wrote: post #36
matt_wulfeck wrote: post #2

> The result is that I find myself doing more and more things in Haskell, which lets me write high-level code with decent performance (still slower than what I get if I go all the way down to C++, but with very good libraries). This strikes me as an odd conclusion to come to if speed was the main motivator.

OP here. Speed is the main motivation, but total time is TimeToWriteCode + TimeToRunCode. Python has the lowest TimeToWriteCode, but very high TimeToRunCode. C++ has lowest TimeToRunCode, but high TimeTowWriteCode. Haskell is often a good compromise for me. Also, with Haskell, it can be very easy to take advantage of 20 CPU cores, while I don't have as much familiarity with high-level C++ threading libraries.

As for high level C++ threading you have OMP. It's incredibly easy to use. In the simplest case you just use a preprocessor directive before a loop to say it should run in parallel. It's probably not as nice as what you get in Haskell because it needs to be done explicitly but it is really easy to use.
Post reply on HN