Live data from Hacker News

Yes, Python is Slow, and I Don’t Care

hackernoon.com

51–60 of 206 posts

Re: Yes, Python is Slow, and I Don’t Care

#51

I was all ready to savage his opinion after reading the headline but I agree looking at my architecture that I designed for the company I work for, CPU isn't the bottleneck. Every time I try to increase performance by multi threading as much as possible, the databases start screaming. On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages…

MyPy provides some useful type-checking for python. I have not had a chance to use it in production code because we are still stuck on python 2 due to Google AppEngine standard not supporting python 3.

http://mypy.readthedocs.io/en/latest/python2.html

You can use comment annotations to typecheck with mypy in py2

Re: Yes, Python is Slow, and I Don’t Care

#52
post #16
post #12

The fact that Python is slow isn't its only problem. What I care more about nowadays is wasting my time hunting bugs that could have been avoided by a static type system.

Please tell me, do you write tests? I've learned that it's necessary. I do 100% code coverage and I'm enjoying TDD a lot.

100% coverage is meaningless with respect to correctness and the ability to prove facts about your code. 100% coverage only means that you've been able to exercise all relevant lines of code at least once, not that you've tested all possible execution paths through the code.

The idea that you could somehow replace the utility of static type checking with a suite of human written tests is laughable. Static typing systems allow you to prove facts about your code which make reasoning about correctness much easier than with dynamic type systems which practically disallow this (unless gradual typing is allowed in your language).

Re: Yes, Python is Slow, and I Don’t Care

#53
post #29

Unless we are talking like circa 1999 I don't think I have heard a complaint yet that Python is slow. I'm curious who or where the author heard that from (not specifically the people themselves but the domain they are in). What I have heard complaints about Python are (and I don't agree with all these points): * Its not statically typed * The python 2/3 compatibility * It has some design flaws: GIL, variable assignin…

For some data processing tasks Python can be brutally slow, especially text processing. NumPy is only fast because it's written in C and offloads hard numerical calculations to BLAS.

Re: Yes, Python is Slow, and I Don’t Care

#54
post #29

Unless we are talking like circa 1999 I don't think I have heard a complaint yet that Python is slow. I'm curious who or where the author heard that from (not specifically the people themselves but the domain they are in). What I have heard complaints about Python are (and I don't agree with all these points): * Its not statically typed * The python 2/3 compatibility * It has some design flaws: GIL, variable assignin…

> I'm curious who or where the author heard that from (not specifically the people themselves but the domain they are in).

In the telecom domain, I've dealt with data big enough that Python wasn't really feasible. Think 100 of millions of records in CSV format that need to be parsed and processed. Doing that in Python is going to be painful.

Re: Yes, Python is Slow, and I Don’t Care

#55

There's are still some big gains python could make, if python implementations were better. Micropython is equivalent to a real-time cooperative-multitasking OS. If it had ~~better~~ support for things like cffi, you could implement posix on top of it. I can imagine a laptop that runs gnu+python in the next few years. That's a whole new usecase, simply because that implementation uses a lot less ram. What usecases wou…

"There's are still some big gains python could make, if python implementations were better."

At this point, I would find it far easier to believe that you are underestimating the difficulty involved in what it takes to speed up Python than that there are enormous gains yet to be had in speeding up Python. I suspect JS has had more optimization effort expended overall, but Python has still had a ton of work by lots of smart people, and generally got an earlier head start on optimization. (They didn't start trying to make JS "fast" right away; they spent rather a lot of time getting JS's hookup to the DOM in the face of things like .innerHTML working first, before anyone even cared to do what we today do routinely without thought in plain ol' Javascript, let alone with our glorious frameworks.)

There are enormous gains to be had in speeding up "a language that is like Python except certain things are banned", but people have already done that analysis too and discovered that broadly speaking, if you do that, too much existing Python breaks. If you want to see something like that, check out the RPython aspect of the PyPy project, which successfully implements a fast subset of Python. But it is a noticeably restricted subset of Python; AIUI it's not even close to something you can just drop in to your code and get faster speeds.

One of the things that I've learned from Python and the other attempts to speed up the scripting languages is that despite the mantra, yes, there is in practice such a thing as an intrinsically slow language. (The theoretical existence of a Python interpreter that can run all existing Python code at C speeds doesn't do us much good if we have no idea after decades of very smart people banging on the problem how to manifest it. Personally I'd suspect that while such a beast theoretically exists it has an exponential complexity startup cost or, if you prefer, exponential compilation costs. And probably a pretty decent code and/or RAM bloat cost, too.) And Python is one of them. Some of the reasons why it is so much fun to use are part of that intrinsic slowness. Some of them really aren't.

I personally think there's a lot of up-and-coming languages that are exploring the space of how to get those nicer programming abstractions and programmer-convenient code without paying anywhere near the runtime cost that the dynamic scripting languages of today do; it's one of the more exciting developments I see coming up. People complain a lot about code bloat and poor performance of our code since right now we have to choose between "fairly inconvenient but fast" and "convenient but slow and bloated". Patience! Better choices are developing, but they're still young.

Re: Yes, Python is Slow, and I Don’t Care

#56

I was all ready to savage his opinion after reading the headline but I agree looking at my architecture that I designed for the company I work for, CPU isn't the bottleneck. Every time I try to increase performance by multi threading as much as possible, the databases start screaming. On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages…

>On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages prevent a lot of bugs and allow for a lot of automated provably correct refactorings that simple cannot be done with a statically typed languages. You can't even reliably do a "find usages" of classes using a dynamically typed languag

Exactly, I get quick and precise code completion, I catch plenty of errors beforehand etc. I'd say I'm about 10x as productive in C# as in Python, with similar amount of experience. Python only shines when there is a library that does something really well that you need. For me any productivity advantage in Python is from lots and lots of libraries.

Also in terms of maintainability, I find my C# code easy to read and modify a year later when I've forgotten completely about it. In Python I need to rescan all of the types into my head until I can understand what the program does.

I mean with var and dynamic, C# offers everything you need for duck typing efficiency, while preserving the very important statically typed interfaces.

Re: Yes, Python is Slow, and I Don’t Care

#57
post #2

The problem is not so much that python is slow. It's that in some scenarios python can't be made fast. Fast prototyping is great but being stuck with a prototype for deployment isn't.

Did you read the section on "Optimizing Python" ?? If so, I'm assuming you read about using CPython to migrate your package/program/module painlessly to C. So given that, can you elaborate on your objection? I'd like to know why you think the article is wrong about optimising your Python to make it fast enough.

"migrate your package/program/module painlessly to C"

Do you or anyone on your python using team can use C professionally for things you'll use in production?

I work with python everyday on a team that uses python everyday, and I worked professionally in C++ for a >2 years time, and I still wouldn't trust myself to write a reliable/safe C backend to something.

And using C++ behind python with something like pybind11 is anything but painless on the other end; it requires careful consideration.

Python is "good enough" in nearly all cases, but there are some instances where you sort of hate yourself for using it for the prototype since you're forced to effectively rewrite afterwards.

Re: Yes, Python is Slow, and I Don’t Care

#58

I was all ready to savage his opinion after reading the headline but I agree looking at my architecture that I designed for the company I work for, CPU isn't the bottleneck. Every time I try to increase performance by multi threading as much as possible, the databases start screaming. On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages…

You sound like someone who hasn't used dynamic languages in anger, or you'd mention some of the things that dynamic languages do well that statically typed languages aren't so great at, to prevent your argument sounding like a straw man.

For example: dropping into a debugger (binding.pry / pry_remote in Ruby) to write code interactively in the context of the application, transferring that code to the source, and continuing on with my next fragment of functionality with a refresh of the web page, no recompiles or restarts required. You can do this with some difficulty and certain caveats in a statically compiled language, but only if things have been set up in just the right way, with automatic dependency recompilation and reloading, hot code replacement, etc. And even then, there are limitations on the kind of code you can write in the editor, depending on the underlying technology. Typically you can't create whole new classes, or introduce new fields etc.

Or consider levering up your language. Dynamic languages often give you the ability to create "new" syntax via expressive literals. This opens up more avenues for declarative programming; construct a data structure that models your problem more directly, and then write code that interprets the data structure. You want extremely lightweight literals for this, readable with minimal ceremony, including lambdas for when you need an escape hatch, a little pocket of custom code embedded in the bigger data structure. Statically typed languages have little context to infer types for such lambdas, unless they can generalize from the operators applied, but then you have a generic function, not a piece of data. So you end up infecting your DSL with type annotations and cruft, and before you know it the whole thing is hiding the forest behind the trees.

Thing is, if you aren't used to using these tools, you might not even know they exist, and you may be missing out on productivity you didn't know you could have.

Re: Yes, Python is Slow, and I Don’t Care

#59

I was all ready to savage his opinion after reading the headline but I agree looking at my architecture that I designed for the company I work for, CPU isn't the bottleneck. Every time I try to increase performance by multi threading as much as possible, the databases start screaming. On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages…

"the idea that dynamic languages are more productive than static languages are laughable." -- being statically typed or dynamically typed comes with its own set of tradeoffs and what a person is more productive in is a highly subjective matter. Lispers are more productive in Lisp than Haskell and vice versa.

"Statically type languages prevent a lot of bugs and allow for a lot of automated provably correct refactorings that simple cannot be done with a statically typed languages." -- not true, Clojure is trying to do that with Clojure.spec and specifications being checked at runtime can get you closer to things you could have automatically proved correct only with languages with dependent types, nothing against statically typed languages but I feel that your sweeping generalizations hurt the point you are trying to make.

Re: Yes, Python is Slow, and I Don’t Care

#60
post #56

I was all ready to savage his opinion after reading the headline but I agree looking at my architecture that I designed for the company I work for, CPU isn't the bottleneck. Every time I try to increase performance by multi threading as much as possible, the databases start screaming. On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages…

>On the other hand, the idea that dynamic languages are more productive than static languages are laughable. Statically type languages prevent a lot of bugs and allow for a lot of automated provably correct refactorings that simple cannot be done with a statically typed languages. You can't even reliably do a "find usages" of classes using a dynamically typed languag Exactly, I get quick and precise code completion,…

Of course that depends how much more you have to write in one language compared to another. Say you're 10x more productive but have to write 5x code, that sort of cuts down the advantage somewhat.
Post reply on HN