Live data from Hacker News

Why Python Is Slow: Looking Under the Hood

jakevdp.github.io

101–110 of 156 posts

Re: Why Python Is Slow: Looking Under the Hood

#101
post #56

"it's slow" "just write a C extension" "but then why use Python?"

Because beautiful control structures and a REPL ?

I'm not sure that these are compelling reasons to use Python, given the alternatives.

Squeezing performance out of Python often involves dropping down to C. This isn't the case with the competition.

Re: Why Python Is Slow: Looking Under the Hood

#102
post #91
post #77

Earlier quoted context omitted.

For a general-purpose problem, Perl won't be particularly fast. For a Perl-type problem (scanning and parsing big files), Perl is very fast. Doing a Perl-type problem in a general-purpose language would be considerably slower. However, Python or others will perform much better in the "can I read my own code six months later" benchmark.

I don't know why I always have to chip in on "perl is unreadable lol" comments, but over the last 8 years apart from a steady trickle of C coding here and there the bulk of my dayjob has moved around from C/Verilog, then I discovered Ruby, then Perl/R/Python, to full-time Python now. It's true that unsupervised, weak coders using perl turn out worse code than in other languages. But it really doesn't take much to pro…

Check out Nim.

    import rdstdin, strutils
    
    let
      time24 = readLineFromStdin("Enter a 24-hour time: ").split(':').map(parseInt)
      hours24 = time24[0]
      minutes24 = time24[1]
      flights: array[8, tuple[since: int,
                              depart: string,
                              arrive: string]] = [(480, "8:00 a.m.", "10:16 a.m."),
                                                  (583, "9:43 a.m.", "11:52 a.m."),
                                                  (679, "11:19 a.m.", "1:31 p.m."),
                                                  (767, "12:47 p.m.", "3:00 p.m."),
                                                  (840, "2:00 p.m.", "4:08 p.m."),
                                                  (945, "3:45 p.m.", "5:55 p.m."),
                                                  (1140, "7:00 p.m.", "9:20 p.m."),
                                                  (1305, "9:45 p.m.", "11:58 p.m.")]
    
    proc minutesSinceMidnight(hours: int = hours24, minutes: int = minutes24): int =
      hours * 60 + minutes
    
    proc cmpFlights(m = minutesSinceMidnight()): seq[int] =
      result = newSeq[int](flights.len)
      for i in 0 .. 
https://github.com/Araq/Nimrod/wiki/Nimrod-for-C-programmers

Re: Why Python Is Slow: Looking Under the Hood

#103
post #81
post #56

Earlier quoted context omitted.

Because beautiful control structures and a REPL ?

There are many languages that offer that alongside an AOT compiler to native code.

Well sure -- but I guess considering Python at all suggests you want or need to. Python control and the REPL are _fine_ reasons to use it. To be clear though, I'm a Tcl-er, but I use it in exactly this context. I meant my comment to stand along the other sibling comments.

Re: Why Python Is Slow: Looking Under the Hood

#104
post #95

Unfortunately this blog post is misleading and uninformed. Python and the related other slow dynamic languages perl, php, and ruby are slow not because the dynamic type checks, branches and indirections are costly, but because they are badly written. They are just inefficient and poor interpreters, with bad ops and data structures. In comparison efficient dynamic languages like lisp, scheme, lua or javascript and man…

Exist a sample I can follow that is made in a imperative language? I'm fluent in python, pascal, C#, obj-c but can survive C if the code is clear...

Re: Why Python Is Slow: Looking Under the Hood

#105

Earlier quoted context omitted.

> The compiler should be able sift through the code and see that "x=1" is an integer, or converts to a float when "x=x+1.0". The short version is "no, it shouldn't". The longer version is that, viewing Python's runtime class memberships as "types", the type system it would represent is not one for which inference without explicit declarations is generally decidable, so an AOT compiler for unmodified, unrestricted Pyt…

Its hard to be worse than an interpreter. 100X slower than compiled code is typical. Whatever 'metadata and indirection' your C target has, it seems likely to be 10X better than interpreted, easily.

> Its hard to be worse than an interpreter.

Its actually quite easy to be worse than an interpreter that is actually a combination of a bytecode compiler and a VM built around the execution model of the language it is for, which is what the main Python interpreter is.

Re: Why Python Is Slow: Looking Under the Hood

#106
post #80

Please forgive my ignorance, but why can't Python just be compiled into assembly like C? The compiler should be able sift through the code and see that "x=1" is an integer, or converts to a float when "x=x+1.0". If I'm doing something like counting from 1 to 10 and summing the count, why can't this be compiled to run the same speed as C? Obviously since it's interpreted, but when I'm done with development, why can't…

Why can't a C++ compiler optimize away virtual function tables? If a C++ compiler could deduce the exact derived type of all objects at compile time, it could call the correct virtual function statically instead of going through the extra indirection at runtime.

If you declare a C++ virtual function final the compiler can avoid the virtual call table for any invocations of that function in the future on that object or any of its derivatives. Of course, you then cannot override it in child classes anymore.

If you declare a virtual function inline, if it is invoked from a well defined object (ie, not through a reference or pointer) you also get a similar effect while still being able to override, except the code is not a static call but is just inlined with that one implementation of the function.

Re: Why Python Is Slow: Looking Under the Hood

#107
post #77

Earlier quoted context omitted.

For a general-purpose problem, Perl won't be particularly fast. For a Perl-type problem (scanning and parsing big files), Perl is very fast. Doing a Perl-type problem in a general-purpose language would be considerably slower. However, Python or others will perform much better in the "can I read my own code six months later" benchmark.

> For a Perl-type problem (scanning and parsing big files), Perl is very fast. I think it's a matter of what you're comparing it to. Compared to using Perl for a general-purpose problem, Perl for scanning/parsing is fast. Compared to scanning/parsing with C, Perl is not fast. $ ruby -e '1.upto(1000000) { |n| puts "This is line number #{n}" }' > file $ time perl -ne 'print if /number 12345/' I gave Perl every possible…

I think the takeaway is that perl is "fast enough" for perl type problems. Writing a complicated file parser in C would be a nightmare.

Re: Why Python Is Slow: Looking Under the Hood

#108
post #91
post #77

Earlier quoted context omitted.

For a general-purpose problem, Perl won't be particularly fast. For a Perl-type problem (scanning and parsing big files), Perl is very fast. Doing a Perl-type problem in a general-purpose language would be considerably slower. However, Python or others will perform much better in the "can I read my own code six months later" benchmark.

I don't know why I always have to chip in on "perl is unreadable lol" comments, but over the last 8 years apart from a steady trickle of C coding here and there the bulk of my dayjob has moved around from C/Verilog, then I discovered Ruby, then Perl/R/Python, to full-time Python now. It's true that unsupervised, weak coders using perl turn out worse code than in other languages. But it really doesn't take much to pro…

Last time I used Perl it went badly, but it was a typical hacked up mess.

For someone mainly in the Python/Java/C++ world is Moose something worth looking at as a mind expansion exercise? Your description makes it sound appealing.

And, dare I ask, what about Perl 6?

Re: Why Python Is Slow: Looking Under the Hood

#109

A related question is: why is Perl so fast? Quite a few years ago I wrote a little Runge-Kutta solver in Perl for some simulation work. It seemed like a good idea at the time. The equations of motion had to be integrated over a very long time, and it could take hours for a single run (still much faster than the Monte Carlo it was being used to do a sanity-check on). I re-wrote everything in C++, and picked up less th…

It's sad that this question hasn't actually even been answered yet.

Some things perl does under the hood to be fast are that integers are (mostly) actually integers under the hood. Arrays are actually arrays under the hood (a fact that makes perl's DBI very fast). And more importantly it has had decades of people trying to make it faster without changing the (sometimes crazy) semantics of the language.

But of course languages with JITs a have massively overtaken it in the performance stakes (PyPy, LuaJIT, and JavaScript). Mostly I think this is a result of lack of funding and huge company spending. Just look at what Facebook have managed to do with PHP.

Re: Why Python Is Slow: Looking Under the Hood

#110
post #85
post #80

Please forgive my ignorance, but why can't Python just be compiled into assembly like C? The compiler should be able sift through the code and see that "x=1" is an integer, or converts to a float when "x=x+1.0". If I'm doing something like counting from 1 to 10 and summing the count, why can't this be compiled to run the same speed as C? Obviously since it's interpreted, but when I'm done with development, why can't…

Runtime metaprogramming. In Python you can create new classes and add methods to existing classes at runtime. If you AOT-compile everything to assembly, you have to take away those runtime features, and then it's not Python anymore.

No, you don't. You "just" need slower fallbacks that does metadata lookup for things that are not statically decidable.

Consider how many people end up implement their own introspectable C object models.

Post reply on HN