Live data from Hacker News

Python’s Weak Performance Matters

metarabbit.wordpress.com

331–336 of 336 posts

Re: Python’s Weak Performance Matters

#331

Earlier quoted context omitted.

The context is that we're talking about people picking up the language for the first time, coming from other languages. What matters is whether the scoping rules make sense to such people. For that, it matters whether they're coming from another function-scoped language (or are familiar with one), and whether they're used to having their variable declarations being obvious or not. For me, when I was first learning Py…

Python is and always has been designed to be an introductory programming language, so your criticism does not apply in the case of this design intent. More people learn Python first than learn another language first. Having seen beginning programmers learn Python, I can say this is an utter nonissue to them. For the case of experienced programmers, the behavior is consistent and simple. In what way does this not "mak…

> the behavior is consistent and simple

Let's look at our example again:

  for foo in bar:
    pass
  # Why is "foo" in scope here???
Well, the answer could be "it's not". It depends on whether bar actually produced anything to assign to foo. Doing a "print(foo)" after that loop might print something, or it might throw a NameError.

Now I understand why that happens (in terms of what the loop actually desugars into), and you understand why it happens. But it's not as consistent and simple as you're trying to make it out to be. You have to really understand what a for-in loop is doing under the hood to explain the behavior.

You seem to feel that I'm attacking Python or something. I'm not. It's a nice language to work in, with a lot to recommend it. But it's not a language I'd choose as a poster child for scope and namespace making sense all the time unless you really dig into what's going on "under the hood".

> Does Haskell's normal order evaluation not "make sense?"

No opinion, really; not enough intimate familiarity with the problem space to have one.

> Does Ruby's optional parentheses for method calls not "make sense?"

Again, no opinion.

> Does a regular expression literal syntax not "make sense?"

It depends on the regexp. If your regexp is simple enough, it's fine. In far too many cases you end up with a write-only monstrosity. Also, you say "syntax" as if there were only one; there are multiple and some make more sense than others.

> Does JSX not "make sense?"

Again, no opinion.

> Do public/protected/private modifiers in Java not "make sense?"

It depends on how they're used.

> Stop confusing a match to your personal comfort zone for actual quality or fitness-for-purpose.

Stop confusing "doesn't always make sense" for "doesn't make sense" (totally different statements, there), and the former statement for a statement about quality of fitness-for-purpose. Lots of things are considered high-quality and fit for purpose while still having flaws. Possibly flaws that could not have been avoided without sacrificing other goals. But determining that requires first admitting that the flaws exist and then evaluating them. If we either pretend that the flaws don't exist, or that flaws existing is somehow an indicator that the entire system is unfit for purpose, it's hard to think productively about the design of the next system.

Re: Python’s Weak Performance Matters

#332
post #24

Where are the main blowouts in python performance? For example, is it compilation, evaluation overhead, or memory management?

> Where are the main blowouts in python performance? I did some research a few years ago that tried to quantify some of this. If you trust my methodology, the biggest problems (depending on application, of course) are: boxing of numbers; list/array indexing with boxed numbers and bounds checking; and late binding of method calls. Basically, doing arithmetic on lists of numbers in pure Python is about the worst thing…

I have not yet done detailed study, but your paper appears to be a fabulous resource. The context from your post is high-value also. Thanks.

Re: Python’s Weak Performance Matters

#333

Earlier quoted context omitted.

Thanks for mentioning Julia, a good solution to slow Python code. I used Julia for 2 weeks last year on a consulting gig, and despite rough spots and given more development time, Julia might become fairly popular. I have never been much of a fan of Python. 15 years ago at lunch Peter Norvig was talking about the advantages of Python (he and I wrote Common Lisp books at the same time). I then tried Python for a few mo…

This. I can't emphasize enough how type hints + decent IDE is useful to productivity. Cython is also pretty good. My tip to write fast Cython is to code as if you were writing pure C and forget higher level constructs. This way it gets translates to C almost 1:1 with all performance benefits. Alternatively, if you do need more abstraction, you can write nice C++14 and use Cython as glue code.

This is the productivity problem that julia aims to solve. The idea is to stay in one language that everyone on the team can work productively in, while still producing hi performance code.

Re: Python’s Weak Performance Matters

#334
post #24

Where are the main blowouts in python performance? For example, is it compilation, evaluation overhead, or memory management?

> Where are the main blowouts in python performance? I did some research a few years ago that tried to quantify some of this. If you trust my methodology, the biggest problems (depending on application, of course) are: boxing of numbers; list/array indexing with boxed numbers and bounds checking; and late binding of method calls. Basically, doing arithmetic on lists of numbers in pure Python is about the worst thing…

Thank you so much for posting the paper!

Re: Python’s Weak Performance Matters

#335
post #211

This is a weird article at this point in time. The question it addresses: "Does Python's performance matter?" Has always had the answer: "Sometimes, and you have options for those cases." The OP found a "sometimes", and he's using one of those options. In this case, he's got Python for prototyping and glue, with Haskell improving performance. This is as it should be. I don't know of any Python advocates who say it's…

> This is a weird article at this point in time. It's a timely article, because a number of things have changed in recent years to make the tradeoffs around using Python quite different from what they once were. Ten years ago, Python was slower than the alternatives by a small constant factor, datasets weren't big enough for python performance to be an issue, Python had a world-class tooling/library ecosystem and hig…

Agree, I really like the why "things are different" part.

Re: Python’s Weak Performance Matters

#336
post #302

Earlier quoted context omitted.

pandas.read_csv is kind of abysmally slow, unfortunately. There are a couple of alternatives, but nothing has really taken hold. Dask exists, but not everyone can run a distributed system to read a multi-gigabyte csv.

Have you tried paratext from wiseio? I have had good experiences with it.

I have. For whatever reason, I had difficulty with it.
Post reply on HN