Earlier quoted context omitted.
Go has nothing on Python in this regard. I write Go every day, and come from a Python background. I often describe Go as the strongly-typed, more performant version of Python. I say this mostly because my Go code isn't too dissimilar from my Python code (structure, naming, packages). But I still drop into Python if I want to do something quickly. I don't really know why. Maybe it's the Go tooling, e.g. unused variabl…
> I often describe Go as the strongly-typed, more performant version of Python. I've heard Go described this way several times, but I've found it to be a significantly lower-level language than Python. For example, it's much more verbose. In this recent blog post [0], the author converts some C++ code to Go - and it gets longer . 57 lines of C++ become 65 lines of Go. The same code in Python is about 20 lines. In par…
Pyston v2: Faster Python
181–190 of 211 posts
Re: Pyston v2: Faster Python
#182Re: Pyston v2: Faster Python
#183Earlier quoted context omitted.
“the argument is about the ongoing value delivered” Which is an admirable sentiment… but the title of this thread is not “Python: still doing useful work” but “Python: now 20% faster”, and being ridiculously self-congratulatory about this when the correct response is to laugh at the silly pointless frivolity of it. Trying to make Python fast is a fool’s errand, because Python is slow by design . A useful argument wou…
> Which is an admirable sentiment… but the title of this thread is not “Python: still doing useful work” but “Python: now 20% faster” No, it's actually, “Pyston v2: 20% faster Python". But...so what? > and being ridiculously self-congratulatory about this when the correct response is to laugh at the silly pointless frivolity of it For the same reasons Python is often a valuable choice, a faster Python is a valuable o…
Everything in Python is late-bound, untyped, and mutable by default. Straight off the bat that’s three key design decisions that make for a slow interpreter. Defining numbers as heap objects, implementing structures as hash tables, and poor parallelization (GIL, dumb POSIX threading) are three more.
Sure, you can throw huge amounts of brains and resources at code analysis, opportunistic JIT, and the rest a-la V8, but at some point you have to say “Is this an effective use of those valuable resources?” Especially when every such optimization could potentially break existing, stable user code running in production.
..
But, let’s get back to larger perspective:
Faster overall takes into account not just the time it takes to run a user program, but also the time it takes to learn, implement, debug, and deploy. Only one of these is machine time, which these days is cheap as chips and nearly inexhaustable; all the rest are human labor, which is both expensive and limited.
Which is not to say that Python is faster at all those human tasks than other languages. To determine that would require real-world practical testing, plus a willingness to accept what those tests tell you (which might not be what you wanted to hear). But I’m willing to bet that the time spent on all those manual tasks vastly outweighs the time saved by 20% faster runtime for the vast majority of use cases.
So at some point you have to stop and ask: Are these fundamental changes adding genuine, measurable value for real-world users solving real-world problems? Or is it just code masturbation basement nerds whose idea of productivity is playing with internal guts in pursuit of some trivial abstract benchmarks?
Because, honestly, “20% faster” is an absolute joke. If I can’t make my program 200% faster just by adding a second hardware box, then I’ll want to know why. And if the answer is no more complex than “because the language isn’t very good at parallelism”, then all other arguments are completely moot.
..
Look, I’ve written slow interpreters. Implemented in Python, no less. A not-very-complex program might take 2 minutes to run. But then, 80% of that painfully long run-time is actually IO-bound operations, and even that is totally irrelevant when those 2 minutes of machine time have replaced 20 minutes of manual work.
That’s 20 minutes of paid human labor, eliminated by a really-slow custom interpreter written in pretty-slow CPython. You can easily put a dollar cost on that human time (salary, etc) and multiply it by the number of work units in a year, and you’ve calculated its real-world benefit.
Let us know when you can calculate the real-world benefit of a 20% quicker proprietary Python-like interpreter that may or may not execute user programs exactly the same as CPython. Otherwise, as I say, anything less than a magnitude’s improvement isn’t even worth getting out of bed for.
Re: Pyston v2: Faster Python
#184Earlier quoted context omitted.
Julia covers your use cases and overwhelmingly fast.
My gripe with Julia etc. as replacements, is that Python is duct tape. I don't need fast duct tape, I need duct tape that is understood and used by essentially everyone I work with, and that has native, fast handling of large amounts of data (NumPy, Pandas). Good user experience as duct tape. From my perspective Julia is sacrifising some amount of "duct tape UX" to gain speed, and that's the wrong direction. Whenever…
This only works sometimes--for problems that allow you to do a relatively large amount of computation in the compiled language to justify the cost of marshalling Python data structures into native data structures. For matrices of scalar values, this works well. For many other problems (consider large graphs of arbitrarily-typed Python objects, or even a dataframe on which you need to invoke a Python callback on each element). If you rewrite a big enough piece of your Python codebase in the compiled language, then it will work, but now you're maintaining a significant C/C++/etc code base and the bindings and the build/packaging system that knows how to integrate the two on all of your target platforms. Python really doesn't have a good answer for these kinds of problems, and these are by far the more common case (though perhaps not more common in data science specifically).
Re: Pyston v2: Faster Python
#185Earlier quoted context omitted.
I understand; lots of people have this sentiment. It’s an inconvenience in many cases, but we’re comparing it against Python, which has no type safety at all much less generics (apart from Mypy, which has many, many other issues).
All dynamic languages are generic by default.
Re: Pyston v2: Faster Python
#186Earlier quoted context omitted.
> I often describe Go as the strongly-typed, more performant version of Python. I've heard Go described this way several times, but I've found it to be a significantly lower-level language than Python. For example, it's much more verbose. In this recent blog post [0], the author converts some C++ code to Go - and it gets longer . 57 lines of C++ become 65 lines of Go. The same code in Python is about 20 lines. In par…
Exactly my opinion! I write python for a living since more than 13 years, and I find Go awfully verbose. It makes simple things feel like a chore. A three-to-one ratio of lines of code sounds about right. That's not a trade-off I can make, no matter how big the performance improvements.
This is crazy. Many of those lines are closing brackets or whitespace. But moreover, optimizing for characters or LOC is absurd. Optimize for maintainability or readability, at which point Go is at least as good as Python (I would argue better). Optimize for tooling, especially package management and build tooling--Go is many times better than Python here. Optimize for performance--Go is literally hundreds or thousands of times better here. Optimize for breadth and quality of ecosystem. Optimize for deployment story (single small artifact vs hundreds of megabytes of dependencies). These are the things that matter, not lines of code.
Re: Pyston v2: Faster Python
#187Earlier quoted context omitted.
Julia covers your use cases and overwhelmingly fast.
My gripe with Julia etc. as replacements, is that Python is duct tape. I don't need fast duct tape, I need duct tape that is understood and used by essentially everyone I work with, and that has native, fast handling of large amounts of data (NumPy, Pandas). Good user experience as duct tape. From my perspective Julia is sacrifising some amount of "duct tape UX" to gain speed, and that's the wrong direction. Whenever…
Interestingly, R is probably a better UX for statisticians/data scientists than Python is (almost all the good parts of Numpy/Pandas were in R first), but it really suffers from not being well known by developers.
To be fair to R though, it's much, much easier to deploy than Python, which is a shocking indictment of the current Python packaging ecosystem.
Re: Pyston v2: Faster Python
#188Earlier quoted context omitted.
I think Julia is strictly better than Python, both for data oriented applications and for web development.
Julia might be. It's certainly higher performance from what I understand, and it offers some syntactic flexibility that I miss from R when using Python (Python could never have a fully complete dplyr). It seems from afar to be rather more complex than those languages, though.
It's still a little too wild-west for my tastes right now, but I want it to improve as I think it's got real potential.
Re: Pyston v2: Faster Python
#189Earlier quoted context omitted.
WRT performance ceiling, I'm mostly talking about things like Pandas which eagerly evaluate and which aren't amenable to a parallel execution model (multiple threads operating on the same data frame with minimal contention). WRT poor APIs, I'm talking about things like matplotlib or pandas or etc that take a whole slew of arguments and try to guess the caller's intent by inspecting the types of the arguments. The ref…
You're getting some pushback, but I tend to agree with you on matplotlib and pandas. Great libraries are designed so that you can get a feel for them and -- with practice -- use them intuitively. Even after years of (admittedly light) use I still find pandas' multi-indexes confusing, and I always have to look up the best of myriad ways to do something in matplotlib. In comparison, R's dplyr and ggplot have stuck with…
Re: Pyston v2: Faster Python
#190Earlier quoted context omitted.
matplotlib and pandas were designed with the idea of mimicking interfaces more popular than the project (when they were first conceived). The "easy" interface is a large part of why those projects are now more popular then their inspirations.
very true; I found matplotlib very appealing because I didn't have to relearn anything coming from matlab