Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster Python with Guido van Rossum

softwareatscale.dev

Post reply on HN
Print view View original

201–210 of 251 posts

Re: Faster Python with Guido van Rossum

#201

Post by KronisLV »

igouy wrote: post #87
pyentropy wrote: post #55

Earlier quoted context omitted.

> They seem to invest a lot in JavaScript which is not what I’d call a performant language JavaScript (V8 at least) is extremely performant, near native code performance. Considering how dynamic it is, it's not an easy feat but Google, Apple and Mozilla work a lot on it. Here's a benchmark where JS is 50x faster than Python: https://github.com/kostya/benchmarks Note that PyPy does much better.

And some where JS is not 50x faster than Python: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Those are some pretty nice synthetic benchmarks! However, if you'd like a look at a more real world scenario or two, have a look at the TechEmpower benchmarks as well.

For example, here are filters for both JS and Python: https://www.techempower.com/benchmarks/#section=data-r20&hw=...

Do note that for most of the "realistic" stacks out there, you'd probably want to filter out all of the micro or no framework approaches, to compare something like Express and Koa against Django and Flask, like in the following link: https://www.techempower.com/benchmarks/#section=data-r20&hw=...

Here's the summary page with composite scores across all tests (though you might also want to filter by data storage solution, for example, MySQL): https://www.techempower.com/benchmarks/#section=data-r20&hw=...

To summarize, the performance from the best to worst currently is:

  Composite scores: Koa (JS) > Express (JS) > Flask (Python) > Django (Python)

Re: Faster Python with Guido van Rossum

#202

Post by takeda »

whimsicalism wrote: post #193
dragonwriter wrote: post #149

Earlier quoted context omitted.

TypeScript:JS is a lot like mypy:Python; the only real difference is that Python adapted so that mypy (originally envisioned as its own language) code is also valid Python as well as the reverse, obviating the need for a compilation step to run mypy in the Python runtime (whereas TS has to compile to JS to run); also, mypy has mypyc to compile (currently, only a subset of) mypy code to something more efficient than n…

Having worked extensively with both, I don't think this is an apt comparison. Type inference & hinting at the IDE level is just not as developed either

Which language did you mean? Also in any case wouldn't that be IDE issue.

I used Python and I noticed over the years that things like type interference drastically improved.

Similarly type checker in Rust extension for IntelliJ was lacking even though Rust has superior type system.

Re: Faster Python with Guido van Rossum

#203

Post by ChrisRackauckas »

prionassembly wrote: post #190
wenc wrote: post #169

Earlier quoted context omitted.

Thanks for providing code. I’ve written code like that before and the first thing that jumps out at me is that you have for loops in Python code which isn’t slow but will never be as fast as loops in a static compiled language. Sometimes for loops are the most readable way of expressing a calculation but as a numerical computation person my instinct is to look for opportunities to vectorize by rewriting loops into ma…

Some of that Python is strange for numerical Python practices, yes, but I was aiming for close floating-point output equivalence. Rust ndarray provides the standard set of matrix ops, but I keep getting different results (\propto 1e-5 on a stable basis, which isn't quite a proof of incorrectness, but....) when I use too much numpy stuff like linalg.norm, etc. (ndarray slices in Rust are annoying to use because of lif…

>I wonder if open source ODE solvers are getting good high-order symplectic methods by now...

Up to 10th order https://diffeq.sciml.ai/stable/solvers/dynamical_solve/#Symp... . Also Magnus methods https://diffeq.sciml.ai/stable/solvers/nonautonomous_linear_... and exponential integrators https://diffeq.sciml.ai/stable/solvers/split_ode_solve/#Ordi... . And the expmv implementations are highly optimized as well: https://github.com/SciML/ExponentialUtilities.jl . A lot of this specialized matrix exponential stuff is rather fun, for example see https://github.com/SciML/ExponentialUtilities.jl/pull/64

Re: Faster Python with Guido van Rossum

#204

Post by ChrisRackauckas »

prionassembly wrote: post #192
fasterbynight wrote: post #188

Earlier quoted context omitted.

I don't know about other languages, but in Julia I've heard people often say that loops end up faster than the equivalent vectorized code. So while this is true for Python/Matlab I don't think it is good universal advice. That said, matrix notation can sometimes be the more readable way of expressing a calculation.

I might have used an early beta of Julia circa 2018 or something, but the chorus that it performs like $static_fastlang doesn't match the experience I had.

Do you have code to share and look at? All we can do is point to real-world code and benchmarks. For this specific case, see LoopVectorization.jl results (https://juliasimd.github.io/LoopVectorization.jl/latest/exam...), and the corresponding effects on stiff ODE solver benchmarks against C and Fortran packages (https://benchmarks.sciml.ai/html/StiffODE/Hires.html).

Re: Faster Python with Guido van Rossum

#205

Post by [deleted] »

fasterbynight wrote: post #188
wenc wrote: post #169

Earlier quoted context omitted.

Thanks for providing code. I’ve written code like that before and the first thing that jumps out at me is that you have for loops in Python code which isn’t slow but will never be as fast as loops in a static compiled language. Sometimes for loops are the most readable way of expressing a calculation but as a numerical computation person my instinct is to look for opportunities to vectorize by rewriting loops into ma…

I don't know about other languages, but in Julia I've heard people often say that loops end up faster than the equivalent vectorized code. So while this is true for Python/Matlab I don't think it is good universal advice. That said, matrix notation can sometimes be the more readable way of expressing a calculation.

[deleted]

Re: Faster Python with Guido van Rossum

#207

Post by rich_sasha »

Qem wrote: post #6
amelius wrote: post #3

Python needs to go multicore, like OCaml. And add a modern concurrent garbage collector. One of the difficulties is of course to not break existing C extensions.

Given Python has the multiprocessing module, I always get confused when people talk about Python lack of support for multicore. What are the shortcomings of the multiprocessing module, that cause people to disregard it?

On Windows multiprocessing is very poor due to lack of forking. This means for example no memory at all is shared; modules, data, external shared libraries etc are all individually loaded by each subprocess. There are also some Windows-specific oddities that make using it a further pain (can’t remember the details).

There is a project to effectively internalise multiprocessing by running separate interpreters inside one process. It’s meant to be cheaper than having separate modules too.

Re: Faster Python with Guido van Rossum

#208

Post by rich_sasha »

fiedzia wrote: post #34
Qem wrote: post #6

Earlier quoted context omitted.

Given Python has the multiprocessing module, I always get confused when people talk about Python lack of support for multicore. What are the shortcomings of the multiprocessing module, that cause people to disregard it?

Sharing data between processes is _very_ expensive, so whole classes of programs are not suited for this approach. For example I had a program that preloads large immutable dataset. and a bunch of threads use it. I couldn't do it efficiently in Python (at least not CPython).

Outside of Windows, I believe, you can do it so that the dataset is read-only shared (or copy-on-write) via fork, and then it’s as quick as local access.

Not on Windows though, as it lacks fork.

Re: Faster Python with Guido van Rossum

#209

Post by igouy »

makapuf wrote: post #197
igouy wrote: post #156

Earlier quoted context omitted.

> Each example just calls out to a very fast c library… No. https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

from re import sub, findall That's where pcre2 is called (or a finely tuned regex c lib)

re is an ordinary standard Python library —

https://docs.python.org/3/library/re.html

pcre2 is not —

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Re: Faster Python with Guido van Rossum

#210

Post by igouy »

Mikeb85 wrote: post #191
igouy wrote: post #186

Earlier quoted context omitted.

Given that you say "basically glue" why the previous surprise about "basically using C types and GMP" ?

Because it's obviously not the idiomatic way to use Python. No one is going to bother writing a program that way in Python, they'd just write it in C or C++ then link it.

Sorry, I gave up thinking I knew what people would or wouldn't bother doing a long time ago.
Post reply on HN