Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You Should Compile Your Python and Here’s Why

glyph.twistedmatrix.com

Post reply on HN
Print view View original

91–100 of 109 posts

Re: You Should Compile Your Python and Here’s Why

#91

Post by BiteCode_dev »

dralley wrote: post #89
BiteCode_dev wrote: post #67

Earlier quoted context omitted.

There are plenty of things for which python is not slow, but people using python don't know a lot about programming, and end up with a slow result. A couple of advises: - the right algo will go a long, long way. - know your data structures. E.G: assigning to a slice is ridiculously fast (even while unpacking), memory views may save a lot on byte heavy workloads, heapq and deque are underrated, etc. Also check out htt…

>pathlib and dataclasses. In a regular code they are great. On a bottleneck however, they are very slow. Do you have any numbers to back this up? Why would Pathlib be slow?

Anthony Sottile has a good bit on it: https://www.youtube.com/watch?v=tFrh9hKMS6Y

TL;DR: Path() initialization is very heavy compared to simple strings, and subsequent path operations produce new Path() objects every time.

The video then illustrates the point with a patch to the black formatter cache yielding a 40X speedup.

Re: You Should Compile Your Python and Here’s Why

#92

Post by dralley »

BiteCode_dev wrote: post #91
dralley wrote: post #89

Earlier quoted context omitted.

>pathlib and dataclasses. In a regular code they are great. On a bottleneck however, they are very slow. Do you have any numbers to back this up? Why would Pathlib be slow?

Anthony Sottile has a good bit on it: https://www.youtube.com/watch?v=tFrh9hKMS6Y TL;DR: Path() initialization is very heavy compared to simple strings, and subsequent path operations produce new Path() objects every time. The video then illustrates the point with a patch to the black formatter cache yielding a 40X speedup.

Oh, that is unfortunate. Thanks for the tip.

Re: You Should Compile Your Python and Here’s Why

#93

Post by mihaic »

It really is a shame that "it's fast enough" is repeated without saying what for. It really is fast enough for scripting logic, that so many other aspects would benefit imensly if more effort were put into speeding up the standard interpreter.

I find Python to be a great language to describe business logic, and honestly it should end in 10 years to be within at most 2x slower than Javascript.

Re: You Should Compile Your Python and Here’s Why

#94

Post by digisign »

pharmakom wrote: post #37
1vuio0pswjnm7 wrote: post #14

"Python is Slow, And That's Fine, Because It's Fast Enough" At least this is honest. No matter what the script does, no matter how fast the libraries, the Python intepreter has a slow startup time. On multiple occasions I have seen people commenting on HN argue that Python is not slow. I think for these commenters Python is "fast enough". For others, like me, it may not be "fast enough". IOW, the question is not whet…

I don’t always know how fast will be fast enough when I start a project and the risk of needing a rewrite really puts me off Python.

It's a great prototyping language. Sometimes the final product will use a better algorithm because it was found easily in Python then ported to bare metal language.

Re: You Should Compile Your Python and Here’s Why

#95

Post by pella »

other alternative : PYSTON "Pyston is a faster and highly-compatible implementation of the Python programming language."

"Our speedup on ARM (30% on a Graviton EC2 instance) is comparable to our speedup for x86 (34% on an Intel i7-6700)." https://blog.pyston.org/2022/04/01/pyston-v2-3-3-arm-support...

2021.aug.30 : "Pyston Team Joins Anaconda to Expand Open-Source Project Development" https://www.anaconda.com/blog/pyston-team-joins-anaconda

Roadmap: https://blog.pyston.org/2021/10/26/pyston-roadmap/

Github: https://github.com/pyston/pyston

Re: You Should Compile Your Python and Here’s Why

#96

Post by stefanos82 »

FrenchAmerican wrote: post #42
stefanos82 wrote: post #31

Earlier quoted context omitted.

Fluent Python; go for it!

Thanks! I appreciate, really.

I've just noticed it did not include my URL in original reply...weird.

Anyway, here it is, this is the second edition of the book https://www.amazon.com/dp/1492056359

I read the first one at work a few years ago and was fascinated with all the goodies it shared with us.

In my humble opinion, it's a must read by all Python developers, experienced or not; everyone will learn something new just by reading it.

Re: You Should Compile Your Python and Here’s Why

#97

Post by autoexec »

mule1 wrote: post #28

Not to be shit-eating, but I never understood why people use python instead of Go other than for ML teams. I program mostly in Java and Go so would love a perspective.

> Not to be shit-eating, but I never understood why people use python instead of Go

I've got a few sysadmin type scripts I wrote in perl that I'm currently rewriting in python because other people will use and need to update those things and they will be more likely to know python than perl. I can't imagine they'll be more likely to know Go either.

Re: You Should Compile Your Python and Here’s Why

#98

Post by sli »

At this point, I think maybe Python ceases to be the correct tool for the purpose. There comes a point where trying to squeeze every ounce of performance out of something that is, relatively speaking, not focusing on raw performance is just stubbornness. Write the slow part in C/C++/Rust/Zig/Nim or something and be done with it instead of making your Python unreadable. At least your native code can be idiomatic instead of being filled with arcane tricks.

Re: You Should Compile Your Python and Here’s Why

#99

Post by ciupicri »

BiteCode_dev wrote: post #67
FrenchAmerican wrote: post #29

I'm learning Python. Do someone knows a resource to learn the well-known tricks for "fast" Python? A book, a website, a cheat sheet or even a MOOC or part of a MOOC?

There are plenty of things for which python is not slow, but people using python don't know a lot about programming, and end up with a slow result. A couple of advises: - the right algo will go a long, long way. - know your data structures. E.G: assigning to a slice is ridiculously fast (even while unpacking), memory views may save a lot on byte heavy workloads, heapq and deque are underrated, etc. Also check out htt…

How do you pre-allocate lists and dicts?

Re: You Should Compile Your Python and Here’s Why

#100

Post by bsder »

fgeiger wrote: post #8

Frankly, I would find it much easier to write the naive C version than this "moderately" optimized Python version in the article although I am much more familiar with Python than with C.

I really hate these kinds of articles because they don't DO anything useful in the sample programs.

Grab a web page and parse it for some data. Suddenly, C looks like trash. The network latency negates any speed advantage from C and the fact that C has to manually manage memory everywhere makes your C code look like a dumpster fire relative to the Python code. You can always pick a task that favors one language or the other.

What people forget is that fast/slow ALSO includes "time to develop the code". If I can write the Python code in 1 hour and the C code in 10 hours, the code has to be a lot slower or be run a lot more times before the time wasted writing the C code pays off.

Post reply on HN