Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You Should Compile Your Python and Here’s Why

glyph.twistedmatrix.com

Post reply on HN
Print view View original

11–20 of 109 posts

Re: You Should Compile Your Python and Here’s Why

#11

Post by sgtnoodle »

I've been writing some python the last few days. Today I did something terrible. I like to believe it was out of wisdom rather than ignorance. I needed to do some nuanced bit manipulations, and so I built up a string of ascii 0's and 1's, sliced the string, and then converted back to integers.

I worked on a codebase many years ago when I was an intern. That code would read frames off a CAN bus in a car. For some reason, it could only handle about 100 frames per second before running out of CPU. It turned out someone had done the same; they turned every 8 byte CAN payload into a string of 64 characters, sliced the relevant fields out, and then converted back to integers. There were about 2000 frames per second coming in, and it was just way too inefficient.

In my usage, it will maybe never happen, outside of a unit test I wrote for it. I am really tempted to rewrite it tomorrow to use but shifts...

Re: You Should Compile Your Python and Here’s Why

#14

Post by 1vuio0pswjnm7 »

"Python is Slow, And That's Fine, Because It's Fast Enough"

At least this is honest.

No matter what the script does, no matter how fast the libraries, the Python intepreter has a slow startup time.

On multiple occasions I have seen people commenting on HN argue that Python is not slow.

I think for these commenters Python is "fast enough".

For others, like me, it may not be "fast enough".

IOW, the question is not whether Python is "fast" or "slow", but whether it is "fast enough" or "too slow" for a given user.

For some users, it might not be fast enough. And that's fine.

Re: You Should Compile Your Python and Here’s Why

#15

Post by e12e »

jwilk wrote: post #4
tails4e wrote: post #2

Some interesting tidbits on python code and compilation performance 1) During AoC I mostly used cpython and numpy. One day my code was running too slow so I fired up pypy. It ran even slower. Turns out numpy and pypy do not play well together in terms of performance. 2) I use matlab in work, and had some code that was slower than I liked, despite efforts to profile and optimise. I used the coder tool to compile it to…

What's AoC?

See eg: https://news.ycombinator.com/item?id=29403522

Re: You Should Compile Your Python and Here’s Why

#16

Post by tempnow987 »

fgeiger wrote: post #7
dikei wrote: post #6

This `mypyc` sounds too good to be true, I wonder what the catch's.

The obvious catch is that the compiled binary is not portable. You need to compile and deploy for each platform separately. You don't need to worry about platforms at all when deploying/sharing pure Python modules.

But you do have to worry about dependency management etc etc.

Re: You Should Compile Your Python and Here’s Why

#17

Post by fabian2k »

> Unlike Cython, mypyc doesn’t directly support interfacing with C libraries or speeding up numeric code.

I'm not sure I understand that part of the mypyc documentation (https://mypyc.readthedocs.io/en/latest/introduction.html#why...). Does that mean that you can't use something like numpy at all?

Re: You Should Compile Your Python and Here’s Why

#19

Post by shankr »

fgeiger wrote: post #7
dikei wrote: post #6

This `mypyc` sounds too good to be true, I wonder what the catch's.

The obvious catch is that the compiled binary is not portable. You need to compile and deploy for each platform separately. You don't need to worry about platforms at all when deploying/sharing pure Python modules.

you do have to worry about the platform specific dependencies.
Post reply on HN