Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You Should Compile Your Python and Here’s Why

glyph.twistedmatrix.com

Post reply on HN
Print view View original

81–90 of 109 posts

Re: You Should Compile Your Python and Here’s Why

#81

Post by teaearlgrey-hot »

teaearlgraycold wrote: post #74

Honestly for a language that’s supposed to be simple and easy to use Python seems really complicated and hard to understand. Anything big or performance critical should probably not be written in Python - and yet people are doing it. And then we end up with a house of cards with this project built on top of the very shaky foundation of mypy.

I think Python is simple to use but difficult to scale. It can be useful to think of programming languages as tools. Assembly is the workbench where tools are made and raw materials are handled. C++ is like your standard set of machining tools. Java is like a bag of tools, hammers, handsaws. Python is like an electric nail gun, or perhaps a gas chainsaw. It does many of the things the other languages say that you can do directly out of the box and often in a manner that’s more sensible to people initially using a tool. But when it comes to building a house and all you have is a chainsaw? You’re gonna have some very cool log cabins, and if we go outside the analogy for a moment: code that uses tricks to accomplish what others would do with better building, architecture, or execution. Now that said, how many people do you know who might go off and build a log cabin vs. a modern home on their own?

Re: You Should Compile Your Python and Here’s Why

#82

Post by cirgue »

pharmakom wrote: post #37
1vuio0pswjnm7 wrote: post #14

"Python is Slow, And That's Fine, Because It's Fast Enough" At least this is honest. No matter what the script does, no matter how fast the libraries, the Python intepreter has a slow startup time. On multiple occasions I have seen people commenting on HN argue that Python is not slow. I think for these commenters Python is "fast enough". For others, like me, it may not be "fast enough". IOW, the question is not whet…

I don’t always know how fast will be fast enough when I start a project and the risk of needing a rewrite really puts me off Python.

I work in python all the time, and I’ve still not found a situation where this happened. Sometimes you might have specific components that you might want to write in other languages, but this is really rare unless you’re manually writing really computationally intensive or time critical procedures. Which hopefully you should know going into the project. Like if you’re rolling your own in-line audio processing you know up front that python is a bad choice. But any CRUD-like app? Python is plenty fast, especially if you’re relying on c implementations of heavy-lifting stuff.

Re: You Should Compile Your Python and Here’s Why

#83

Post by bhandziuk »

IshKebab wrote: post #80
joelthelion wrote: post #48

Earlier quoted context omitted.

> the Python intepreter has a slow startup time. This really depends on your perspective. Sure, it's a lot slower than running a native binary. But it's still fast enough for interactive tools that you run often. If you compare that to java tools, for example, which take seconds if not dozens of seconds to start (gradle, I'm looking at you!), python is far better.

> it's still fast enough for Interactive tools that you run often. Oh really? https://www.mercurial-scm.org/wiki/OxidationPlan > chg's very existence is because we need hg to be a native binary in order to avoid Python startup overhead. If hg weren't a Python script, we wouldn't need chg to be a separate program.

god I love Hg over git. I don't understand why git is more popular.

Re: You Should Compile Your Python and Here’s Why

#84

Post by [deleted] »

1vuio0pswjnm7 wrote: post #14

"Python is Slow, And That's Fine, Because It's Fast Enough" At least this is honest. No matter what the script does, no matter how fast the libraries, the Python intepreter has a slow startup time. On multiple occasions I have seen people commenting on HN argue that Python is not slow. I think for these commenters Python is "fast enough". For others, like me, it may not be "fast enough". IOW, the question is not whet…

[deleted]

Re: You Should Compile Your Python and Here’s Why

#85

Post by [deleted] »

bitwize wrote: post #30
taneq wrote: post #25

Earlier quoted context omitted.

Reminds me of that Tao of Programming koan (if I remember it rightly) that went something like: The novice, in his frustration, struck the side of his computer. The master walked over and asked what he was doing. The novice exclaimed, "my computer is not working and I do not know why!" The master admonished him, saying "you cannot solve the problem by striking the computer without knowing what is wrong." Then the mas…

It was one of the AI koans in Jargon: > A novice was trying to fix a broken Lisp machine by turning the power off and on. > Knight, seeing what the student was doing, spoke sternly: “You cannot fix a machine by just power-cycling it with no understanding of what is going wrong.” > Knight turned the machine off and on. > The machine worked.

[deleted]

Re: You Should Compile Your Python and Here’s Why

#86

Post by yifanl »

taneq wrote: post #25
sgtnoodle wrote: post #11

I've been writing some python the last few days. Today I did something terrible. I like to believe it was out of wisdom rather than ignorance. I needed to do some nuanced bit manipulations, and so I built up a string of ascii 0's and 1's, sliced the string, and then converted back to integers. I worked on a codebase many years ago when I was an intern. That code would read frames off a CAN bus in a car. For some reas…

Reminds me of that Tao of Programming koan (if I remember it rightly) that went something like: The novice, in his frustration, struck the side of his computer. The master walked over and asked what he was doing. The novice exclaimed, "my computer is not working and I do not know why!" The master admonished him, saying "you cannot solve the problem by striking the computer without knowing what is wrong." Then the mas…

A variation of the "knowing where to hit the hammer" story.

Re: You Should Compile Your Python and Here’s Why

#87

Post by pharmakom »

cirgue wrote: post #82
pharmakom wrote: post #37

Earlier quoted context omitted.

I don’t always know how fast will be fast enough when I start a project and the risk of needing a rewrite really puts me off Python.

I work in python all the time, and I’ve still not found a situation where this happened. Sometimes you might have specific components that you might want to write in other languages, but this is really rare unless you’re manually writing really computationally intensive or time critical procedures. Which hopefully you should know going into the project. Like if you’re rolling your own in-line audio processing you kno…

Any CPU heavy algorithm that does not have a clear boundary to start the C/C++ part is a poor match for Python.

Re: You Should Compile Your Python and Here’s Why

#88

Post by dralley »

bhandziuk wrote: post #83
IshKebab wrote: post #80

Earlier quoted context omitted.

> it's still fast enough for Interactive tools that you run often. Oh really? https://www.mercurial-scm.org/wiki/OxidationPlan > chg's very existence is because we need hg to be a native binary in order to avoid Python startup overhead. If hg weren't a Python script, we wouldn't need chg to be a separate program.

god I love Hg over git. I don't understand why git is more popular.

It's pretty easy to understand. Github.

Re: You Should Compile Your Python and Here’s Why

#89

Post by dralley »

BiteCode_dev wrote: post #67
FrenchAmerican wrote: post #29

I'm learning Python. Do someone knows a resource to learn the well-known tricks for "fast" Python? A book, a website, a cheat sheet or even a MOOC or part of a MOOC?

There are plenty of things for which python is not slow, but people using python don't know a lot about programming, and end up with a slow result. A couple of advises: - the right algo will go a long, long way. - know your data structures. E.G: assigning to a slice is ridiculously fast (even while unpacking), memory views may save a lot on byte heavy workloads, heapq and deque are underrated, etc. Also check out htt…

>pathlib and dataclasses. In a regular code they are great. On a bottleneck however, they are very slow.

Do you have any numbers to back this up? Why would Pathlib be slow?

Re: You Should Compile Your Python and Here’s Why

#90

Post by slkdk32 »

adgjlsfhk1 wrote: post #54
VWWHFSfQ wrote: post #44

Earlier quoted context omitted.

This is premature optimization right

Yes and no. One of the biggest disadvantages of python is that the way people typically optimize python code is to 1. rewrite in numpy 2. rewrite in Cython (or other python compiler) 3. rewrite in C++ The problem with this workflow is that it means you end up rewriting large chunks of your code over several years in systems that have fairly different idioms. If you suspect you might in the future be performance bottl…

No post body was provided.
Post reply on HN