I'm learning Python. Do someone knows a resource to learn the well-known tricks for "fast" Python? A book, a website, a cheat sheet or even a MOOC or part of a MOOC?
Fluent Python; go for it!
You Should Compile Your Python and Here’s Why
31–40 of 109 posts
Re: You Should Compile Your Python and Here’s Why
#32Not to be shit-eating, but I never understood why people use python instead of Go other than for ML teams. I program mostly in Java and Go so would love a perspective.
Consider that a lot of ML programs sit at 100% CPU use (on a single core), and 50-60% GPU use (if you're lucky).
The common use of Python is a huge issue.
Re: You Should Compile Your Python and Here’s Why
#33I've been typing SQLAlchemy for some months now and there's lots of scenarios where if you end up on mypy's issue tracker (which is very often), you will see "oh use a #type: ignore to deal with this occasional use case" as official advice. And here we see a compiler that considers all "# type: ignore" to be bugs and will eventually refuse to compile them. That's simply not realistic for the way Python typing works right now and I am skeptical.
Re: You Should Compile Your Python and Here’s Why
#34A big portion of python's slowness to start up is that 'import' actually evaluates any code at the top level of the file it's importing.
If you just have function and class definitions, this isn't too bad, but when you start doing things like setting up caches, reading files, and testing whether or not there's a GPU at the top level, you can add several seconds to the startup and balloon your memory usage. One wrong import somewhere in your code base, and your application will craw to a standstill on launch.
Re: You Should Compile Your Python and Here’s Why
#35"Python is Slow, And That's Fine, Because It's Fast Enough" At least this is honest. No matter what the script does, no matter how fast the libraries, the Python intepreter has a slow startup time. On multiple occasions I have seen people commenting on HN argue that Python is not slow. I think for these commenters Python is "fast enough". For others, like me, it may not be "fast enough". IOW, the question is not whet…
Function calls sadly are also very slow due to heavy overhead
Re: You Should Compile Your Python and Here’s Why
#36Frankly, I would find it much easier to write the naive C version than this "moderately" optimized Python version in the article although I am much more familiar with Python than with C.
Especially reading and understanding the "moderately" optimized Python version is a lot harder than reading the C version.
Also I really dislike python optimization tricks like `output = stdout.buffer.write`.
Re: You Should Compile Your Python and Here’s Why
#37"Python is Slow, And That's Fine, Because It's Fast Enough" At least this is honest. No matter what the script does, no matter how fast the libraries, the Python intepreter has a slow startup time. On multiple occasions I have seen people commenting on HN argue that Python is not slow. I think for these commenters Python is "fast enough". For others, like me, it may not be "fast enough". IOW, the question is not whet…
I don’t always know how fast will be fast enough when I start a project and the risk of needing a rewrite really puts me off Python.
Re: You Should Compile Your Python and Here’s Why
#38Not to be shit-eating, but I never understood why people use python instead of Go other than for ML teams. I program mostly in Java and Go so would love a perspective.
Probably for the same reasons people use Java and Go instead of C. It's just more convenient sometimes. But sometimes not.
In terms of building a team at a company? In my experience it's been arbitrary. Manager / Lead has experience in language X, decides to hire people who also know it. Or all the other teams are already using language X. Or Manager / Lead has heard "X language is good at Y" and decides to go with that. Or there's simply 10x more engineers (and cheaper) available for language X than Y.
The times I've seen a language picked for a particular purpose:
- Perl/Python used for web apps. It's interpreted so you can just upload your source code and refresh your browser. Faster and simpler than having to compile/package it, lots of useful frameworks and modules, lots of developers.
- Erlang/OTP for telecom. Kind of on the nose, but there you are.
- C and C++ for embedded applications.
Re: You Should Compile Your Python and Here’s Why
#39Does anyone have any personal experience in applying mypyc on type-hinted, but otherwise not specially optimized python code? I'm especially interested in what kind of performance speed-ups could be achieved.
Re: You Should Compile Your Python and Here’s Why
#40I'm learning Python. Do someone knows a resource to learn the well-known tricks for "fast" Python? A book, a website, a cheat sheet or even a MOOC or part of a MOOC?
I would advise against optimizing your Python programs for speed. Fighting against a language's nature never leads to good results, it's much easier and better to use a faster language if (or where) you need the speed.
This is even more relevant if you are still learning the language. Focus on learning it, and leave arcane always changing implementation details for after you know it well.