Live data from Hacker News

Python-based compiler achieves orders-of-magnitude speedups

news.mit.edu

161–170 of 193 posts

Re: Python-based compiler achieves orders-of-magnitude speedups

#161

Earlier quoted context omitted.

If you code a website, your speed issues probably come from the database layer. Not your Python.

What are you doing to your database? Lol Most databases I’ve dealt with will happily outstrip Python for a good chunk of the common queries.

Measuring database speed is ultimately I/O bound, measuring a language's speed is typically CPU-bound.

Re: Python-based compiler achieves orders-of-magnitude speedups

#162

Can we please not? Humanity wasted close to 50 years optimizing compilers for one garbage language. Wasted unimaginable efforts, money and developer hours... and all could've been avoided if the same people dedicated a fraction of those resources to language design. Same thing happened with Java. And now the existence of a well-developed compiler became an argument in its own right in favor of choosing a bad language…

> if the same people dedicated a fraction of those resources to language design.

What do you mean by language design here? Is it the user-facing bits and the ergonomics? Because it seems to me (as a non python dev) that that's the bit that python devs really like.

Re: Python-based compiler achieves orders-of-magnitude speedups

#163
post #31

Earlier quoted context omitted.

Hardly a nitpick. It's key to the claim.

It's a little sad because PyPy literally is written in (a restricted subset of) Python, hence the name.

What does “literally” buy you here?

Re: Python-based compiler achieves orders-of-magnitude speedups

#164

Earlier quoted context omitted.

One great move I've discovered recently is to simply type annotate the python thoroughly and use mypyc to build a c package

Would you mind sharing more details of your experience using MypyC? What domain are you working in, and what kind of effort and speedups did you see?

Here's a detailed write-up from one of the Black maintainers: https://ichard26.github.io/blog/2022/05/compiling-black-with.... That's on the low side of the speedup curve, in my experience.

Re: Python-based compiler achieves orders-of-magnitude speedups

#165
post #154

Disclaimer: developer on Pyston, which could be considered a competitor My concern is: there have been a few projects already that are, from the outside, more or less the same approach and set of tradeoffs as this. And they haven't been that successful. Given that this is treading familiar ground I would expect some words about how this is different, and the lack thereof makes me a bit skeptical to say this will beco…

> and the lack thereof makes me a bit skeptical to say this will become successful when others did not.

Success of projects like this is not usually based on merit, but on how many people you can convince to go along with it until it eventually becomes a thing of its own sustainability. So, the recipe here would be to:

1. Be "good enough" and easy enough to get started such that early adopters have a great first experience speeding up something important to them. (hook them) 2. Be open and friendly to potential incoming contributors, letting them land changes, have a say in the discussion, and generally be part of it all. (community build) 3. Encourage people to share their successes and hopes / dreams for how great $X is on their blogs, HN, social media, etc. (propaganda) 4. Goto 1.

In this case, step 3 will work best by highlighting that "You actually don't need most of the dynamic features of Python" as the central narrative.

One big caveat is that Codon choose to not use Python semantics for `%` so the basic test of `print(-2 % 5)` fails unless you run it with `-numerics=py`... which should just be the default behavior -- and a great first community patch / discussion!

Re: Python-based compiler achieves orders-of-magnitude speedups

#166
post #109

Am losing count of all these efforts to rescue Python's performance - they all seem to amount to the same thing: it's not very hard to achieve this if you throw out fundamental aspects that make Python what it is. The premise is always that syntax is the barrier, and that people struggle so much to learn a new syntax that this is what keeps them using Python even though its performance is abysmal. But what if this is…

What is this talk about "rescuing" Python performance? Python does not need to be rescued. Its fast enough. For 90% of applications, you are kidding yourself if you need more speed. These are enhancements on Python, where you want to run stuff even faster on par with other languages.

Its performance and poor support for parallelism has prevented me from using it in places I've wanted to for years.

Is it "fast enough"? fast enough 90% of the time? Or just fast enough to leave you uncertain that it's even a good choice?

Sorry, we have productive choices now that don't leave me worrying about this situation. I still like it though, and if they could solve those issues I'd probably use it a lot more!

Re: Python-based compiler achieves orders-of-magnitude speedups

#167
post #126

Earlier quoted context omitted.

Seconded, I deploy large packages with gigabytes of deep learning and GIS dependencies in single executables with Nuitka and it works very well. Also handles including data files into the executable if needed.

Out of curiosity, are the GIS dependencies the proprietary ones ( cough ESRI cough )?

Please no, not ArcGIS. I’m sure our dependencies would clash with arcpy’s if we tried. Fortunately my company uses FOSS packages almost exclusively.

Re: Python-based compiler achieves orders-of-magnitude speedups

#168

Since this is highly incompatible with most python ecosystem right now, may I plug nuitka? https://nuitka.net/index.html It's a compiler for python code that can create stand alone executables, and up to 4 times the speed of the initial code. Best of all, it's extremely reliable, with a high level of support of event the tricky things like the scientic and gui stacks.

> high level of support of event the tricky things like the scientic and gui stacks Could it compile an app that uses Pillow and AggDraw and ReportLab and OpenPyXL with a TKInter GUI into a standalone app I can give to a coworker? That would be extremely useful!

I don’t know half of these but I’m almost sure that Pillow and TKInter would work.

Re: Python-based compiler achieves orders-of-magnitude speedups

#169
post #168

Earlier quoted context omitted.

> high level of support of event the tricky things like the scientic and gui stacks Could it compile an app that uses Pillow and AggDraw and ReportLab and OpenPyXL with a TKInter GUI into a standalone app I can give to a coworker? That would be extremely useful!

I don’t know half of these but I’m almost sure that Pillow and TKInter would work.

AggDraw is an "anti-grain geometry" graphics library that works with Pillow for drawing high quality images. ReportLab is a very big PDF generating library, and OpenPyXL reads and writes XLSX format Excel spreadsheets. I use these in many of my work-related apps. Tkinter is the big question for me because it involves a lot of behind-the-scenes files. Thanks for your comment, I will give Nuitka a try!

Re: Python-based compiler achieves orders-of-magnitude speedups

#170

Earlier quoted context omitted.

> high level of support of event the tricky things like the scientic and gui stacks Could it compile an app that uses Pillow and AggDraw and ReportLab and OpenPyXL with a TKInter GUI into a standalone app I can give to a coworker? That would be extremely useful!

I’ve used Nuitka to package an app that involves Pillow, mupdf, PyQt, and several other libraries, and it handles them with no problem.

That's very encouraging, I will give it a go!

And I have to ask, does a powersnail live in a powershell? =) (also a language I like that needs better GUI and EXE packaging)

Post reply on HN