Live data from Hacker News

Python-based compiler achieves orders-of-magnitude speedups

news.mit.edu

81–90 of 193 posts

Re: Python-based compiler achieves orders-of-magnitude speedups

#81

>“Google users in America have searched for Python more often than for Kim Kardashian.” I wonder how Kim Kardashian programming language looks like. I guess low level but with garbage collector. :D

It's all garbage collector.

Seems like more like a scaled-up garbage producer than garbage collector...

Re: Python-based compiler achieves orders-of-magnitude speedups

#82
post #2

There are other python implementations like pypy which includes a JIT (Just In Time compiler). There are other jit which can run with official python (cpython) like numba (not all code can be optimized, but if you only need optimize your hot code path). You can use a superset language of python called cython that generate C code. It can be used to generate C bindings or fast python (for cpython) modules implemented i…

One great move I've discovered recently is to simply type annotate the python thoroughly and use mypyc to build a c package

Re: Python-based compiler achieves orders-of-magnitude speedups

#83

Surprised there is no comparison to MyPyC. That said the availability of a "JIT" compiler in the style of Numba but with much broader Python feature support sounds great to me.

I took their fib example and ran it in mypyc too out of curiosity. I got speedups of ~10x rather than codon's 100x. Still pretty good, I like mypyc.

Re: Python-based compiler achieves orders-of-magnitude speedups

#84
post #77
post #74

Earlier quoted context omitted.

A fair implication, but they mean "Python-based" in that the language the compiler implements is based on Python.

They should have said python compiler (shorter) or python compiler in c++ (more accurate and only one character longer, including spaces). Considering at least 2 people have gone to look at the source and then come here to comment, it would have been a net benefit for all involved. Plus, what does it say about the potential quality of your compiler if you can't even make correct English statements? This seems easier…

> This seems easier to get right than if( x = *p++ )

For people with native or fluent English, for sure. For the others, probably not.

Re: Python-based compiler achieves orders-of-magnitude speedups

#85

Surprised there is no comparison to MyPyC. That said the availability of a "JIT" compiler in the style of Numba but with much broader Python feature support sounds great to me.

I took their fib example and ran it in mypyc too out of curiosity. I got speedups of ~10x rather than codon's 100x. Still pretty good, I like mypyc.

mypyc keeps Python's "BigIntegers", unicode string implementation, reference counting, and has little to no floating point-related optimizations yet. It prioritizes compatibility over overall performance, so I'm not surprised. I was also disappointed at how poor mypyc is at compiling across multiple files, but that they can fix at some point.

The BigInteger "issue" pretty much makes something like Fibonacci a worst case scenario for it.

Re: Python-based compiler achieves orders-of-magnitude speedups

#86

>“Google users in America have searched for Python more often than for Kim Kardashian.” I wonder how Kim Kardashian programming language looks like. I guess low level but with garbage collector. :D

Since I have the world's greatest creative assistant handy, here's what GPT-4 thinks a Kim Kardashian language would look like:

Hey dolls, let me introduce you to the Kimmie programming language, it's like totally fab and easy to use!

To declare a variable, just use the hashtag symbol and the variable name, like this:

#my_var

To assign a value to the variable, use the word "like" followed by the value, like this:

#my_var like 10

To print out a message, use the word "OMG" followed by the message in double quotes, like this:

OMG "Hello, dolls!"

To add two variables together, use the word "add" followed by the two variables, like this:

#var1 like 5 #var2 like 7 #sum like add #var1 #var2

Re: Python-based compiler achieves orders-of-magnitude speedups

#87
post #3

So the differences: https://docs.exaloop.io/codon/general/differences So more limited types (integers) and more type checking and collections have to have one kind of thing in them. There are other python compilers though, like https://github.com/Nuitka/Nuitka I wonder really what the advantages/disadvantages of these are?

>a few of Python's dynamic features are disallowed. For example, [...] or adding objects of different types to a collection.

The fine print is strong with this one. It makes me wonder why they didn't just start with RPython.

Re: Python-based compiler achieves orders-of-magnitude speedups

#88
post #51

Earlier quoted context omitted.

A lot of effort is dedicated to trying to improve the speed because python is so widely used that improving performance could have a massive beneficial impact. Migrating to a new language is not easy when you have millions of lines of code.

I didn't say that anyone should switch (see my preface), and I perfectly well understand this point. Having to retread this point over and over just serves to make our comments long and redundant and full of qualifiers, but I guess here we are again.

You said you don’t want to “crap on python” which I never said you were. I’m simply pointing out that what you’re “shocked by” makes complete sense if you look at what can be gained by people “bikeshed and optimize and compile python”

Re: Python-based compiler achieves orders-of-magnitude speedups

#89
post #67
post #60

Earlier quoted context omitted.

And yet the same supposedly "io bound" workloads (like "parse request, fetch something from DB, return it as JSON") still have widely difference performance characteristics in some languages vs others, with 10x to 100x requests handled per second...

2x, yes. If you’re seeing 100x you’re comparing different things like creating and serializing complex objects versus simple types or using a JSON parser which loads an entire document into objects versus one which only retrieved specific values.

Well, perhaps not 100x in rps, but 10x sure:

Overall top performing frameworks (JS, Java, and Rust) at 650K rps. That's 7x over the top Python based framework at: 86K rps.

And another very popular Python framework (flask) gets just to 2K rps. That's 325 times worse to the best.

And that's the "single DB query" benchmark: https://www.techempower.com/benchmarks/#section=data-r21&tes...

https://github.com/TechEmpower/FrameworkBenchmarks/wiki/Proj...

Re: Python-based compiler achieves orders-of-magnitude speedups

#90
post #77
post #74

Earlier quoted context omitted.

A fair implication, but they mean "Python-based" in that the language the compiler implements is based on Python.

They should have said python compiler (shorter) or python compiler in c++ (more accurate and only one character longer, including spaces). Considering at least 2 people have gone to look at the source and then come here to comment, it would have been a net benefit for all involved. Plus, what does it say about the potential quality of your compiler if you can't even make correct English statements? This seems easier…

These headlines usually work by the dept asking a researcher for a 10 sentence summary of their work. Someone in the dept summarizes that to 3 sentences, and sends that to the university pr dept. The university turns that 3 sentences into 1 and that's what's published. My take is this says more about the weird game of telephone being played than it does about the research product itself.
Post reply on HN