Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codon: A high-performance Python-like compiler using LLVM

github.com

Post reply on HN
Print view View original

111–120 of 184 posts

Re: Codon: A high-performance Python-like compiler using LLVM

#111

Post by darawk »

wpietri wrote: post #69
darawk wrote: post #51

Earlier quoted context omitted.

That list actually seems genuinely pretty minimal. Reading your comment I was expecting a long major list of changes, but it's only 3 things, most of which seem relatively unlikely to impact most programs, with the possible exception of dictionary sort order.

Really? The lack of Unicode strings immediately disqualifies this for most things I've worked on the last few years. No emojis, no diacritics, no non-US users. Ok for internal tools for American companies with an older workforce, I guess, but I wouldn't use this for anything that takes input from the general public (e.g., customers).

I suppose I'm thinking more about data science / engineering oriented things, since that's what I tend to use Python for.

Re: Codon: A high-performance Python-like compiler using LLVM

#112

Post by bastawhiz »

PaulHoule wrote: post #89

How does this relate to https://cython.org/ ? Would it be possible to write performance-sensitive parts of a Python system in Codon and link that to a CPython or PyPy runtime that supports more dynamic features?

Cython takes python-ish code and compiles it to C for use as CPython C extensions. This compiles directly to machine code without the need for CPython, as far as I can tell.

Re: Codon: A high-performance Python-like compiler using LLVM

#113

Post by register »

miohtama wrote: post #103
jnxx wrote: post #91

Just out of curiosity: Why is it possible to compile Common Lisp Code (or Scheme, or Clojure) to high-performance native or jit-compiled code, but not Python? It is said that "Python is too dynamic", but is not everything in Lisp dynamic, too? And none of these languages is less powerful than Lisp, lack Unicode support, or whatever, so this can't be the reason.

It’s because Python object attributes can change any time, as they are accessed dynamically. Nothing can be inlined easily. The object structure is pointer heavy. Here is some old 2014 post: http://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/ As other commenters pointed out, some of these Python features, which are unused 99,99% time, could be sacrified for additional speedup by breaking backwards compatibil…

The same applies to Common Lisp. Maybe it's because type deduction is more difficult in Python than in CL?

Re: Codon: A high-performance Python-like compiler using LLVM

#114

Post by adsharma »

ot wrote: post #34

Can we change the title to say Python-like or something similar? Based on the comments so far, it seems that the detail that it compiles its own Python-inspired language, not actual Python, is lost on many. EDIT: A list of differences here: https://docs.exaloop.io/codon/general/differences The summary minimizes with "many Python programs will work with few if any modifications", but it actually looks like a substanti…

For py2many, there is an informal specification here:

https://github.com/py2many/py2many/blob/main/doc/langspec.md

Would be great if all the authors of "python-like" languages get together and come up with a couple of specs.

I say a couple, because there are ones that support the python runtime (such as cython) and the ones which don't (like py2many).

Re: Codon: A high-performance Python-like compiler using LLVM

#115

Post by redleader55 »

sillysaurusx wrote: post #53
redleader55 wrote: post #5

Free for non-production use... it's a "no" for me.

Surprised this sentiment is so common. It's like, do you want open source devs to work for free forever? Even Redis had to pivot to a business license. I'm not sure what the terms are in this particular case, but in general, wanting someone to pay if they're deploying it to lots of customers seems reasonable.

There are a lot of products released under GPL, for example, that make money for their authors. It's just that they don't make money with licensing fees.

Re: Codon: A high-performance Python-like compiler using LLVM

#116

Post by __mharrison__ »

munificent wrote: post #58

Since Codon performs static type checking ahead of time, a few of Python's dynamic features are disallowed. For example, monkey patching classes at runtime (although Codon supports a form of this at compile time) or adding objects of different types to a collection. This seems like a very different language from Python if it won't let you do: [1, 'a string']

Not "Python", but if you are doing that in Python, then you are doing it wrong.

Re: Codon: A high-performance Python-like compiler using LLVM

#117

Post by mytherin »

jnxx wrote: post #91

Just out of curiosity: Why is it possible to compile Common Lisp Code (or Scheme, or Clojure) to high-performance native or jit-compiled code, but not Python? It is said that "Python is too dynamic", but is not everything in Lisp dynamic, too? And none of these languages is less powerful than Lisp, lack Unicode support, or whatever, so this can't be the reason.

It is possible to JIT compile Python just fine. There are projects like PyPy that have been doing this for a long time [1]. The reason these alternative projects never take off is because many of Python's most used libraries are written against CPython's C API. This API is giant, and exposes all of the nitty gritty implementation details of the CPython interpreter. As a result, changing anything significant about the implementation of the interpreter means those libraries no longer work. In order to not break compatibility with the enormous amounts of packages the internals of the CPython interpreter are mostly locked in at this point with little wiggle room for large performance improvements.

The only real way out is to make Python 4 - but given the immense pain of the Python 2 -> 3 transition that seems unlikely.

[1] https://www.pypy.org

Re: Codon: A high-performance Python-like compiler using LLVM

#118

Post by amelius »

terhechte wrote: post #105
joshmaker wrote: post #95

Earlier quoted context omitted.

In 25 years you’ve never once created a list with more than one type of object in it?

Same here. If I needed that I might use a tuple.

And then it crashes when you convert it to JSON.

Re: Codon: A high-performance Python-like compiler using LLVM

#119

Post by syrusakbary »

weinzierl wrote: post #85
odo1242 wrote: post #61

Earlier quoted context omitted.

WebAssembly? Try compiling something to WebAssembly and running it in python?

I haven't thought of that. It's a good idea. I know how to compile to WebAssembly but how do I run it in python? A quick search leads to pywasm and it is even native python. But is it usable? Any other options?

Wasmer Python can be used if you want to run Wasm in Python. Hope this helps!

https://github.com/wasmerio/wasmer-python

Re: Codon: A high-performance Python-like compiler using LLVM

#120

Post by amelius »

dr_zoidberg wrote: post #29
harvie wrote: post #18

Unfortunately stuff like this never makes it to the upstream. And i am afraid to ask why. We had pypy for years, but never got merged with python. That is why there are still minor incompatibilities between pypy and "The Python", so it's not that useful as it might have been if it got merged with cpython at some point.

I got a massive jump in performance when moving from Python 3.8 to 3.10 (over some function call optimizations I think, based on the project). And 3.11 got even better (up to 50% faster on special cases, and 10~15% on average) with respect to 3.10. Python 3.12 is already getting even more speedups and a there's a lot more down the road[0]. But Python core developers value keeping "not breaking anyones code" (Python 3…

Interesting. What is the status of the GIL these days?
Post reply on HN