Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Numba: A High Performance Python Compiler

numba.pydata.org

Post reply on HN
Print view View original

1–10 of 63 posts

Re: Numba: A High Performance Python Compiler

#2

Post by chazeon »

Software from our group (cij[1], qha[2]) were developed when numba seems to be the best option for JIT. It generates more pain in the hindsight. It generates a lot of depreciated warning due to unstable API, locked numpy to a certain version (i remember 1.21) due to compatibility issues, and when M1 Mac comes out, there were for a long time lack of llvmlite porting to the new platform, so cannot run on these new Macs.

If I had to do it again I would just use plain numpy or use the JAX from Google if JIT is really necessary.

[1]: https://github.com/MineralsCloud/cij

[2]: https://github.com/MineralsCloud/qha

Re: Numba: A High Performance Python Compiler

#4

Post by gjvc »

chazeon wrote: post #2

Software from our group (cij[1], qha[2]) were developed when numba seems to be the best option for JIT. It generates more pain in the hindsight. It generates a lot of depreciated warning due to unstable API, locked numpy to a certain version (i remember 1.21) due to compatibility issues, and when M1 Mac comes out, there were for a long time lack of llvmlite porting to the new platform, so cannot run on these new Macs…

What if I'm (in Python) doing non-numerical stuff like parsing text and generating code? What JIT / AOT tooling (if any) is suitable?

Re: Numba: A High Performance Python Compiler

#7

Post by _Wintermute »

I've only used numba once but I was really impressed. We have an analysis at work that runs hundreds of times a day which uses a Hampel filter written in numpy, but still requires iterating over an array. Just adding a @numba.jit decorator above the function gave us a 10x speed improvement.

Re: Numba: A High Performance Python Compiler

#8

Post by samsquire »

How would this compare to Pypy?

I didn't think Pypy uses LLVM so I wonder who produced better code.

That said, they're targeted at different audiences. I feel Numba is targeted at data science and machine learning and even AI.

I feel a large portion of using or programming a computer is structural and not the actual work of adding numbers together. Very little of the code generated does the useful part a computer does: addition. The rest is control flow management and data placement! It's all preparation for the code to do an addition. The hard part is putting together the structure for the computer to do things that are useful.

So we invented methods, variables, classes, functions, closures, expressions to create that structure easier.

I thought about creating a language which tries to eliminate the structure that most programs accumulate and focus on the critical addition or calculation and let the computer do the arrangement. A JIT compiler for structure.

Re: Numba: A High Performance Python Compiler

#9

Post by auxym »

gjvc wrote: post #4
chazeon wrote: post #2

Software from our group (cij[1], qha[2]) were developed when numba seems to be the best option for JIT. It generates more pain in the hindsight. It generates a lot of depreciated warning due to unstable API, locked numpy to a certain version (i remember 1.21) due to compatibility issues, and when M1 Mac comes out, there were for a long time lack of llvmlite porting to the new platform, so cannot run on these new Macs…

What if I'm (in Python) doing non-numerical stuff like parsing text and generating code? What JIT / AOT tooling (if any) is suitable?

Pypy, probably. You could also consider writing pre compiled extensions for your "hot" code, eg. in Cython.

Re: Numba: A High Performance Python Compiler

#10

Post by ptype »

gjvc wrote: post #4
chazeon wrote: post #2

Software from our group (cij[1], qha[2]) were developed when numba seems to be the best option for JIT. It generates more pain in the hindsight. It generates a lot of depreciated warning due to unstable API, locked numpy to a certain version (i remember 1.21) due to compatibility issues, and when M1 Mac comes out, there were for a long time lack of llvmlite porting to the new platform, so cannot run on these new Macs…

What if I'm (in Python) doing non-numerical stuff like parsing text and generating code? What JIT / AOT tooling (if any) is suitable?

I think what will be the most maintainable and bring you the least long term pain is Cython
Post reply on HN