Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's Write an LLVM Specializer for Python

dev.stephendiehl.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Let's Write an LLVM Specializer for Python

#11

Post by exDM69 »

danohuiginn wrote: post #8
exDM69 wrote: post #6

Earlier quoted context omitted.

Simply put: it's a better language. The whole "discussion" is whether it makes sense to migrate since many parts of the ecosystem (many important libraries and frameworks) have not made the transition. And many distros ship Python 2 by default, Python 3 is optional. Python 3 only is not feasible. To me, the killer feature is better lazy evaluation (generators). In particular, important builtins like map, filter, zip,…

(process(line) for line in open('giantfile.txt') if line.lstrip()[0] != '#') Is that line really using any new features in Python 3? The lazy evaluation there is in the file object and the generator expression, both of which have long been present in python2.

No, it wasn't. I added map(str.upper, xxx), now it does.

Re: Let's Write an LLVM Specializer for Python

#12

Post by danohuiginn »

halflings wrote: post #7
exDM69 wrote: post #6

Earlier quoted context omitted.

Simply put: it's a better language. The whole "discussion" is whether it makes sense to migrate since many parts of the ecosystem (many important libraries and frameworks) have not made the transition. And many distros ship Python 2 by default, Python 3 is optional. Python 3 only is not feasible. To me, the killer feature is better lazy evaluation (generators). In particular, important builtins like map, filter, zip,…

The example you gave works perfectly in Python 2.7 (would also be a generator, and you're not using map filter or else); but I agree: those should've been generators from day 1, especially zip and enumerate since they make more elegant code but often come with a performance overhead in Python 2.7

Turning them into generators would have broken a lot of existing code, though, so it's reasonable to leave them until a major version change. Rather, these are the kind of small, obviously-useful changes that should have come immediately in 3.0, giving people some encouragement to switch.

Re: Let's Write an LLVM Specializer for Python

#13

Post by maxerickson »

halflings wrote: post #7
exDM69 wrote: post #6

Earlier quoted context omitted.

Simply put: it's a better language. The whole "discussion" is whether it makes sense to migrate since many parts of the ecosystem (many important libraries and frameworks) have not made the transition. And many distros ship Python 2 by default, Python 3 is optional. Python 3 only is not feasible. To me, the killer feature is better lazy evaluation (generators). In particular, important builtins like map, filter, zip,…

The example you gave works perfectly in Python 2.7 (would also be a generator, and you're not using map filter or else); but I agree: those should've been generators from day 1, especially zip and enumerate since they make more elegant code but often come with a performance overhead in Python 2.7

Python 2 enumerate returns an 'enumerate' object that is more or less a light weight wrapper of the sequence that was passed in.

Generators provide a convenient syntax to implement that sort of object.

Re: Let's Write an LLVM Specializer for Python

#14

Post by exDM69 »

maxerickson wrote: post #13
halflings wrote: post #7

Earlier quoted context omitted.

The example you gave works perfectly in Python 2.7 (would also be a generator, and you're not using map filter or else); but I agree: those should've been generators from day 1, especially zip and enumerate since they make more elegant code but often come with a performance overhead in Python 2.7

Python 2 enumerate returns an 'enumerate' object that is more or less a light weight wrapper of the sequence that was passed in. Generators provide a convenient syntax to implement that sort of object.

D'oh. I put in a map(), that returns a list.

Re: Let's Write an LLVM Specializer for Python

#15

Post by exDM69 »

danohuiginn wrote: post #12
halflings wrote: post #7

Earlier quoted context omitted.

The example you gave works perfectly in Python 2.7 (would also be a generator, and you're not using map filter or else); but I agree: those should've been generators from day 1, especially zip and enumerate since they make more elegant code but often come with a performance overhead in Python 2.7

Turning them into generators would have broken a lot of existing code, though, so it's reasonable to leave them until a major version change. Rather, these are the kind of small, obviously-useful changes that should have come immediately in 3.0, giving people some encouragement to switch.

Most of the breaking changes (including making map, zip, filter, etc lazy) were done in Python 3.0.

Re: Let's Write an LLVM Specializer for Python

#16

Post by exDM69 »

gamesbrainiac wrote: post #5
gtirloni wrote: post #4

Earlier quoted context omitted.

there is this discussion (flame war?) about python3 not bringing too many benefits. i haven't made my mind yet. could you elaborate what you saw in 3.4 that was nice?

I'd like to know too. Are there any LLVM specific enhancements that python 3.4 brings to the table?

No. AFAIK, there's nothing related to LLVM in core Python. And not in 3.4 changes either.

Re: Let's Write an LLVM Specializer for Python

#17

Post by Erwin »

Interesting. I currently use Python's AST to convert some nested logical query expression (in a syntax unique to my application) into bytecode executed by a specialized VM (I originally tried using V8 and LuaJit for this but performance wise that was unsuccessful; the project replaced some old Boost::Python C++ code). This article should make it easy to get started attempting an LLVM replacement.

Re: Let's Write an LLVM Specializer for Python

#18

Post by baq »

gtirloni wrote: post #4
walkamages wrote: post #2

An excellent article! I had wanted to get back into some python recently after seeing the changes in 3.4, I had also wanted to become more familiar with LLVM, and this does both.

there is this discussion (flame war?) about python3 not bringing too many benefits. i haven't made my mind yet. could you elaborate what you saw in 3.4 that was nice?

much better exceptions and sane unicode are the biggest improvements.

Re: Let's Write an LLVM Specializer for Python

#19

Post by smazga »

exDM69 wrote: post #6
gtirloni wrote: post #4

Earlier quoted context omitted.

there is this discussion (flame war?) about python3 not bringing too many benefits. i haven't made my mind yet. could you elaborate what you saw in 3.4 that was nice?

Simply put: it's a better language. The whole "discussion" is whether it makes sense to migrate since many parts of the ecosystem (many important libraries and frameworks) have not made the transition. And many distros ship Python 2 by default, Python 3 is optional. Python 3 only is not feasible. To me, the killer feature is better lazy evaluation (generators). In particular, important builtins like map, filter, zip,…

We went to Python 3 for the multiprocessing module. At the time it was 3.3, but now 3.4 has all that async magic. I wish I still worked on that project.
Post reply on HN