Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

First Python 2.7 interpreter to use multiple cores

mail.python.org

Post reply on HN
Print view View original

11–20 of 42 posts

Re: First Python 2.7 interpreter to use multiple cores

#11

Post by slurgfest »

tocomment wrote: post #2

I'm not following this. Does it mean the GIL is finally gone?

I will get excited when it actually works out to be faster than interpreters with the GIL.

I'm a bit disappointed that Python seems to have no interest in standardizing a lightweight-thread (greenlet, coroutine) interface that isn't an awful mess. yield is powerful but coroutine implementations with it are baroque and opaque. Java-style threading has a million gotchas and the interface is just a dog. I want to be able to teach people with a week of experience to write concurrent programs. (What about writing turtle programs with multiple turtles running at the same time?)

If Python doesn't do it, then I want someone else to eat their lunch.

Re: First Python 2.7 interpreter to use multiple cores

#12

Post by jmtulloss »

slurgfest wrote: post #11
tocomment wrote: post #2

I'm not following this. Does it mean the GIL is finally gone?

I will get excited when it actually works out to be faster than interpreters with the GIL. I'm a bit disappointed that Python seems to have no interest in standardizing a lightweight-thread (greenlet, coroutine) interface that isn't an awful mess. yield is powerful but coroutine implementations with it are baroque and opaque. Java-style threading has a million gotchas and the interface is just a dog. I want to be abl…

Stackless works fine for this kind of thing.

Re: First Python 2.7 interpreter to use multiple cores

#13

Post by freyrs3 »

halayli wrote: post #6

I much prefer share-nothing approach and use sockets + subprocess for communication. It scales nicely.

Depends on what you're doing. Shoving a numpy array with 10^7 elements over the wire isn't going to scale nicely at all. There are still a lot of computing scenarios having shared memory is the best approach.

Re: First Python 2.7 interpreter to use multiple cores

#14

Post by jandrewrogers »

freyrs3 wrote: post #13
halayli wrote: post #6

I much prefer share-nothing approach and use sockets + subprocess for communication. It scales nicely.

Depends on what you're doing. Shoving a numpy array with 10^7 elements over the wire isn't going to scale nicely at all. There are still a lot of computing scenarios having shared memory is the best approach.

You do not need threads for shared memory. I know of quite a few high-performance software systems that use process isolation and yet leverage shared memory via mechanisms like mmap() for IPC.

Re: First Python 2.7 interpreter to use multiple cores

#15

Post by Tobu »

slurgfest wrote: post #11
tocomment wrote: post #2

I'm not following this. Does it mean the GIL is finally gone?

I will get excited when it actually works out to be faster than interpreters with the GIL. I'm a bit disappointed that Python seems to have no interest in standardizing a lightweight-thread (greenlet, coroutine) interface that isn't an awful mess. yield is powerful but coroutine implementations with it are baroque and opaque. Java-style threading has a million gotchas and the interface is just a dog. I want to be abl…

Which of the existing coroutine implementations have you tried, and what did you dislike about it?

Re: First Python 2.7 interpreter to use multiple cores

#16

Post by seunosewa »

jandrewrogers wrote: post #14
freyrs3 wrote: post #13

Earlier quoted context omitted.

Depends on what you're doing. Shoving a numpy array with 10^7 elements over the wire isn't going to scale nicely at all. There are still a lot of computing scenarios having shared memory is the best approach.

You do not need threads for shared memory. I know of quite a few high-performance software systems that use process isolation and yet leverage shared memory via mechanisms like mmap() for IPC.

Good point, but shared memory via mmap isn't very Pythonic since you have to manage the shared memory manually, you can't allocate Python objects on the shared region, etc.

Re: First Python 2.7 interpreter to use multiple cores

#17

Post by freyrs3 »

seunosewa wrote: post #16
jandrewrogers wrote: post #14

Earlier quoted context omitted.

You do not need threads for shared memory. I know of quite a few high-performance software systems that use process isolation and yet leverage shared memory via mechanisms like mmap() for IPC.

Good point, but shared memory via mmap isn't very Pythonic since you have to manage the shared memory manually, you can't allocate Python objects on the shared region, etc.

Yes, but if you're doing HPC idiomatic Python comes second to raw performance.

Re: First Python 2.7 interpreter to use multiple cores

#18

Post by jlgreco »

Tobu wrote: post #15
slurgfest wrote: post #11

Earlier quoted context omitted.

I will get excited when it actually works out to be faster than interpreters with the GIL. I'm a bit disappointed that Python seems to have no interest in standardizing a lightweight-thread (greenlet, coroutine) interface that isn't an awful mess. yield is powerful but coroutine implementations with it are baroque and opaque. Java-style threading has a million gotchas and the interface is just a dog. I want to be abl…

Which of the existing coroutine implementations have you tried, and what did you dislike about it?

In my humble opinion, the greatest problem with all of them is that there are so many. Most of them are rather fine I think, but it can just be a serious pain in the ass getting your head around all of them.

Re: First Python 2.7 interpreter to use multiple cores

#20

Post by Tobu »

jlgreco wrote: post #18
Tobu wrote: post #15

Earlier quoted context omitted.

Which of the existing coroutine implementations have you tried, and what did you dislike about it?

In my humble opinion, the greatest problem with all of them is that there are so many. Most of them are rather fine I think, but it can just be a serious pain in the ass getting your head around all of them.

I've only needed one (Twisted, which is actually slightly outside the space occupied by gevent et al). You can't stop people from building more, but feel free to bet on a popular project if you don't want to deal with choice.
Post reply on HN