An update about the Software Transactional Memory subproject of PyPy

#1

Post by tshepang » Wed, Jun 05, 2013, 6:56 PM UTC

An update about the Software Transactional Memory subproject of PyPy
morepypy.blogspot.com

Re: An update about the Software Transactional Memory subproject of PyPy

#2

Post by apendleton » Wed, Jun 05, 2013, 8:11 PM UTC

If I'm reading the post correctly, the HN title is inaccurate. The in-progress STM was already written in C. It was the garbage collector that was written in RPython, and that has been rewritten in C as part of a combined STM/GC library.

edit: fixed now.

Re: An update about the Software Transactional Memory subproject of PyPy

#3

Post by tshepang » Wed, Jun 05, 2013, 8:22 PM UTC

apendleton wrote: post #2

If I'm reading the post correctly, the HN title is inaccurate. The in-progress STM was already written in C. It was the garbage collector that was written in RPython, and that has been rewritten in C as part of a combined STM/GC library. edit: fixed now.

Thanks for the correction. Must be more careful next time.

Re: An update about the Software Transactional Memory subproject of PyPy

#4

Post by hartror » Wed, Jun 05, 2013, 11:06 PM UTC

The original blog post they did on STM is well worth reading, especially if you're not familiar with the python GIL and STM.

http://morepypy.blogspot.com.au/2011/08/we-need-software-tra...

Re: An update about the Software Transactional Memory subproject of PyPy

#5

Post by fiatmoney » Thu, Jun 06, 2013, 2:14 AM UTC

Any potential to leverage the Haswell hardware TM intrinsics?

Re: An update about the Software Transactional Memory subproject of PyPy

#6

Post by timbaldridge » Thu, Jun 06, 2013, 2:38 AM UTC

fiatmoney wrote: post #5

Any potential to leverage the Haswell hardware TM intrinsics?

As mentioned in their 2013 PyCon talk, the problem with HTM (at least with current designs) is the hard limits to transaction sizes. They are attempting run run entire interpreted code loops in transactions, while HTM is designed more around smaller transactions.

Specifically I think Haswell requires that all mutated references must fit in the L1 cache. That doesn't really leave much wiggle room

Re: An update about the Software Transactional Memory subproject of PyPy

#7

Post by Tobu » Thu, Jun 06, 2013, 11:23 AM UTC

hartror wrote: post #4

The original blog post they did on STM is well worth reading, especially if you're not familiar with the python GIL and STM. http://morepypy.blogspot.com.au/2011/08/we-need-software-tra...

This is also informative: http://pypy.org/tmdonate.html

There haven't been any recent blog posts on that topic (and they aren't tagged consistently), this is the latest, which may explain the choice to go with C: http://morepypy.blogspot.com/2012/08/multicore-programming-i...