Ask HN: Looking for an open-source project to parallelise

#1

Post by pufuwozu » Thu, Jul 29, 2010, 6:44 AM UTC

I've just started a Parallel Programming unit at University and our major project (worth 50%) is to take an application and manually parallelise it to scale across multiple cores.

I'd like to do the assignment where the target:

* Is interesting

* Requires a fair amount of effort and skill

* Has some sort of barrier to making it parallel

* Is welcoming to me taking up the project

Does anyone know a project that fulfils most of these? The project's language or libraries aren't an issue.

Re: Ask HN: Looking for an open-source project to parallelise

#2

Post by karanbhangui » Thu, Jul 29, 2010, 1:22 PM UTC

how about one of the common compression libraries like zip or gzip?

Re: Ask HN: Looking for an open-source project to parallelise

#3

Post by Jtsummers » Thu, Jul 29, 2010, 2:55 PM UTC

You could look into accelerating some of the common filters/operations in programs like Audacity, GIMP or some other open source media editor.

Re: Ask HN: Looking for an open-source project to parallelise

#4

Post by jsarch » Thu, Jul 29, 2010, 8:21 PM UTC

Are you looking for a project no one has done yet? Or a project to simply learn from? As an HPC programmer I can tell you that there are lots of scientific applications that you could try. However, I'd suggest that you start with something manageable and known to be parallelizable so that you can feel great when you finish it.

I think @jtsummers and @karanbhangui have the right idea, so I'd suggest making the Apache2 mod_deflate module parallel.

Re: Ask HN: Looking for an open-source project to parallelise

#5

Post by VanL » Fri, Jul 30, 2010, 4:54 AM UTC

Try Python. There is lots of interest in getting CPython to scale across multiple cores, but the use of refcounting to manage object lifetime and the Global Interpreter Lock makes it hard.

If you want, you could even try to prototype using PyPy (python in python).

Re: Ask HN: Looking for an open-source project to parallelise

#6

Post by ggbaker » Fri, Jul 30, 2010, 5:46 AM UTC

VanL wrote: post #5

Try Python. There is lots of interest in getting CPython to scale across multiple cores, but the use of refcounting to manage object lifetime and the Global Interpreter Lock makes it hard. If you want, you could even try to prototype using PyPy (python in python).

Honestly, I think this one is way beyond an undergraduate project. You aren't the first one to think Python should be parallelized: many smart people have tried to do it and gotten nowhere. Some of the other libraries mentioned here will be a more reasonable size (unless "unit" means PhD :-).

Re: Ask HN: Looking for an open-source project to parallelise

#7

Post by pufuwozu » Fri, Jul 30, 2010, 10:05 PM UTC

ggbaker wrote: post #6

VanL wrote: post #5

Try Python. There is lots of interest in getting CPython to scale across multiple cores, but the use of refcounting to manage object lifetime and the Global Interpreter Lock makes it hard. If you want, you could even try to prototype using PyPy (python in python).

Honestly, I think this one is way beyond an undergraduate project. You aren't the first one to think Python should be parallelized: many smart people have tried to do it and gotten nowhere. Some of the other libraries mentioned here will be a more reasonable size (unless "unit" means PhD :-).

It's only an undergraduate subject. Python is way off limits.

Thanks for all of the suggestions guys! This has really helped a lot.