Live data from Hacker News

The future of programming languages in a massively concurrent world

news.ycombinator.com

11–20 of 33 posts

Re: The future of programming languages in a massively concurrent world

#11
post #10
post #9

Earlier quoted context omitted.

Concurrency in todays web darlings, Ruby, PHP, and Python will be challenging. Heard of stackless python? [ http://www.stackless.com/ ]. Eve Online is the highest profile app using it I know of. Introduction to Concurrent Programming with Stackless Python [ http://members.verizon.net/olsongt/stackless/why_stackless.h... ]

Stackless is interesting, but I found it to be a little unwieldy when I wrote a volume visualization test. When each task needs access to the entire volume of data, Stackless gets REALLY slow. I didn't look through the Stackless internals the way I looked into Erlang, but the slowdown was undeniable. Again, for massive datasets, accessed by many cores over a massive number of threads, it needs a little more developme…

That's an excellent test algorithm because of the shared memory issues. Might be better if the rays affected the environment -- say laser beams. (For some reason I have a picture of sharks with laser beams)

Ray-tracing is a nice, simple problem domain -- enough to be complicated, but not too complicated.

Re: The future of programming languages in a massively concurrent world

#12
Occam.

Specifically, http://transterpreter.org

Yes, the language it runs (Occam) is 20 years old. But the language was designed for programs running on dozens to thousands of nodes, and in the transterpreter implementation, there's the possibility of doing this on heterogeneous hardware, where the fast nodes do things like splitting and merging the data set, and the smaller "grunt compute" nodes do the actual work.

Parallel programming is hard, but that's inherent hardness. You can't get around things like memory bandwidth and latency at a programming language level, no matter how much you try. You can only get away from those things by dealing with the fact you have thousands of machines, or tens of thousands.

It's only going to get worse from here on in, as "faster" comes to mean more processors, not higher clock rates. You'll see this: 2 core! 3 core! 4 core! 8 core! and pretty soon (within 10 years) we'll see 64 and 128 core desktop machines, maybe even a revival of unusual architectures like wafter scale integration with 3D optical interconnects (i.e. upward pointing tiny lasers and photocells fabricated on the chip) to handle getting data on and off the processors.

We've seen unambiguously that GIGANTIC data sets have their own value. Google's optimization of their algorithms clearly uses enormous amounts of observed user behavior. Translation efforts with terabyte source cannons. Image integration algorithms like that thing that Microsoft were demonstrating recently... gigantic data sets have power because statistics draw relationships out of the real world, rather than having programmers guessing about what the relationships are.

I strongly suspect that 20 years from now, there are going to be three kinds of application programming:

1> Interface programming

2> Desktop programming (in the sense of programming things which operate on your personal objects - these things are like pens and paper and you have your own.)

3> Infrastructure programming - supercomputer cluster programming (Amazon and Google are supercomputer applications companies) - which will provide yer basic services.

One of the concepts I'm pitching to the military right now is using the massive data sets they have from satellite sources to provide "precision agriculture" support for the developing world. Precision Agriculture in America is tractors with GPS units that vary their fertilizer and pesticide distribution on a meter-by-meter basis (robotic valves consult the dataset as you drive around the land.)

In a developing world context, your farmers get the GPS coordinates for their land tied to their cell phone numbers either by an aid worker, or by their own cell phone company.

Then the USG runs code over their sat data, and comes up with farming recommendations for that plot of land. If the plots are small enough (and they often are) the entire plot is a single precision agriculture cell.

But if you think about the size of the datasets - we're talking about doing this for maybe 20 - 30% of the planet's landmass - and the software to interpret the images is non-trivial and only going to get more complex as modeling of crops and farming practices improves...

Real applications - change the world applications - need parallel supercomputer programming. Occam was right in the same way that Lisp is right but for a different class of problems. That's because Occam is CSP (concurrent sequential processes) and those are a Good Thing. There may need to be refinements to handle the fact we have much faster nodes, but much slower networks, than Occam was originally designed for - but that may also turn out to be a non-issue.

I'm also working on similar stuff around expert systems for primary health care - medical expert systems are already pretty well understood - so the notion is to develop an integrated set of medical practices (these 24 drugs which don't require refrigeration, don't produce overdose easily, and are less than $10 per course) with an expert system which can be accessed both by patients themselves to figure out if their symptoms are problematic or not, and by slightly trained health care workers who would use the systems to figure out what to prescribe from their standard pharmacopoeia.

It's not much, but for the poorest two or three billion, this could be the only health care service they ever see. None of the problems are particularly intractable, but you better bet there's a VAST - and I mean VAST - distributed call center application at the core of this.

Of course, the Right Way to do this is FOLDING@HOME or SETI - we've already proven that public interest supercomputing on a heterogeneous distributed network works.

Now we just need to turn it to something directly lifesaving, rather than indirectly important for broader reasons.

Remember that the richest 50% of the human race have cell phones already, and rumor has it (i.e. I read it on the internet) that phone numbers and internet users in Africa have doubled every year for the past seven years. 10 years from now the network is going to be ubiquitous, even among many of the very, very poorest.

We get a do-over here in our relationship with the developing world. We can't fix farm subsidies, but we can ensure that when they plug into the network for the first time, there is something useful there.

Re: The future of programming languages in a massively concurrent world

#13
post #9
post #7

Earlier quoted context omitted.

I think 'race' is one way of looking at it. Another way to look at it is that there will be more than one 'winner'. Just like today, I think in the world of tomorrow people will use more than one programming language. So all of the languages that expect to be around tomorrow will need to evolve into languages for controlling concurrent systems on massively parallel architectures. You are correct in saying that some l…

Concurrency in todays web darlings, Ruby, PHP, and Python will be challenging. Heard of stackless python? [ http://www.stackless.com/ ]. Eve Online is the highest profile app using it I know of. Introduction to Concurrent Programming with Stackless Python [ http://members.verizon.net/olsongt/stackless/why_stackless.h... ]

Isn't stackless not truly concurrent since it's still beholden to the Python GIL?

Or is that not the case?

Re: The future of programming languages in a massively concurrent world

#14
post #4

The way things are headed, we'll soon be running most of our applications in the browser. Someone needs to come up with a multi-threaded JavaScript. Yes, I know, it's not going to be pretty.

Unless somebody comes up with an app that's doing massive amounts of data processing in JS, I think giving each tab/window its own thread in the browser will be an adequate solution for the foreseeable future.

Re: The future of programming languages in a massively concurrent world

#15
I really hope that if multiple cores increase, they are done in a way which hides them from the programmer. For example a 32 core CPU, but that appears as a very very fast single core. That's where the concurrency/'threading' issues should live, not in everyones code.

Threads are usually the problem IMHO not the solution.

I don't agree with the suggestion that javascript will need threads either. Javascript works extremely well in a single thread. There isn't really much of a need for threads. Having multiple cores doesn't change that, it just means you might need some abstraction layer like I described above, that utilizes all cores, whilst appearing as a single core.

Re: The future of programming languages in a massively concurrent world

#16
post #5

There is some chicken-and-egg effect: multicore processors are selling well if you have applications/languages and vice-versa. As far as I see, OpenMP has some potential as well as Erlang and others.

No, manufacturers have their own reasons to sell multi-cores, namely, that that's the only way they can sell them as being 'faster'. Intel is going to stop selling single-core processors not too long from now (eg).

Re: The future of programming languages in a massively concurrent world

#17
post #9

Earlier quoted context omitted.

Concurrency in todays web darlings, Ruby, PHP, and Python will be challenging. Heard of stackless python? [ http://www.stackless.com/ ]. Eve Online is the highest profile app using it I know of. Introduction to Concurrent Programming with Stackless Python [ http://members.verizon.net/olsongt/stackless/why_stackless.h... ]

Isn't stackless not truly concurrent since it's still beholden to the Python GIL? Or is that not the case?

Stackless is an example of the communicating sequential process model, not concurrency. You avoid all the issues around locking etc because switching between processes (tasklets) is explicit - it's cooperative multitasking not preemptive. It's a very nice language but doesn't help at all with running on multiple cores. Once you start having multiple processes running simultaneously and shared mutable state you have to start worrying about things like locks. Erlang nicely sidesteps the problem by not having mutable variables.

Re: The future of programming languages in a massively concurrent world

#18
post #7
post #3

I think it's a race: In one corner are languages like Erlang that have been designed for concurrency. In another corner are languages with massive user bases, that don't do concurrency very well (Java for example), that will have to undergo modifications to work better. In another corner, perhaps, are languages that are just now being created. They're the outsiders, but have more agility in their design because they…

I think 'race' is one way of looking at it. Another way to look at it is that there will be more than one 'winner'. Just like today, I think in the world of tomorrow people will use more than one programming language. So all of the languages that expect to be around tomorrow will need to evolve into languages for controlling concurrent systems on massively parallel architectures. You are correct in saying that some l…

I don't think erlang is that great for the scenarios that started this thread - it is designed for 1000's of processes, not dozens. And by all reports its straight line performance is much much much slower then anything else out there.

There is of course functional approaches like haskell (but it will take longer to be mainstream).

Re: The future of programming languages in a massively concurrent world

#19
post #7

Earlier quoted context omitted.

I think 'race' is one way of looking at it. Another way to look at it is that there will be more than one 'winner'. Just like today, I think in the world of tomorrow people will use more than one programming language. So all of the languages that expect to be around tomorrow will need to evolve into languages for controlling concurrent systems on massively parallel architectures. You are correct in saying that some l…

I don't think erlang is that great for the scenarios that started this thread - it is designed for 1000's of processes, not dozens. And by all reports its straight line performance is much much much slower then anything else out there. There is of course functional approaches like haskell (but it will take longer to be mainstream).

Erlang is functional, but in any case, you're right about its performance - it's more or less that of a fast interpreted language, rather than a compiled language.

Re: The future of programming languages in a massively concurrent world

#20
post #7

Earlier quoted context omitted.

I think 'race' is one way of looking at it. Another way to look at it is that there will be more than one 'winner'. Just like today, I think in the world of tomorrow people will use more than one programming language. So all of the languages that expect to be around tomorrow will need to evolve into languages for controlling concurrent systems on massively parallel architectures. You are correct in saying that some l…

Concurrency in web languages is a non-issue, because each request is independent (or should be, if you have a proper shared-nothing architecture). You simply run multiple processes and give each process a full core. Most FastCGI/SCGI webserver modules have functionality built-in to multiplex among backend processes. Concurrency in the database is more interesting, particularly since that's where the bottleneck is in…

Well, there is actually a latency gain to be had from parallellizing indidvidual requests, rather than just running several requests in parallel.

Think about this example: You have 10 printers each capable of printing 10 pages per minute. Then 10 jobs are submitted each with 10 pages. If you run those jobs in parallel, all of them will finish after 60 seconds. If you parallelize each job and print page 1 on printer 1, page 2 on printer 2 etc., then the first job will finish in 6 seconds, the second in 12, and the last one in 60 seconds. The average latency is then (6 s + 12 s + ... + 60 s) / 10 = 33 s.

Your throughput will be the same, except for a bit of parallelization overhead.

Post reply on HN