Programming languages worth checking out

Could someone provide more up-to-date Windows binaries of both the core and the addons?

No.

CUDA yes. Cg yes. HLSL yes. Verilog yes. VHDL yes. C++ or Java with MapReduce yes. PHP and MySQL with memcached yes. Erlang yes — and it really is functional, inside each process, anyway — that is, the level where you aren't getting any parallelism. Octave or R, potentially, but not today, as far as I know. Mathematica yes, and it, too, is mostly functional.

In theory, side effects are what make parallelism hard, and so languages whose semantics are side-effect-free (unlike F# or Mathematica or Erlang) should make it easy. So we all thought in 1980. Since then we spent 20 years or so trying to make that happen, and it basically didn't work.

There are basically four kinds of parallelism within easy reach today. There's SIMD, like MMX, SSE, 3DNow, AltiVec, and the like; you'd think that data-parallel languages and libraries like Numpy and Octave would be all over this, but except for Mathematica, that doesn't seem to be happening. There's running imperative code on a bunch of tiny independent processors that share no data; AFAIK that's what the shader languages are doing. There's instruction-level parallelism on a superscalar processor, which largely benefits from things like contiguous arrays in memory, or maybe what Sun is doing with Niagara, where the processor pretends to be a bunch of tiny independent slow processors. And then there's splitting up your data across a shared-nothing cluster, which is how every high-traffic web site works, and that's what MapReduce makes simpler.

Uh, and then there's designing your own hardware or programming FPGAs, which is what Verilog and VHDL are for.

Languages like OCaml (I don't know anything about F# except that it's like OCaml, but for the CLR) have no special advantage for any of these scenarios. They don't even have the theoretical advantage that they have no side effects and therefore you can speculatively multithread them without breaking the language semantics. They do have the massive practical disadvantage, in most of the scenarios I described, of needing unpredictable amounts of memory, having massive libraries, and using pointers all over the place. Using pointers all over the place kills your locality of reference and your ILP. Having massive libraries and using unpredictable amounts of memory makes it impossible to run them inside your GPU and means they can't run on an FPGA (except by using external memory, like the awesome Reduceron). And nothing about the language semantics helps with SIMD either.

So, sheesh, go read Alan Bawden's dissertation or whatever, but don't go around claiming that ML (or even Haskell) is going to magically make your algorithms parallel. We tried that. It didn't work. We're trying something else now.

It's not that it magically makes thing parallel. It's that it puts the tools to do it in the hands of Joe Developer who has hankered to do functional programming in the "real world" and now can say to his manager, look, it's part of Visual Studio now, it's official, there's no reason I shouldn't do this. I am confused as to why you say OCaml/F# don't even have theoretical applications here - it it because the .NET libraries aren't side-effect free? It's about practicality, not about technology. The former is what was missing, the latter's been around for 20 years or more.

And please, PHP and MySQL? For computation? Are you serious?

It's one of my favorite languages, and is rapidly crowding out Python. It's a tremendously clean and powerful language. Since Lua was designed for embedding, its design favored adding a few meta-features that could be used to build task-appropriate features into the language, rather than adding to the language core. It's also a relatively quick language to learn, and is extraordinarily portable - the whole language is written in pure ANSI C.

It also has tail-call optimization, closures, coroutines, and other aspects that are interesting from a pure CS standpoint, such as the way the table datatype is implemented and its register-based VM.

For a good taste of Lua, skim "The Implementation of Lua 5.0" (http://www.tecgraf.puc-rio.br/~lhf/ftp/doc/jucs05.pdf), the quick intro to Lua in the LPEG paper (http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html , around page 6 of the PDF), or _Programming in Lua_ (http://www.inf.puc-rio.br/~roberto/pil2/). The first edition of the latter is online, though the second edition has a bunch of added material, and IMHO is well-worth buying if you get into Lua. Also, check out LuaJIT (http://luajit.org/), if you're using i386 hardware -- it's a FAST JIT.

The main thing Joe Developer lacks to do parallel programming is massively parallel hardware, and if he's happy with just getting a 100× speedup with GPGPU hacking, maybe he doesn't even lack that. High-performance parallel programming has very little to do with functional programming languages, today and in the foreseeable future.

And please, PHP and MySQL? For computation? Are you serious?

What do you think people use PHP and MySQL for, fertilizing flowerpots? Computation is all they do! Obviously the PHP5 interpreter isn't the most computationally efficient medium in the world, but it's where it's at when it comes to horizontal web site scaling, i.e. parallelism.