So how do you program such a beast? What progress is being made on that front? Cache coherency seems really hard to give up on, and even CPU-GPU cache coherency is becoming the expected norm, with even ARM delivering it.
It's very hard. In the mid 1980s there was a CPU called a "Transputer" [1] made some of the people who moved to ARM. These CPUs could be connected together in huge networks and directly talk to each other. The network of CPU's could auto-discover its topology, but coding for so many CPU's was difficult. Some specific algorithms scaled well with the number of CPUs, but most did not. [1] https://en.wikipedia.org/wiki/T…
I think the reason transputers didn't 'make it' is not because they were super hard to program (it was only a little bit harder than programming a regular computer), but because the price premium you paid for a transputer set-up was too high and x86 got faster very rapidly.
This is right around the time when the first 386 machines were launched and in a very short time we went from 12-20 MHz 286 boxes (and some 68K machines for the lucky ones) to 33 MHz 386 machines with a ton of RAM.
So the advantage that transputers had was eroded very quickly and I don't think INMOS was ready to match pace.
Now that we've reached the end of the line for that kind of speed increase we are seeing a renewed interest in multi-cpu fabric architectures, of which the transputer was an instance.