Live data from Hacker News

A 32nm 1000-Processor Array

vcl.ece.ucdavis.edu

41–50 of 107 posts

Re: A 32nm 1000-Processor Array

#41
Thinking out loud.

Instead of imagining applications per processor, I imagine this device could map threads or message handlers to processors. It could work better with a functional language or at least some language that didn't explicitly manage parallelism in code but rather in the runtime. Offload the app writer to just coding algorithm and not thread synchronization.

E.g. imagine each timer wait being a processor spinning; each I/o loop being a processor that blocked/looped on an I/o pin state. With so many processors to schedule, it wouldn't stall application progress to spin or block an individual strand (until you ran out or processors). To make this efficient, they'd want interrupts and semaphore state to be hardware-supported. Instead of polling a memory location, block on a shared register masked where each bit was a condition. So instead of a 'kernel call' it'd be an opcode or small loop. Latency of wakeup then becomes about 1 machine cycle.

I imagine with the right runtime support this could be a useful device for a large I/o server. It could reduce latency of processing each client message to just the execution time. No time burned in kernel calls, process switching, stack copying, interrupt/event latency.

Re: A 32nm 1000-Processor Array

#42
post #18

It is surprising the paper makes no comparisons with GPUs. So I will do it. For starters it looks like they are talking about integer operations (I only skimmed the paper and it mentions an ALU, not an FPU), whereas my GPU numbers below are single precision floating point numbers. So it is apples vs oranges. So, a modern 14-16nm GPU like the Tesla P100 or RX 480 does about 5 to 10 trillion ops/sec at 200-300 W, and a…

Also important is the memory bandwidth. You can do all the TFLOPs you want, but if you can't access the data fast enough, then there's a bottleneck.

Re: A 32nm 1000-Processor Array

#43

Thinking out loud. Instead of imagining applications per processor, I imagine this device could map threads or message handlers to processors. It could work better with a functional language or at least some language that didn't explicitly manage parallelism in code but rather in the runtime. Offload the app writer to just coding algorithm and not thread synchronization. E.g. imagine each timer wait being a processor…

I'm an ex game engine developer and I bristle anytime anyone thinks any existing functional language is better for multicore. Specifically garbage collection alone will make any language an order of magnitude slower generally per a single core. Also the C/C++ game development community at least has great approaches to multicore which makes C/C++ linearly scale with scores to boot, see for example: http://www.gdcvault.com/play/1022186/Parallelizing-the-Naugh....

I love functional languages, more for thinking in them, prototyping ideas, especially compilers/visualizers, and etc. But for any language that adds garbage collection, immutable data structures (way more operations per write and crazy memory thrashing/alignment issues), unless used sparsely or in a mixed paradigm (ugh, except maybe scala/clojure) are going to pay a magnitude of performance loss.

Mind you there are tricks around using more system languages (C/C++/Rust/D etc) for a lot of the heavy lifting with the application core being functional that gets you closer to the best of both worlds.

Re: A 32nm 1000-Processor Array

#44
AIseek[0] had 10,000 processor chip in 2006, with simple connectivity and simple ops that were perfectly matched for graph algorithms. Their demo[1] would not be impressive today, but doing it in real time in 2006 was quite a feat and basically impossible without specialized hardware. Unfortunately, they're no longer in business.

[0] http://www.extremetech.com/extreme/75766-new-ai-chip-would-m... [1] https://www.youtube.com/watch?v=VeSQI2hinp0

Re: A 32nm 1000-Processor Array

#45

Thinking out loud. Instead of imagining applications per processor, I imagine this device could map threads or message handlers to processors. It could work better with a functional language or at least some language that didn't explicitly manage parallelism in code but rather in the runtime. Offload the app writer to just coding algorithm and not thread synchronization. E.g. imagine each timer wait being a processor…

> So instead of a 'kernel call' it'd be an opcode or small loop.

Like a Wait For Interrupt instruction? If all idle processors sleep, I imagine it could save some power as well.

Re: A 32nm 1000-Processor Array

#46

Thinking out loud. Instead of imagining applications per processor, I imagine this device could map threads or message handlers to processors. It could work better with a functional language or at least some language that didn't explicitly manage parallelism in code but rather in the runtime. Offload the app writer to just coding algorithm and not thread synchronization. E.g. imagine each timer wait being a processor…

I'm an ex game engine developer and I bristle anytime anyone thinks any existing functional language is better for multicore. Specifically garbage collection alone will make any language an order of magnitude slower generally per a single core. Also the C/C++ game development community at least has great approaches to multicore which makes C/C++ linearly scale with scores to boot, see for example: http://www.gdcvault…

> I bristle anytime anyone thinks any existing functional language is better

The parent comment doesn't say that, though. I imagine current functional languages are about as ill suited as current imperative languages for running on a 1000 core machine.

Something new might be needed.

Re: A 32nm 1000-Processor Array

#47
post #5

Earlier quoted context omitted.

It's very hard. In the mid 1980s there was a CPU called a "Transputer" [1] made some of the people who moved to ARM. These CPUs could be connected together in huge networks and directly talk to each other. The network of CPU's could auto-discover its topology, but coding for so many CPU's was difficult. Some specific algorithms scaled well with the number of CPUs, but most did not. [1] https://en.wikipedia.org/wiki/T…

Occam did it quite nicely. I think the reason transputers didn't 'make it' is not because they were super hard to program (it was only a little bit harder than programming a regular computer), but because the price premium you paid for a transputer set-up was too high and x86 got faster very rapidly. This is right around the time when the first 386 machines were launched and in a very short time we went from 12-20 MH…

>> x86 got faster very rapidly.

Is ARM ( or something else) going to surpass x86 (amd64 whatever you want to call it) in the near future?

Re: A 32nm 1000-Processor Array

#48
post #39
post #36

Earlier quoted context omitted.

Architecturally it's a perfect match for such CPUs. If you listen to Joe talks he was talking about designing for 1000 core cpus more then 10 years ago. It is his stated opinion that Erlang is perfect match for such situations.

Of course it's (much) better than anything else... though my experience is that it's quite hard to scale your application that much... You always need some shared resource and it gets ugly. And I also suspect Erlang has some implementation quirks (for example the algorithm of deciding which scheduler gets which process) that will prevent it from scaling that much. Yeah, I do believe that some day we'll reach such sca…

Each scheduler has it's own run queue there is higher level migration logic to balance run queues based on statistics collected.

Re: A 32nm 1000-Processor Array

#49

I'm certainly hopeful this will lead somewhere useful. Intel tried with Larrabee but apparently couldn't get the perf they wanted, especially with comparisons to GPUs It would be great to know what the trade-offs are with this architecture

you mean xeon phi. I think it's still around. It is plenty fast, but not much better tdp than a gpu (which are commodity (read this as cheap)).

Re: A 32nm 1000-Processor Array

#50
post #7

Does anyone have a link or the name of another weird architecture that was posted a while ago? (~3 months maybe?) I remember that there were a large number of cores that all communicated with each other in some weird way and that they like didn't have main system memory or something like that...

Maybe a Systolic Array thing? https://en.wikipedia.org/wiki/Systolic_array

Maybe a Massively Parallel Processor Array thing? https://en.wikipedia.org/wiki/Massively_parallel_processor_a...

Post reply on HN