A 32nm 1000-Processor Array • Hacker News Board

Re: A 32nm 1000-Processor Array

#81

Post by stcredzero » Mon, Jun 20, 2016, 5:00 PM UTC

jacquesm wrote: post #11

RachelF wrote: post #5

Earlier quoted context omitted.

It's very hard. In the mid 1980s there was a CPU called a "Transputer" [1] made some of the people who moved to ARM. These CPUs could be connected together in huge networks and directly talk to each other. The network of CPU's could auto-discover its topology, but coding for so many CPU's was difficult. Some specific algorithms scaled well with the number of CPUs, but most did not. [1] https://en.wikipedia.org/wiki/T…

Occam did it quite nicely. I think the reason transputers didn't 'make it' is not because they were super hard to program (it was only a little bit harder than programming a regular computer), but because the price premium you paid for a transputer set-up was too high and x86 got faster very rapidly. This is right around the time when the first 386 machines were launched and in a very short time we went from 12-20 MH…

I think the reason transputers didn't 'make it' is not because they were super hard to program

You could replace "transputers" with [insert-architecture] and wind up with a nice overarching theory to explain the history of computer hardware.

Re: A 32nm 1000-Processor Array

#82

Post by trsohmers » Mon, Jun 20, 2016, 5:12 PM UTC

foota wrote: post #7

Does anyone have a link or the name of another weird architecture that was posted a while ago? (~3 months maybe?) I remember that there were a large number of cores that all communicated with each other in some weird way and that they like didn't have main system memory or something like that...

Pretty sure you mean us (REX Computing; http://rexcomputing.com), though I would not consider our network on chip "weird" :P

Our big difference that you were trying to remember was our use of scratchpad memory which is simply stated as this: We can radically reduce power consumption, increase density, and increase speed of on chip memory (SRAM) by removing the traditional hardware caching system. We instead use a purely software managed memory system, through some very fancy (or do I dare say "smart") compiler techniques that are enabled by a a very simplified architecture and the ability to guarantee latency for all memory operations. We can still have main system memory (meaning DRAM), it is just instead of having a bunch of complex hardware that burns a lot of power and wastes a lot of space in order to automatically fetch pages out of DRAM, we structure the code for each core to efficiently pull only the data necessary when it is needed.

Re: A 32nm 1000-Processor Array

#83

Post by theparanoid » Mon, Jun 20, 2016, 5:45 PM UTC

stcredzero wrote: post #78

duckingtest wrote: post #24

Earlier quoted context omitted.

(Dynamic) Power consumption goes up with square of voltage, so a comparison with a gpu's max power efficiency point (almost certainly underclocked and undervolted) could change the comparison significantly.

Power consumption goes up with square of voltage This needs to become common knowledge amongst nerds and tech types.

It was more widely known, when every geek overclocked; no more.

Re: A 32nm 1000-Processor Array

#84

Post by valarauca1 » Mon, Jun 20, 2016, 6:05 PM UTC

zbobet2012 wrote: post #66

endergen wrote: post #43

Earlier quoted context omitted.

I'm an ex game engine developer and I bristle anytime anyone thinks any existing functional language is better for multicore. Specifically garbage collection alone will make any language an order of magnitude slower generally per a single core. Also the C/C++ game development community at least has great approaches to multicore which makes C/C++ linearly scale with scores to boot, see for example: http://www.gdcvault…

Garbage collection is an idea. It can be slower, or faster, than other memory management techniques depending on implementation and specific usage. Functional programming languages do not _require_ a GC. They just largely have it. "Fibers" like you linked in the presentation (m:n green thread scheduling) have been in use for decades. Many, many languages other than C++ have had them for over a decade. Go is built on…

>Functional programming languages do not _require_ a GC. They just largely have it.

No they just require Infinite Memory [1] XOR GC.

Pure Functional programming has no concept of Alloc/Delloc. Let alone the concept of binding/assignment can fail. These are real. To quote James Michens [2]

>Pointers are real. They’re what the hardware understands. Somebody has to deal with them. You can’t just place a LISP book on top of an x86 chip and hope that the hardware learns about lambda calculus by osmosis. Denying the existence of pointers is like living in ancient Greece and denying the existence of Krackens and then being confused about why none of your ships ever make it to Morocco, or Ur-Morocco, or whatever Morocco was called back then. Pointers are like Krackens—real, living things that must be dealt with so that polite society can exist.

[1] Infinite memory simply means more memory then the program can ever consume... But the halting problem exists so you can't actually know how much memory your program will consume :P

[2] http://scholar.harvard.edu/files/mickens/files/thenightwatch...

Re: A 32nm 1000-Processor Array

#85

Post by monocasa » Mon, Jun 20, 2016, 6:13 PM UTC

ryao wrote: post #54

jacquesm wrote: post #38

Earlier quoted context omitted.

Another big difference is that most GPU architectures are multi-lane SIMD (so single instructions acting on multiple data but multiple sets of those) whereas the linked architure is MIMD. In simpler terms: these processors all execute independent code whereas a GPU tends to have multiple cores but a (sometimes much) smaller number of threads of execution.

Nvidia switched to MIMD in their Fermi architecture. I assume that they are the reason why jacquesm says most.

Eh, not really. 32 'threads' share a program counter on Fermi; it's still actually pretty heavily SIMD.

Re: A 32nm 1000-Processor Array

#86

Post by stcredzero » Mon, Jun 20, 2016, 6:39 PM UTC

theparanoid wrote: post #83

stcredzero wrote: post #78

Earlier quoted context omitted.

Power consumption goes up with square of voltage This needs to become common knowledge amongst nerds and tech types.

It was more widely known, when every geek overclocked; no more.

I was underclocking when everyone else was overclocking. Because I wanted quiet!

Re: A 32nm 1000-Processor Array

#87

Post by kazinator » Mon, Jun 20, 2016, 7:06 PM UTC

endergen wrote: post #43

JoeAltmaier wrote: post #41

Thinking out loud. Instead of imagining applications per processor, I imagine this device could map threads or message handlers to processors. It could work better with a functional language or at least some language that didn't explicitly manage parallelism in code but rather in the runtime. Offload the app writer to just coding algorithm and not thread synchronization. E.g. imagine each timer wait being a processor…

I'm an ex game engine developer and I bristle anytime anyone thinks any existing functional language is better for multicore. Specifically garbage collection alone will make any language an order of magnitude slower generally per a single core. Also the C/C++ game development community at least has great approaches to multicore which makes C/C++ linearly scale with scores to boot, see for example: http://www.gdcvault…

> I bristle anytime anyone thinks any existing functional language is better for multicore.

.. and have they read the paper for this multicore? Though it has a large number of processors, there are severe resource constraints per node, with respect to how large a local program can be and how much memory is available.

Re: A 32nm 1000-Processor Array

#88

Post by kazinator » Mon, Jun 20, 2016, 7:18 PM UTC

valarauca1 wrote: post #84

zbobet2012 wrote: post #66

Earlier quoted context omitted.

Garbage collection is an idea. It can be slower, or faster, than other memory management techniques depending on implementation and specific usage. Functional programming languages do not _require_ a GC. They just largely have it. "Fibers" like you linked in the presentation (m:n green thread scheduling) have been in use for decades. Many, many languages other than C++ have had them for over a decade. Go is built on…

>Functional programming languages do not _require_ a GC. They just largely have it. No they just require Infinite Memory [1] XOR GC. Pure Functional programming has no concept of Alloc/Delloc. Let alone the concept of binding/assignment can fail. These are real. To quote James Michens [2] >Pointers are real. They’re what the hardware understands. Somebody has to deal with them. You can’t just place a LISP book on top…

Pure functional programming doesn't require any special memory management beyond the stack, if it avoids any data representations that use reference semantics and have indefinite lifetimes. Lazy evaluation and higher order functions with environment are pretty much out. But even C can be functional:

   int (*pg)(int) = f();
   int x = h() + 2*z;
   int w = pg(y);
   /* ... etc */

Here, we just introduce new variables instead of assigning new ones, don't malloc anything and indirect only by means of dumb function pointers carrying no environments.

There could be some higher level (though nonetheless quite primitive) assignment-free language for specifying tasks for the 1000 cores of this chip, instead of programming them in assembler.

Re: A 32nm 1000-Processor Array

#89

Post by gnufx » Mon, Jun 20, 2016, 7:39 PM UTC

willvarfar wrote: post #4

So how do you program such a beast? What progress is being made on that front? Cache coherency seems really hard to give up on, and even CPU-GPU cache coherency is becoming the expected norm, with even ARM delivering it.

High performance computing relies on message passing -- if not pure and simple -- normally with the MPI standard. There are applications which scale at least past petaflops and 1000 cores is unexceptional.

The developers of the PETSc system wrote a position paper eschewing the typical hybrid (MPI+threading) techniques for trundling towards "exascale" -- if with MPI's shared memory support, which doesn't require cache coherence. (Apart from remote memory access, MPI departs from actor-/CSP-like systems in supporting things like collective communication (and i/o), and dynamic process management.)

Re: A 32nm 1000-Processor Array

#90

Post by gnufx » Mon, Jun 20, 2016, 7:52 PM UTC

trsohmers wrote: post #82

foota wrote: post #7

Does anyone have a link or the name of another weird architecture that was posted a while ago? (~3 months maybe?) I remember that there were a large number of cores that all communicated with each other in some weird way and that they like didn't have main system memory or something like that...

Pretty sure you mean us (REX Computing; http://rexcomputing.com ), though I would not consider our network on chip "weird" :P Our big difference that you were trying to remember was our use of scratchpad memory which is simply stated as this: We can radically reduce power consumption, increase density, and increase speed of on chip memory (SRAM) by removing the traditional hardware caching system. We instead use a pu…

Perhaps you've noticed the architecture of the new top top500 system. There's currently an HPC guessing game on filling in blanks in " rel="nofollow">http://www.netlib.org/utk/people/JackDongarra/PAPERS/sunway-... but of course CPU comparisons stop at Knights Landing.

One of the things I wondered about is how Linux is adapted to such an architecture. I couldn't find anything from REX on operating system support.