Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A 32nm 1000-Processor Array

vcl.ece.ucdavis.edu

Post reply on HN
Print view View original

91–100 of 107 posts

Re: A 32nm 1000-Processor Array

#91

Post by gnufx »

jacquesm wrote: post #11
RachelF wrote: post #5

Earlier quoted context omitted.

It's very hard. In the mid 1980s there was a CPU called a "Transputer" [1] made some of the people who moved to ARM. These CPUs could be connected together in huge networks and directly talk to each other. The network of CPU's could auto-discover its topology, but coding for so many CPU's was difficult. Some specific algorithms scaled well with the number of CPUs, but most did not. [1] https://en.wikipedia.org/wiki/T…

Occam did it quite nicely. I think the reason transputers didn't 'make it' is not because they were super hard to program (it was only a little bit harder than programming a regular computer), but because the price premium you paid for a transputer set-up was too high and x86 got faster very rapidly. This is right around the time when the first 386 machines were launched and in a very short time we went from 12-20 MH…

Yes, but in the areas I saw at the time, transputers lost out in particular to i860 for floating point systems and to m68k for things like our mostly-integer "streaming map-reduce" -- it seems! -- system. (Being able to house 68ks in a blade system, i.e. VME/VXI crates, helped.)

Re: A 32nm 1000-Processor Array

#92

Post by valarauca1 »

kazinator wrote: post #88
valarauca1 wrote: post #84

Earlier quoted context omitted.

>Functional programming languages do not _require_ a GC. They just largely have it. No they just require Infinite Memory [1] XOR GC. Pure Functional programming has no concept of Alloc/Delloc. Let alone the concept of binding/assignment can fail. These are real. To quote James Michens [2] >Pointers are real. They’re what the hardware understands. Somebody has to deal with them. You can’t just place a LISP book on top…

Pure functional programming doesn't require any special memory management beyond the stack, if it avoids any data representations that use reference semantics and have indefinite lifetimes. Lazy evaluation and higher order functions with environment are pretty much out. But even C can be functional: int (*pg)(int) = f(); int x = h() + 2*z; int w = pg(y); /* ... etc */ Here, we just introduce new variables instead of…

>Pure functional programming doesn't require any special memory management beyond the stack,

Yes. Just an infinite amount of stack. Why? See my previous post.

If we pretend for a minute we live in the real world... Oh guess what your stack can still over flow and assignment/binding can still fail.

Also in a pure stack based language you can't preform multithreading. If you do... Well now your building a whole OoO super-scalar functional VM on top of a physical processor just to avoid doing GC.

Re: A 32nm 1000-Processor Array

#93

Post by dnautics »

gnufx wrote: post #90
trsohmers wrote: post #82

Earlier quoted context omitted.

Pretty sure you mean us (REX Computing; http://rexcomputing.com ), though I would not consider our network on chip "weird" :P Our big difference that you were trying to remember was our use of scratchpad memory which is simply stated as this: We can radically reduce power consumption, increase density, and increase speed of on chip memory (SRAM) by removing the traditional hardware caching system. We instead use a pu…

Perhaps you've noticed the architecture of the new top top500 system. There's currently an HPC guessing game on filling in blanks in " rel="nofollow">http://www.netlib.org/utk/people/JackDongarra/PAPERS/sunway-... but of course CPU comparisons stop at Knights Landing. One of the things I wondered about is how Linux is adapted to such an architecture. I couldn't find anything from REX on operating system support.

I don't think you're not supposed to run linux on the current rex architecture. I think you're supposed to (pre/) compile a program for it, issue it to the chips, pass the data on over, and then sic the programs on the data.

Re: A 32nm 1000-Processor Array

#94

Post by trsohmers »

gnufx wrote: post #90
trsohmers wrote: post #82

Earlier quoted context omitted.

Pretty sure you mean us (REX Computing; http://rexcomputing.com ), though I would not consider our network on chip "weird" :P Our big difference that you were trying to remember was our use of scratchpad memory which is simply stated as this: We can radically reduce power consumption, increase density, and increase speed of on chip memory (SRAM) by removing the traditional hardware caching system. We instead use a pu…

Perhaps you've noticed the architecture of the new top top500 system. There's currently an HPC guessing game on filling in blanks in " rel="nofollow">http://www.netlib.org/utk/people/JackDongarra/PAPERS/sunway-... but of course CPU comparisons stop at Knights Landing. One of the things I wondered about is how Linux is adapted to such an architecture. I couldn't find anything from REX on operating system support.

Yep, have seen it... it seems to not be exactly what was described by the Chinese at ISC or SC last year (their original description being much more in line with a DSP), though there is not much data available. All of the articles I am seeing today are calling it DEC Alpha "like", though the only real source for this seems to be the wikipedia article for the family of processors, with the latest version being in 2013. Dongarra even specifically says in the paper you linked that it was NOT related to the Alpha ISA, so it seems like all of the media articles are incorrect ;)

As for REX, as dnautics said, we've been mostly focused on running raw compute kernels on the current simulated versions (software and FPGA), and for the soon to be in hand silicon (coming this fall)... one of our projects internally is to port the L4 microkernel, and a telecom focused RTOS, but that is as far as our operating systems plans go for the near future. I'd also love to get a Plan 9/inferno demo running on it for fun, but we've got more important work to do at the moment.

Re: A 32nm 1000-Processor Array

#95

Post by sedachv »

rwmj wrote: post #6
willvarfar wrote: post #4

So how do you program such a beast? What progress is being made on that front? Cache coherency seems really hard to give up on, and even CPU-GPU cache coherency is becoming the expected norm, with even ARM delivering it.

This is "the" question. I suppose one could look at the Connection Machine: https://en.wikipedia.org/wiki/Connection_Machine The CM-2a had 4K or 8K processors and was reputedly very difficult to program. Looked great though: http://www.corestore.org/cm2a.htm

The Connection Machine was a SIMD design and the languages available for it (StarLisp: https://omohundro.files.wordpress.com/2009/03/omohundro86_th... and C*: http://people.csail.mit.edu/bradley/cm5docs/CStarProgramming...) were actually pretty good compared to OpenCL.

The Connection Machine Lisp programming language described in Daniel Hillis' PhD dissertation was essentially going to be Lisp with parallel map/reduce but AFAIK was never done being implemented.

I think the big problem was that most of the SIMD algorithms were yet to be discovered at the time. For example this paper by Hillis and Steele was a very big deal but looks kind of basic today: http://uenics.evansville.edu/~mr56/ece757/DataParallelAlgori...

Guy Blelloch did a lot of work on Connection Machines and basically wrote the book on SIMD programming: http://www.cs.cmu.edu/~blelloch/papers/Ble90.pdf He also made a very nice programming language for parallel computing (NESL), as did Gary Sabot who worked at Connection Machines (Paralations). When you compare those to Hadoop or OpenCL it really is a wonder where we went wrong and what the designers of the latter were thinking (or not).

Re: A 32nm 1000-Processor Array

#96

Post by mrb »

duckingtest wrote: post #24
mrb wrote: post #18

It is surprising the paper makes no comparisons with GPUs. So I will do it. For starters it looks like they are talking about integer operations (I only skimmed the paper and it mentions an ALU, not an FPU), whereas my GPU numbers below are single precision floating point numbers. So it is apples vs oranges. So, a modern 14-16nm GPU like the Tesla P100 or RX 480 does about 5 to 10 trillion ops/sec at 200-300 W, and a…

(Dynamic) Power consumption goes up with square of voltage, so a comparison with a gpu's max power efficiency point (almost certainly underclocked and undervolted) could change the comparison significantly.

You are very right. I should have mentioned it in my comment. Back when I was mining bitcoins on AMD GPUs, undervolting and underclocking would easily double or triple the power efficiency.

Re: A 32nm 1000-Processor Array

#97

Post by nickpsecurity »

valarauca1 wrote: post #84
zbobet2012 wrote: post #66

Earlier quoted context omitted.

Garbage collection is an idea. It can be slower, or faster, than other memory management techniques depending on implementation and specific usage. Functional programming languages do not _require_ a GC. They just largely have it. "Fibers" like you linked in the presentation (m:n green thread scheduling) have been in use for decades. Many, many languages other than C++ have had them for over a decade. Go is built on…

>Functional programming languages do not _require_ a GC. They just largely have it. No they just require Infinite Memory [1] XOR GC. Pure Functional programming has no concept of Alloc/Delloc. Let alone the concept of binding/assignment can fail. These are real. To quote James Michens [2] >Pointers are real. They’re what the hardware understands. Somebody has to deal with them. You can’t just place a LISP book on top…

PreScheme was a LISP to replace C for low-level programming. Used manual, memory management instead of GC. Very fast and efficient.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.3.4...

Formally verified plus a Scheme48 interpreter as part of VLISP project.

https://en.wikipedia.org/wiki/PreScheme

ATS has no garbage collector that I'm aware of. I've seen it used in device drivers and 8-bit MCU's.

https://en.wikipedia.org/wiki/ATS_(programming_language)

LinearML is GC-less, functional, parallel programming.

https://github.com/pikatchu/LinearML

So, functional languages neither require GC's nor infinite memory. Two also combined low-level efficiency with easy, formal verification vs C programs. So, that's higher mapping of idea to code, high efficiency, and better safety all at once.

"You can’t just place a LISP book on top of an x86 chip"

I believe I just did with PreScheme. Microsoft goes further to straight-up use a theorem prover to do x86 coding.

http://research.microsoft.com/en-us/um/people/nick/coqasm.pd...

Re: A 32nm 1000-Processor Array

#98

Post by nickpsecurity »

imtringued wrote: post #71
raphaelj wrote: post #63

Earlier quoted context omitted.

Garbage collection is not slow. Actually it's probably the fastest dynamic allocation method. Allocating data with a copying garbage collector is O(1), which is as fast as allocating something on the stack, while malloc() is usually O(log(n)) with n being the number of live objects in the heap. Running a collection on a such GC is usually O(n), with n begin the number of live objects. This is way faster than calling…

The only problem with GCs is the stop the world pause. I'm seriously wondering why not more languages have a gc per "process" like erlang.

The only problem with "stop-the-world" GC's on hardware not built for them. There's "pauseless" collectors out there where that either doesn't happen or happens so fast you don't experience it. Some do microseconds. One was in a Scheme machine where they put it into the memory subsystem. So, the program just allocated, deleted, whatever with a parallel, hardware GC managing pages in the background.

Many things one can do in GC's. The only one I know with mainstream success is Azul's:

http://www.azulsystems.com/sites/default/files/images/c4_pap...

https://www.azul.com/products/zing/

Note: No affiliation with them. Their Azul systems and pauseless GC were simply the best stuff I found researching Java & GC hardware. Assuming they match marketing claims. ;)

Re: A 32nm 1000-Processor Array

#99

Post by kazinator »

valarauca1 wrote: post #92
kazinator wrote: post #88

Earlier quoted context omitted.

Pure functional programming doesn't require any special memory management beyond the stack, if it avoids any data representations that use reference semantics and have indefinite lifetimes. Lazy evaluation and higher order functions with environment are pretty much out. But even C can be functional: int (*pg)(int) = f(); int x = h() + 2*z; int w = pg(y); /* ... etc */ Here, we just introduce new variables instead of…

>Pure functional programming doesn't require any special memory management beyond the stack, Yes. Just an infinite amount of stack. Why? See my previous post. If we pretend for a minute we live in the real world... Oh guess what your stack can still over flow and assignment/binding can still fail. Also in a pure stack based language you can't preform multithreading. If you do... Well now your building a whole OoO sup…

Infinite stack. Why? Oh, because there is no iteration so recursion has to be used? Iteration rewrites to tail calls though; they don't require stack. As far as real tail-calls go, you avoid the algorithms that blow stacks: stick to strict divide and conquer, to keep the stack depth logarithmic.

About multithreading: this language can describes a computation done by each of up to 1000 nodes. Those implicitly run in parallel.

Re: A 32nm 1000-Processor Array

#100

Post by pjssjppjs »

tremon wrote: post #51
geon wrote: post #46

Earlier quoted context omitted.

> I bristle anytime anyone thinks any existing functional language is better The parent comment doesn't say that, though. I imagine current functional languages are about as ill suited as current imperative languages for running on a 1000 core machine. Something new might be needed.

I think that the concept of purity, present mostly in functional languages, helps a lot for writing parallellizable (?) code. My naive assumption would be that pure code could be adapted for a 1000-core machine with only changes to the compiler and runtime environment. That said, few things in CS are written with this kind of parallellism in mind: even most algorithms' pseudocode is written in imperative style, and a…

> We definitely need something new.

Well, if only there would have been some ideas around…

like TTA[0] or Dataflow architecture[1]

These are just things that come to mind when thinking about an architecture like this. You'll probably still need something new, but this, instead of some bottle neck computing would be my starting point.

Admittedly I'm still waiting for processors like thisat consumer prices, but those ideas seem to have been forgotten at a time when I was still figuring out how to tie shoelaces and such.

[0]https://en.wikipedia.org/wiki/Transport_triggered_architectu... [1]https://en.wikipedia.org/wiki/Dataflow_architecture

Post reply on HN