Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I want a good parallel computer

raphlinus.github.io

Post reply on HN
Print view View original

121–130 of 209 posts

Re: I want a good parallel computer

#121

Post by numpad0 »

rjsw wrote: post #114
turtletontine wrote: post #35

Earlier quoted context omitted.

Your wish sounds to me a lot like Larrabee/Xeon Phi or manycore CPUs. Maybe I am misunderstanding something, but it sounds like a good idea to me and I don’t totally see why it inherently can’t compete with GPUs.

I think Intel should have made more of an effort to get cheap Larrabee boards to developers, they could have been ones with chips that had some broken cores or unable to make the design speed.

RAM size seemed to have been a problem, lowest end Phi only had 6GB GDDR5 for its 57 cores(228 threads).

Re: I want a good parallel computer

#122

Post by touisteur »

newpavlov wrote: post #50
grg0 wrote: post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

>What I would like is a CPU with a highly parallel array of "worker cores" all addressing the same memory and speaking the same goddamn language that the CPU does. For "embarrassingly parallel" jobs vector extensions start to eat tiny bits of the GPU pie. Unfortunately, just slapping thousands of cores works poorly in practice. You quickly get into the synchronization wall caused by unified memory. GPUs cleverly work…

Seems to me there's a trend of applying explicit distributed systems (network of small-SRAM-ed cores each with some SIMD, explicit high-bandwidth message-passing between them, maybe some specialized ASICs such as tensor cores, FFT blocks...) looking at tenstorrent, cerebras, even kalray... out of the CUDA/GPU world, accelerators seem to be converging a bit. We're going to need a whole lot of tooling, hopefully relatively 'meta'.

Re: I want a good parallel computer

#123

Post by creata »

vgatherps wrote: post #76
Grosvenor wrote: post #30

Earlier quoted context omitted.

What I want is a Linear Algebra interface - As Gilbert Strang taught it. I'll "program" in LinAlg, and a JIT can compile it to whatever wonky way your HW requires. I'm not willing to even know about the HW at all, the higher level my code the more opportunities for the JIT to optimize my code. What I really want is something like Mathematica that can JIT to GPU. As another commenter mentioned all the API's assume you…

https://github.com/jax-ml/jax

To expand on this link, this is probably the closest you're going to get to 'I'll "program" in LinAlg, and a JIT can compile it to whatever wonky way your HW requires.' right now. JAX implements a good portion of the Numpy interface - which is the most common interface for linear algebra-heavy code in Python - so you can often just write Numpy code, but with `jax.numpy` instead of `numpy`, then wrap it in a `jax.jit` to have it run on the GPU.

Re: I want a good parallel computer

#124

Post by znpy »

ryukoposting wrote: post #67
deviantbit wrote: post #49

"I believe there are two main things holding it back." He really science’d the heck out of that one. I’m getting tired of seeing opinions dressed up as insight—especially when they’re this detached from how real systems actually work. I worked on the Cell processor and I can tell you it was a nightmare. It demanded an unrealistic amount of micromanagement and gave developers rope to hang themselves with. There’s a re…

One of the most important steps of my career was being forced to write code for an 8051 microcontroller. Then writing firmware for an ARM microcontroller to make it pretend it was that same 8051 microcontroller. I was made to witness the horrors of archaic computer architecture in such depth that I could reproduce them on totally unrelated hardware.

I had a similar experience. Our professor in high school would have us program a z80 system entirely by hand: flow chart, assembly code, computing jump offsets by hand, writing the hex code by hand (looking up op-codes from the z80 data sheet) and the loading the opcodes one byte at the time on a hex keypads.

It took three hours and your of us to code an integer division start to finish (we were like 17 though).

The amount of understanding it gave has been unrivalled so far.

Re: I want a good parallel computer

#126

Post by pca006132 »

deviantbit wrote: post #84
0xbadcafebee wrote: post #82

Earlier quoted context omitted.

> There's a whole generation of engineers that don't seem to realize why we architected things this way in the first place. Nobody teaches it, and nobody writes books about it (not that anyone reads anymore)

So, there are books out there. I use Computer Architecture: A Quantitative Approach by Hennessy and Patterson. Recent revisions have removed historical information. I understand why they did remove it. I wanted to use Stallings book, but the department had already made arrangements with the publisher. The biggest problem on why we don't write books is that people don't buy them. They take the PDF and stick it on gith…

While financial incentives is important to some, a lot of people write books to share their knowledge and give the book out for free. I think more people are doing this now, and there are also open collaborative textbook projects.

And I personally think that it is weird to write books during your working hour, and also get monet from selling that book.

Re: I want a good parallel computer

#127

Post by raphlinus »

adrian_b wrote: post #110
Retr0id wrote: post #13

Earlier quoted context omitted.

Isn't Xeon Phi just an instance of Larrabee?

It is an instance of Larrabee in the same sense as AMD Zen 4 is an instance of Larrabee. The "Larrabee New Instructions" is an instruction set that has been designed before AVX and also its first hardware implementation has been introduced before AVX, in 2010 (AVX was launched in 2011, with Sandy Bridge). Unfortunately while the hardware design of Sandy Bridge with the inferior AVX ISA has been done by the Intel A te…

> It is an instance of Larrabee in the same sense as AMD Zen 4 is an instance of Larrabee.

This is an odd claim. Clearly Xeon Phi is the shipping version of Larrabee, while Zen 4 is a completely different chip design that happens to run AVX-512. The first shipping Xeon Phi (Knights Corner) used the exact same P54C cores as Larrabee, while as you point out later versions of Xeon Phi switched to Atom.

It is extremely common to refer to all these as Larrabee, for example the Ian Cutress article on the last Xeon Phi chip was entitled "The Larrabee Chapter Closes: Intel's Final Xeon Phi Processors Now in EOL" [1]. Pat Gelsinger's recent interview at GTC [2] also refers to Larrabee. The section from around 44:00 has a discussion of workloads becoming more dynamic, and at 53:36 there's a section on Larrabee proper.

[1]: https://www.anandtech.com/show/14305/intel-xeon-phi-knights-...

[2]: https://www.youtube.com/live/pgLdJq9FRBQ

Re: I want a good parallel computer

#128

Post by znpy »

newpavlov wrote: post #50
grg0 wrote: post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

>What I would like is a CPU with a highly parallel array of "worker cores" all addressing the same memory and speaking the same goddamn language that the CPU does. For "embarrassingly parallel" jobs vector extensions start to eat tiny bits of the GPU pie. Unfortunately, just slapping thousands of cores works poorly in practice. You quickly get into the synchronization wall caused by unified memory. GPUs cleverly work…

It’s weird that no one mentioned xeon phi cards… that’s essentially what they were. Up to 188 (iirc?) x86 atom cores, fully generically programmable.

Re: I want a good parallel computer

#129

Post by thuanao »

grg0 wrote: post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

So greenarrays F18? :)

Re: I want a good parallel computer

#130

Post by actionfromafar »

tliltocatl wrote: post #72
grg0 wrote: post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

Larrabee was something like that, didn't took off. IMHO, the real issue is cache coherence. GPUs are spared from doing a lot of extra work by relaxing coherence guarantees quite a bit. Regarding the vendor situation - that's basically how most of computing hardware is, save for the PC platform. And this exception is due to Microsoft successfully commoditizing their complements (which caused quite some woe on the soft…

I miss, not exactly Larrabee, but what it could have become. I want just an insane number of very fast, very small cores with their own local memory.
Post reply on HN