Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I want a good parallel computer

raphlinus.github.io

Post reply on HN
Print view View original

141–150 of 209 posts

Re: I want a good parallel computer

#141

Post by bee_rider »

raphlinus wrote: post #127
adrian_b wrote: post #110

Earlier quoted context omitted.

It is an instance of Larrabee in the same sense as AMD Zen 4 is an instance of Larrabee. The "Larrabee New Instructions" is an instruction set that has been designed before AVX and also its first hardware implementation has been introduced before AVX, in 2010 (AVX was launched in 2011, with Sandy Bridge). Unfortunately while the hardware design of Sandy Bridge with the inferior AVX ISA has been done by the Intel A te…

> It is an instance of Larrabee in the same sense as AMD Zen 4 is an instance of Larrabee. This is an odd claim. Clearly Xeon Phi is the shipping version of Larrabee, while Zen 4 is a completely different chip design that happens to run AVX-512. The first shipping Xeon Phi (Knights Corner) used the exact same P54C cores as Larrabee, while as you point out later versions of Xeon Phi switched to Atom. It is extremely c…

I think it is not right to say that Larrabee and Phi are as distant as Larrabee and Zen. But, they did retreat a bit from the “graphics card” like functionality, and to scale back the ambitions to become something a bit more familiar.

Re: I want a good parallel computer

#142

Post by aleph_minus_one »

deviantbit wrote: post #49

"I believe there are two main things holding it back." He really science’d the heck out of that one. I’m getting tired of seeing opinions dressed up as insight—especially when they’re this detached from how real systems actually work. I worked on the Cell processor and I can tell you it was a nightmare. It demanded an unrealistic amount of micromanagement and gave developers rope to hang themselves with. There’s a re…

> What amazes me more is the comment section—full of people waxing nostalgic for architectures they clearly never had to ship stable software on.

Isn't it much more plausible that the people who love to play with exotic (or also retro), complicated architectures (with in this case high performance opportunities) are different people than those who love to "set up or work in an assembly line for shipping stable software"?

> I really believe I need to spend 2-weeks requiring students write code on an Amiga, and the programs have to run at the same time. If anyone of them crashes, they all will fail my course. A new found appreciation may flourish.

I rather believe that among those who love this kind of programming a hate for the incompetent fellow student will happen (including wishes that these become weed out by brutal exams).

Re: I want a good parallel computer

#143

Post by imtringued »

vgatherps wrote: post #76
Grosvenor wrote: post #30

Earlier quoted context omitted.

What I want is a Linear Algebra interface - As Gilbert Strang taught it. I'll "program" in LinAlg, and a JIT can compile it to whatever wonky way your HW requires. I'm not willing to even know about the HW at all, the higher level my code the more opportunities for the JIT to optimize my code. What I really want is something like Mathematica that can JIT to GPU. As another commenter mentioned all the API's assume you…

https://github.com/jax-ml/jax

I was about to say that it is literally just Jax.

It genuinely deserves to exist alongside pytorch. It's not just Google's latest framework that you're forced to use to target TPUs.

Re: I want a good parallel computer

#144

Post by imtringued »

mikewarot wrote: post #125

Any computing model that tries to parallelize von Neumann machines, that is, has program counters or address space, just isn't going to scale.

The problem isn't address space or program counters. It's that each processor is going to need instruction memory stored in SRAM or an extremely efficient multi port memory for a shared instruction cache.

GPUs get around this limitation by executing identical instructions over multiple threads.

Re: I want a good parallel computer

#145

Post by nwallin »

dist-epoch wrote: post #18
wmf wrote: post #8

Earlier quoted context omitted.

A big one is video encoding. It seems like GPUs would be ideal for it but in practice limitations in either the hardware or programming model make it hard to efficiently run on GPU shader cores. (GPUs usually include separate fixed-function video engines but these aren't programmable to support future codecs.)

Video encoding is done with fixed-function for power efficiency. A new popular codec like H26x codec appears every 5-10 years, there is no real need to support future ones.

Video encoding is two domains. And there's surprisingly little overlap between them.

You have your real time video encoding. This is video conferencing, live television broadcasts. This is done fixed-function not just for power efficiency, but also latency.

The second domain is encoding at rest. This is youtube, netflix, blu-ray, etc. This is usually done in software on the CPU for compression ratio efficiency.

The problem with fixed function video encoding is that the compression ratio is bad. You either have enormous data, or awful video quality, or both. The problem with software video encoding is that it's really slow. OP is asking why we can't/don't have the best of both worlds. Why can't/don't we write a video encoder in OpenCL/CUDA/ROCm. So that we have the speed of using the GPU's compute capability but compression ratio of software.

Re: I want a good parallel computer

#146

Post by winwang »

sitkack wrote: post #100
winwang wrote: post #78

Earlier quoted context omitted.

Personal opinion: it's the software (and software tooling). The hardware is good enough (even if we're only talking 10x efficiency). Part of the issue seems slightly cultural, i.e. repetitively putting down the idea of traditional task parallelism (not-super-SIMD/data parallelism) on GPUs. Obviously, one would lose a lot of efficiency if we literally ran 1 thread per warp. But it could be useful for lightly-data-para…

I agree. Many things in software are in the "you're doing it wrong" but that wrong way is subjective and arbitrary. > maybe using warp-wide semantics to implement something like a "software" microcode engine. https://github.com/beehive-lab/ProtonVM

Thanks for the share (and reminder)! Turns out I had this bookmarked somehow, lol.

Re: I want a good parallel computer

#147

Post by shae »

mikewarot wrote: post #14
IshKebab wrote: post #3

Having worked for a company that made a "hundreds of small CPUs on a single chip", I can tell you now that they're all going to fail because the programming model is too weird, and nobody will write software for them. Whatever comes next will be a GPU with extra capabilities, not a totally new architecture. Probably an nVidia GPU.

The key transformation required to make any parallel architecture work is going to be taking a program that humans can understand, and translating it into a directed acyclic graph of logical Boolean operations. This type of intermediate representation could then be broken up into little chunks for all those small CPUS. It could be executed very slowly using just a few logic gates and enough ram to hold the state, or…

This sounds like graph reduction as done by https://haflang.github.io/ and that flavor of special purpose CPU.

The downside of reducing a large graph is the need for high bandwidth low latency memory.

The upside is that tiny CPUs attached directly to the memory could do reduction (execution).

Re: I want a good parallel computer

#148

Post by mikewarot »

imtringued wrote: post #144
mikewarot wrote: post #125

Any computing model that tries to parallelize von Neumann machines, that is, has program counters or address space, just isn't going to scale.

The problem isn't address space or program counters. It's that each processor is going to need instruction memory stored in SRAM or an extremely efficient multi port memory for a shared instruction cache. GPUs get around this limitation by executing identical instructions over multiple threads.

Instructions are the problem, you have to have an architecture which just operates on data flows all in parallel and all at once, like an FPGA, but without all the fiddly special sauce parts.

Re: I want a good parallel computer

#149

Post by EVa5I7bHFq9mnYK »

grg0 wrote: post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

"want to protect their turd" - golden!

Re: I want a good parallel computer

#150

Post by [deleted] »

Brian_K_White wrote: post #117
deviantbit wrote: post #66

Earlier quoted context omitted.

[flagged]

What the ever loving hell, it was a perfectly reasonable idea in response to another idea. They weren't saying it should be done, and went out of the way to make it explicit that they are not claiming it would be better. It was a thought exploration, and a valid one, even if it would not pan out if carried all the way to execution at scale. Yes it was handwaving. So what? All ideas start as mere thoughts, and it is u…

[deleted]
Post reply on HN