Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I want a good parallel computer

raphlinus.github.io

Post reply on HN
Print view View original

91–100 of 209 posts

Re: I want a good parallel computer

#91

Post by monocasa »

nicoburns wrote: post #83
deviantbit wrote: post #49

"I believe there are two main things holding it back." He really science’d the heck out of that one. I’m getting tired of seeing opinions dressed up as insight—especially when they’re this detached from how real systems actually work. I worked on the Cell processor and I can tell you it was a nightmare. It demanded an unrealistic amount of micromanagement and gave developers rope to hang themselves with. There’s a re…

> They forget why we moved on. Modern systems are built with constraints like memory protection, isolation, and stability in mind. You can’t just “flatten address spaces” and ignore the consequences. Is there any reason why GPU-style parallelism couldn't have memory protection?

It does. GPUs have full MMUs.

Re: I want a good parallel computer

#92

Post by pabs3 »

mikewarot wrote: post #14
IshKebab wrote: post #3

Having worked for a company that made a "hundreds of small CPUs on a single chip", I can tell you now that they're all going to fail because the programming model is too weird, and nobody will write software for them. Whatever comes next will be a GPU with extra capabilities, not a totally new architecture. Probably an nVidia GPU.

The key transformation required to make any parallel architecture work is going to be taking a program that humans can understand, and translating it into a directed acyclic graph of logical Boolean operations. This type of intermediate representation could then be broken up into little chunks for all those small CPUS. It could be executed very slowly using just a few logic gates and enough ram to hold the state, or…

Reminds me of Mill Computing's stuff.

https://millcomputing.com/

Re: I want a good parallel computer

#93

Post by zozbot234 »

pabs3 wrote: post #92
mikewarot wrote: post #14

Earlier quoted context omitted.

The key transformation required to make any parallel architecture work is going to be taking a program that humans can understand, and translating it into a directed acyclic graph of logical Boolean operations. This type of intermediate representation could then be broken up into little chunks for all those small CPUS. It could be executed very slowly using just a few logic gates and enough ram to hold the state, or…

Reminds me of Mill Computing's stuff. https://millcomputing.com/

Mill Computing's proposed architecture is more like VLIW with lots of custom "tricks" in the ISA and programming model to make it nearly as effective as the usual out-of-order execution than a "generic sea" of small CPU's. VLIW CPU's are far from 'tiny' in a general sense.

Re: I want a good parallel computer

#94

Post by monocasa »

zozbot234 wrote: post #79
tliltocatl wrote: post #72

Earlier quoted context omitted.

Larrabee was something like that, didn't took off. IMHO, the real issue is cache coherence. GPUs are spared from doing a lot of extra work by relaxing coherence guarantees quite a bit. Regarding the vendor situation - that's basically how most of computing hardware is, save for the PC platform. And this exception is due to Microsoft successfully commoditizing their complements (which caused quite some woe on the soft…

Is cache coherence a real issue, absent cache contention ? AIUI, cache coherence protocols are sophisticated enough that they should readily adapt to workloads where the same physical memory locations are mostly not accessed concurrently except in pure "read only" mode. So even with a single global address space, it should be possible to make this work well enough if the programs are written as if they were running o…

It is because cache coherence requires extra communication to make sure that the cache is coherent. There's cute stratgies for reducing the traffic, but ultimately you need to broadcast out reservations to all of the other cache coherent nodes, so there's an N^2 scaling at play.

Re: I want a good parallel computer

#95

Post by dragontamer »

There's a lot here that seems to misunderstand GPUs and SIMD.

Note that raytracing is a very dynamic problem, where the GPU isn't sure if a ray hits a geometry or if it misses. When it hits, the ray needs to bounce, possibly multiple times.

Various implementations of raytracing, recursion, dynamic parallelism or whatever. Its all there.

Now the software / compilers aren't ready (outside of specialized situations like Microsofts DirectX Raytracing, which compiles down to a very intriguing threading model). But what was accomplished with DirectX can be done in other situations.

-------

Connection Machine is before my time, but there's no way I'd consider that 80s hardware to be comparable to AVX2 let alone a modern GPU.

Connection Machine was a 1-bit computer for crying out loud, just 4096 of them in parallel.

Xeon Phi (70 core Intel Atoms) is slower and weaker than 192 core Modern EPYC chips.

-------

Today's machines are better. A lot better than the past machines. I cannot believe any serious programmer would complain about the level of parallelism we have today and wax poetic about historic and archaic computers.

Re: I want a good parallel computer

#96

Post by nromiun »

What about unified memory? I know these APUs are slower than traditional GPUs but still it seems like the simpler programming model will be worth it.

The biggest problem is that most APUs don't even support full unified memory (system SVM in OpenCL). From my research only Apple M series, some Qualcomm Adreno and AMD APUs support them.

Re: I want a good parallel computer

#97

Post by throwawaynin »

newpavlov wrote: post #50
grg0 wrote: post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

>What I would like is a CPU with a highly parallel array of "worker cores" all addressing the same memory and speaking the same goddamn language that the CPU does. For "embarrassingly parallel" jobs vector extensions start to eat tiny bits of the GPU pie. Unfortunately, just slapping thousands of cores works poorly in practice. You quickly get into the synchronization wall caused by unified memory. GPUs cleverly work…

Networks of cores ... Congrats you have just taken a computer and shrunk it so there are many on a single chip ... Just gonna say here AWS does exactly this network of computers thing ... Might be profitable

Re: I want a good parallel computer

#98

Post by Animats »

amelius wrote: post #51
Animats wrote: post #32

Interesting article. Other than as an exercise, it's not clear why someone would write a massively parallel 2D renderer that needs a GPU. Modern GPUs are overkill for 2D. Now, 3D renderers, we need all the help we can get. In this context, a "renderer" is something that takes in meshes, textures, materials, transforms, and objects, and generates images. It's not an entire game development engine, such as Unreal, Unit…

> Other than as an exercise, it's not clear why someone would write a massively parallel 2D renderer that needs a GPU. Modern GPUs are overkill for 2D. Depends on how complicated your artwork is.

There are only so many screen pixels.

Re: I want a good parallel computer

#99

Post by throwawayabcdef »

The AIE arrays on Versal and Ryzen with XDNA are a big grid of cores (400 in an 8 x 50 array) that you program with streaming work graphs.

https://docs.amd.com/r/en-US/am009-versal-ai-engine/Overview

Each AIE tile can stream 64 Gbps in and out and perform 1024 bit SIMD operations. Each shares memory with its neighbors and the streams can be interconnected in various ways.

Re: I want a good parallel computer

#100

Post by sitkack »

winwang wrote: post #78
sitkack wrote: post #70

This essay needs more work. Are you arguing for a better software abstraction, a different hardware abstraction or both? Lots of esoteric machines are name dropped, but it isn't clear how that helps your argument. Why not link to Vello? https://github.com/linebender/vello I think a stronger essay would at the end give the reader a clear view of what Good means and how to decide if a machine is closer to Good than ano…

Personal opinion: it's the software (and software tooling). The hardware is good enough (even if we're only talking 10x efficiency). Part of the issue seems slightly cultural, i.e. repetitively putting down the idea of traditional task parallelism (not-super-SIMD/data parallelism) on GPUs. Obviously, one would lose a lot of efficiency if we literally ran 1 thread per warp. But it could be useful for lightly-data-para…

I agree.

Many things in software are in the "you're doing it wrong" but that wrong way is subjective and arbitrary.

> maybe using warp-wide semantics to implement something like a "software" microcode engine.

https://github.com/beehive-lab/ProtonVM

Post reply on HN