Live data from Hacker News

I want a good parallel computer

raphlinus.github.io

31–40 of 209 posts

Re: I want a good parallel computer

#31
post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

doesn't matter. the issues you raise are abstractable at the language level, or maybe even the runtime. unfortunately there are others like which of the many kinds of parallelism to use (ILP, thread, vector/SIMD, distributed memory with much lower performance, etc.) that are harder to hide behind a compiler with acceptable performance.

Re: I want a good parallel computer

#32
Interesting article.

Other than as an exercise, it's not clear why someone would write a massively parallel 2D renderer that needs a GPU. Modern GPUs are overkill for 2D. Now, 3D renderers, we need all the help we can get.

In this context, a "renderer" is something that takes in meshes, textures, materials, transforms, and objects, and generates images. It's not an entire game development engine, such as Unreal, Unity, or Bevy. Those have several more upper levels above the renderer. Game engines know what all the objects are and what they are doing. Renderers don't.

Vulkan, incidentally, is a level below the renderer. Vulkan is a cross-hardware API for asking a GPU to do all the things a GPU can do. WGPU for Rust, incidentally, is an wrapper to extend that concept to cross-platform (Mac, Android, browsers, etc.)

While it seems you can write a general 3D renderer that works in a wide variety of situations, that does not work well in practice. I wish Rust had one. I've tried Rend3 (abandoned), and looked at Renderling (in progress), Orbit (abandoned), and Three.rs (abandoned). They all scale up badly as scene complexity increases.

There's a friction point in design here. The renderer needs more info to work efficiently than it needs to just draw in a dumb way. Modern GPSs are good enough that a dumb renderer works pretty well, until the scene complexity hits some limit. Beyond that point, problems such as lighting requiring O(lights * objects) time start to dominate. The CPU driving the GPU maxes out while the GPU is at maybe 40% utilization. The operations that can easily be parallelized have been. Now it gets hard.

In Rust 3D land, everybody seems to write My First Renderer, hit this wall, and quit.

The big game engines (Unreal, etc.) handle this by using the scene graph info of the game to guide the rendering process. This is visually effective, very complicated, prone to bugs, and takes a huge engine dev team to make work.

Nobody has a good solution to this yet. What does the renderer need to know from its caller? A first step I'm looking at is something where, for each light, the caller provides a lambda which can iterate through the objects in range of the light. That way, the renderer can get some info from the caller's spatial data structures. May or may not be a good idea. Too early to tell.

[1] https://github.com/linebender/vello/

Re: I want a good parallel computer

#33
post #26

Something that frustrates me a little is that my system (apple silicon) has unified memory, which in theory should negate the need to shuffle data between CPU and GPU. But, iiuc, the GPU programming APIs at my disposal all require me to pretend the memory is not unified - which makes sense because they want to be portable across different hardware configurations. But it would make my life a lot easier if I could just…

Unified memory doesn't mean unified address space. It frustrates me when no one understands unified memory.

If you fix the pages tables (partial tutorial online) you can have continuous unified address space on Apple Silicon.

Re: I want a good parallel computer

#34
post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

Please explain how these "worker cores" should operate.

Re: I want a good parallel computer

#35
post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

Your wish sounds to me a lot like Larrabee/Xeon Phi or manycore CPUs. Maybe I am misunderstanding something, but it sounds like a good idea to me and I don’t totally see why it inherently can’t compete with GPUs.

Re: I want a good parallel computer

#36
post #33

Earlier quoted context omitted.

Unified memory doesn't mean unified address space. It frustrates me when no one understands unified memory.

If you fix the pages tables (partial tutorial online) you can have continuous unified address space on Apple Silicon.

Let’s be honest, saying “just fix the page tables” is like telling someone they can fly if they “just rewrite gravity.”

Yes, on Apple Silicon, the hardware supports shared physical memory, and with enough “convincing”, you can rig up a contiguous virtual address space for both the CPU and GPU. Apple’s unified memory architecture makes that possible, but Apple’s APIs and memory managers don’t expose this easily or safely for a reason. You’re messing with MMU-level mappings on a tightly integrated system that treats memory as a first-class citizen of the security model.

I can tell you never programmed on an Amiga.

Re: I want a good parallel computer

#37
post #3

Having worked for a company that made a "hundreds of small CPUs on a single chip", I can tell you now that they're all going to fail because the programming model is too weird, and nobody will write software for them. Whatever comes next will be a GPU with extra capabilities, not a totally new architecture. Probably an nVidia GPU.

The key transformation required to make any parallel architecture work is going to be taking a program that humans can understand, and translating it into a directed acyclic graph of logical Boolean operations. This type of intermediate representation could then be broken up into little chunks for all those small CPUS. It could be executed very slowly using just a few logic gates and enough ram to hold the state, or…

Like interaction nets?

Re: I want a good parallel computer

#38
post #3

Having worked for a company that made a "hundreds of small CPUs on a single chip", I can tell you now that they're all going to fail because the programming model is too weird, and nobody will write software for them. Whatever comes next will be a GPU with extra capabilities, not a totally new architecture. Probably an nVidia GPU.

Yep, transputers failed miserably. I wrote a ton a code for them. Everything had to be solved in a serial bus, which defeated the purpose of the transputer.

Re: I want a good parallel computer

#39
post #33

Earlier quoted context omitted.

If you fix the pages tables (partial tutorial online) you can have continuous unified address space on Apple Silicon.

Let’s be honest, saying “just fix the page tables” is like telling someone they can fly if they “just rewrite gravity.” Yes, on Apple Silicon, the hardware supports shared physical memory, and with enough “convincing”, you can rig up a contiguous virtual address space for both the CPU and GPU. Apple’s unified memory architecture makes that possible, but Apple’s APIs and memory managers don’t expose this easily or saf…

Oh yes I programmed all the Amiga models, mostly in assembly level. I reprogrammed the ROMs. I also published a magazine on all the Commodore computers internals and build lots of hardware for these machines.

We had the parallel Inmos Transputer systems during the heyday of the Amiga, they where much better designed than any the custom Amiga chips.

Re: I want a good parallel computer

#40
post #33

Earlier quoted context omitted.

If you fix the pages tables (partial tutorial online) you can have continuous unified address space on Apple Silicon.

Let’s be honest, saying “just fix the page tables” is like telling someone they can fly if they “just rewrite gravity.” Yes, on Apple Silicon, the hardware supports shared physical memory, and with enough “convincing”, you can rig up a contiguous virtual address space for both the CPU and GPU. Apple’s unified memory architecture makes that possible, but Apple’s APIs and memory managers don’t expose this easily or saf…

I know the APIs don't make it easy, that's precisely why I want different APIs.
Post reply on HN