151–160 of 209 posts
Earlier quoted context omitted.
It lacks support for the serial portions of the execution graph, but yes. You should play around with ONNX, it can be used for a lot more than just ML stuff.
What do you mean by serial portions? Aren't operations automatically serialized if there are dependencies between them?
ONNX doesn't have the direct capabilities to be the compilation target for regular imperative code.
> The GPU in your computer is about 10 to 100 times more powerful than the CPU, depending on workload. For real-time graphics rendering and machine learning, you are enjoying that power, and doing those workloads on a CPU is not viable. Why aren’t we exploiting that power for other workloads? What prevents a GPU from being a more general purpose computer? What other workloads would benefit from a GPU? Computers are s…
Possibly compilation and linking. That's very slow for big programs like Chromium. There's really interesting work on GPU compilers (co-dfns and Voetter's work). Optimization problems like scheduling and circuit routing. Search in theorem proving (the classical parts like model checking, not just LLM). There's still a lot that is slow and should be faster, or at the very least made to run using less power. GPUs are g…
The tasks that GPUs are good at right now - graphics, number crunching, etc - are all very simple algorithms at the core (mostly elementary linear algebra), and the problems are, in most cases, embarassingly parallel.
CPUs are not very good at branching either - see all the effort being put towards getting branch prediction right - but they are way better at it than GPUs. The main appeal of GPGPU programming is, in my opinion, that if you can get the CPU to efficiently divide the larger problem into a lot of small, simple subtasks, you can achieve faster speeds.
You mentioned compilers. See a related example, for reference all the work Daniel Lemire has been doing on SIMD parsing: the algorithms he (co)invented are all highly specialized to the language, and highly nontrivial. Branchless programming requires an entirely different mindset/intuition than "traditional" programming, and I wouldn't expect the average programmer to come up with such novel ideas.
A GPU is a specialized tool that is useful for a particular purpose, not a silver bullet to magically speed up your code. Theree is a reason that we are using it for its current purposes.
Earlier quoted context omitted.
They weren't talking about instruction level parallelism.
Similarities between two things don't require them to be identical.
They aren't just different, they are at completely opposite ends of the programming spectrum. There are literally the two extremes of trying to make throughput faster.
Interesting article. Other than as an exercise, it's not clear why someone would write a massively parallel 2D renderer that needs a GPU. Modern GPUs are overkill for 2D. Now, 3D renderers, we need all the help we can get. In this context, a "renderer" is something that takes in meshes, textures, materials, transforms, and objects, and generates images. It's not an entire game development engine, such as Unreal, Unit…
Raytracing (in its many forms) is one solution. Precomputing lightmaps, probes, occluder volumes, or other forms of precomputed visibility are another.
In the end it comes down to a combination of target hardware, art direction and requirements, and technical skill available for each game.
There's not going to be one general purpose renderer you can plug into anything, _and_ expect it to be fast, because there's no general solution to light transport and geometry processing that fits everyone's requirements. Precomputation doesn't work for dynamic scenes, and for large games leads to issues with storage size and workflow slow downs across teams. No precomputation at all requires extremely modern hardware and cutting edge research, has stability issues, and despite all that is still very slow.
It's why game engines offer several different forms of lighting methods, each with as many downsides as they have upsides. Users are supposed to pick the one that best fits their game, and hope it's good enough. If it's not, you write something custom (if you have the skills for that, or can hire someone who can), or change your game to fit the technical constraints you have to live with.
> Nobody has a good solution to this yet. What does the renderer need to know from its caller? A first step I'm looking at is something where, for each light, the caller provides a lambda which can iterate through the objects in range of the light. That way, the renderer can get some info from the caller's spatial data structures. May or may not be a good idea. Too early to tell.
Some games may have their own acceleration structures. Some won't. Some will only have them on the GPU, not the CPU. Some will have an approximate structure used only for specialized tasks (culling, audio, lighting, physics, etc), and cannot be generalized to other tasks without becoming worse at their original task.
Fully generalized solutions will be slow be flexible, and fully specialized solutions will be fast but inflexible. Game design is all about making good tradeoffs.
Something that frustrates me a little is that my system (apple silicon) has unified memory, which in theory should negate the need to shuffle data between CPU and GPU. But, iiuc, the GPU programming APIs at my disposal all require me to pretend the memory is not unified - which makes sense because they want to be portable across different hardware configurations. But it would make my life a lot easier if I could just…
At least for Vulkan/DirectX12. Metal is often weird, I don't know what's available there.
This essay needs more work. Are you arguing for a better software abstraction, a different hardware abstraction or both? Lots of esoteric machines are name dropped, but it isn't clear how that helps your argument. Why not link to Vello? https://github.com/linebender/vello I think a stronger essay would at the end give the reader a clear view of what Good means and how to decide if a machine is closer to Good than ano…
I don't speak for Raph, but imo it seems like he was arguing for both, and I agree with him.
On the hardware side, GPUs have struggled with dynamic workloads at the API level (not e.g. thread-level dynamism, that's a separate topic) for around a decade. Indirect commands gave you some of that so at least the size of your data/workload can be variable if not the workloads themselves, then mesh shaders gave you a little more access to geometry processing, and finally workgraphs and device generated commands lets you have an actually dynamically defined workload (e.g. completely skipping dispatches for shading materials that weren't used on screen this frame). However it's still very early days, and the performance issues and lack of easy portability are problematic. See https://interplayoflight.wordpress.com/2024/09/09/an-introdu... for instance.
On the software side shading languages have been garbage for far longer than hardware has been a problem. It's only in the last year or two that a proper language server for writing shaders has even existed (Slang's LSP). Much less the innumerable driver compiler bugs, lack of well defined semantics and memory model until the last few years, or the fact that we're still manually dividing work into the correct cache-aware chunks.
There's a lot here that seems to misunderstand GPUs and SIMD. Note that raytracing is a very dynamic problem, where the GPU isn't sure if a ray hits a geometry or if it misses. When it hits, the ray needs to bounce, possibly multiple times. Various implementations of raytracing, recursion, dynamic parallelism or whatever. Its all there. Now the software / compilers aren't ready (outside of specialized situations like…
The problems I'm having are very different than those for raytracing. Sure, it's dynamic, but at a fine granularity, so the problems you run into are divergence, and often also wanting function pointers, which don't work well in a SIMT model, By contrast, the way I'm doing 2D there's basically no divergence (monoids are cool that way) but there is a need to schedule dynamically at a coarser (workgroup) level. But the…
1. Divergence of threads within a workgroup/SM/whatever
2. Dynamically scheduling new workloads (i.e. dispatches, draws, etc) in response to the output of a previous workload
Raytracing is problem #1 (and has it's own solutions, like shader execution reodering), while Raph is talking about problem #2.
Earlier quoted context omitted.
I agree. Many things in software are in the "you're doing it wrong" but that wrong way is subjective and arbitrary. > maybe using warp-wide semantics to implement something like a "software" microcode engine. https://github.com/beehive-lab/ProtonVM
Thanks for the share (and reminder)! Turns out I had this bookmarked somehow, lol.