Exciting if an Ai that is helping in its own improvements finds this and incorporates it into its own architecture. Then it starts reading and running all the worlds binary and gains intelligence as a fully actualized "computer". Finally becoming both a master of language and of binary bits. Thinking in poetry and in pure precise numerical calculations.
A CPU that runs entirely on GPU
51–60 of 144 posts
Re: A CPU that runs entirely on GPU
#52I don‘t understand why you would train a NN for an operation like sqrt that the GPU supports in silicon.
Re: A CPU that runs entirely on GPU
#53A fun experiment but I wonder how many out there seriously think we could ever completely rid ourselves of the CPU. It seems to be a rising sentiment. The cost of communicating information through space is dealt with in fundamentally different ways here. On the CPU it is addressed directly. The actual latency is minimized as much as possible, usually by predicting the future in various ways and keeping the spatial ex…
> I wonder how many out there seriously think we could ever completely rid ourselves of the CPU. How do you class systems like the PS5 that have an APU plugged into GDDR instead of regular RAM? The primary remaining issue is the limited memory capacity. I wonder if we might see a system with GPU class HBM on the package in lieu of VRAM coupled with regular RAM on the board for the CPU portion?
I don’t think the remaining issue is memory capacity. CPUs are designed to handle nonlinear memory access and that is how all modern software targeting a CPU is written. GPUs are designed for linear memory access. These are fundamentally different access patterns the optimal solution is to have 2 distinct processing units
Re: A CPU that runs entirely on GPU
#54As foretold six years ago. [1] [1]: https://breandan.net/2020/06/30/graph-computation#roadmap
https://en.wikipedia.org/wiki/Xeon_Phi#Knights_Landing ?
Re: A CPU that runs entirely on GPU
#55A fun experiment but I wonder how many out there seriously think we could ever completely rid ourselves of the CPU. It seems to be a rising sentiment. The cost of communicating information through space is dealt with in fundamentally different ways here. On the CPU it is addressed directly. The actual latency is minimized as much as possible, usually by predicting the future in various ways and keeping the spatial ex…
I see us not getting rid of CPU, but CPU and GPU being eventually consolidated in one system of heterogeneous computing units.
CPU and GPU have very different ways of scheduling instructions, requiring somehow different interfaces and programming models.. I'd hazard to say that a GPU and CPU with unified memory access (like the Apple's M series, and most mobile chips) is already such a consolidated system.
Re: A CPU that runs entirely on GPU
#56I don‘t understand why you would train a NN for an operation like sqrt that the GPU supports in silicon.
I see it as a practical joke or a fun hack, like CPUs implemented in the Game of Life, or in Minecraft.
Re: A CPU that runs entirely on GPU
#57Earlier quoted context omitted.
> We need llms to be able to tap that not add the same functionality a layer above and MUCH less efficiently. Agents, tool-integrated reasoning, even chain of thought (limited, for some math) can address this.
You're both completely missing the point. It's important that an LLM be able to perform exact arithmetic reliably without a tool call. Of course the underlying hardware does so extremely rapidly, that's not the point.
Could you explain why that is?
Re: A CPU that runs entirely on GPU
#58Someone needs to implement LLVMPipe to target this isa, then one can run software OpenGL emulation and call it "hardware accelerated".
Re: A CPU that runs entirely on GPU
#59Earlier quoted context omitted.
> I wonder how many out there seriously think we could ever completely rid ourselves of the CPU. How do you class systems like the PS5 that have an APU plugged into GDDR instead of regular RAM? The primary remaining issue is the limited memory capacity. I wonder if we might see a system with GPU class HBM on the package in lieu of VRAM coupled with regular RAM on the board for the CPU portion?
I don’t think the remaining issue is memory capacity. CPUs are designed to handle nonlinear memory access and that is how all modern software targeting a CPU is written. GPUs are designed for linear memory access. These are fundamentally different access patterns the optimal solution is to have 2 distinct processing units
If anything, GPUs combine large private per-compute unit private address spaces and a separate shared/global memory, which doesn't mesh very well with linear memory access, just high locality. You can kinda get to the same arrangement on CPU by pushing NUMA (Non-Uniform Memory: only the "global" memory is truly Unified on a GPU!) to the extreme, but that's quite uncommon. "Compute-in-memory" is a related idea that kind of points to the same constraint: you want to maximize spatial locality these days, because moving data in bulk is an expensive operation that burns power.
Re: A CPU that runs entirely on GPU
#60Earlier quoted context omitted.
You're both completely missing the point. It's important that an LLM be able to perform exact arithmetic reliably without a tool call. Of course the underlying hardware does so extremely rapidly, that's not the point.
Could you explain why that is?
A tool call is like 100,000,000x slower isnt it?