Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I want a good parallel computer

raphlinus.github.io

Post reply on HN
Print view View original

21–30 of 209 posts

Re: I want a good parallel computer

#21

Post by dr_kiszonka »

andrewstuart wrote: post #9

AMD Strix Halo APU is a CPU with very powerful integrated GPU. It’s faster at AI than an Nvidia RTX4090, because 96GB of the 128GB can be allocated to the GPU memory space. This means it’s doesn’t have the same swapping/memory thrashing that a discrete GPU experiences when processing large models. 16 CPU cores and 40 GPU compute units sounds pretty parallel to me. Doesn’t that fit the bill?

It looks like it will be available in the Framework Desktop! I would love to see it in a more budget mini PC at some point from another company. (Framework is great but not in my price range.)

Re: I want a good parallel computer

#22

Post by andrewstuart »

bigyabai wrote: post #17
andrewstuart wrote: post #9

AMD Strix Halo APU is a CPU with very powerful integrated GPU. It’s faster at AI than an Nvidia RTX4090, because 96GB of the 128GB can be allocated to the GPU memory space. This means it’s doesn’t have the same swapping/memory thrashing that a discrete GPU experiences when processing large models. 16 CPU cores and 40 GPU compute units sounds pretty parallel to me. Doesn’t that fit the bill?

> It’s faster at AI than an Nvidia RTX4090, because 96GB of the 128GB can be allocated to the GPU memory space I love AMD's Ryzen chips and will recommend their laptops over an Nvidia model all day. However, this is a pretty facetious comparison that falls apart when you normalize the memory. Any chip can be memory bottlenecked, and if we take away that arbitrary precondition the Strix Halo gets trounced in terms of…

“ AMD also claims its Strix Halo APUs can deliver 2.2x more tokens per second than the RTX 4090 when running the Llama 70B LLM (Large Language Model) at 1/6th the TDP (75W).”

https://www.tomshardware.com/pc-components/cpus/amd-slides-c...

You could argue it’s invalid claim because it’s from AMD not independent.

Re: I want a good parallel computer

#23

Post by Animats »

dist-epoch wrote: post #20
bigyabai wrote: post #17

Earlier quoted context omitted.

> It’s faster at AI than an Nvidia RTX4090, because 96GB of the 128GB can be allocated to the GPU memory space I love AMD's Ryzen chips and will recommend their laptops over an Nvidia model all day. However, this is a pretty facetious comparison that falls apart when you normalize the memory. Any chip can be memory bottlenecked, and if we take away that arbitrary precondition the Strix Halo gets trounced in terms of…

> However, this is a pretty facetious comparison that falls apart when you normalize the memory Why would you normalize though? You can't buy a 96 GB RTX4090. So it's fair to compare the whole deal, slowish APU with large RAM versus very fast GPU with limited RAM.

> You can't buy a 96 GB RTX4090

You can now buy a 96 GB RTX5090.[1] NVidia gives it a "Pro" designation and charges more, but it's the same chip.

[1] https://www.tomshardware.com/pc-components/gpus/nvidia-rtx-p...

Re: I want a good parallel computer

#25

Post by [deleted] »

Retr0id wrote: post #13
bee_rider wrote: post #6

It is odd that he talks about Larabee so much, but doesn’t mention the Xeon Phis. (Or is it Xeons Phi?). > As a general trend, CPU designs are diverging into those optimizing single-core performance (performance cores) and those optimizing power efficiency (efficiency cores), with cores of both types commonly present on the same chip. As E-cores become more prevalent, algorithms designed to exploit parallelism at sca…

Isn't Xeon Phi just an instance of Larrabee?

[deleted]

Re: I want a good parallel computer

#26

Post by Retr0id »

Something that frustrates me a little is that my system (apple silicon) has unified memory, which in theory should negate the need to shuffle data between CPU and GPU. But, iiuc, the GPU programming APIs at my disposal all require me to pretend the memory is not unified - which makes sense because they want to be portable across different hardware configurations. But it would make my life a lot easier if I could just target the hardware I have, and ignore compatibility concerns.

Re: I want a good parallel computer

#27

Post by morphle »

I haven't yet read the full blog post but so far my response is you can have this good parallel computer. See my previous HN comments the past months on building an M4 Mac mini supercomputer.

For example reverse engineering the Apple M3 Ultra GPU and Neural Engine instruction set and IOMMU and pages tables that prevent you from programming all processor cores in the chip (146 cores to over ten thousand depending on how you delineate what a core is) and making your own Abstract Syntax Tree to assembly compiler for these undocumented cores will unleash at least 50 trillion operations per second. I still have to benchmark this chip and make the roofline graphs for the M4 to be sure, it might be more.

https://en.wikipedia.org/wiki/Roofline_model

Re: I want a good parallel computer

#28

Post by Grosvenor »

KerrAvon wrote: post #16
mikewarot wrote: post #14

Earlier quoted context omitted.

The key transformation required to make any parallel architecture work is going to be taking a program that humans can understand, and translating it into a directed acyclic graph of logical Boolean operations. This type of intermediate representation could then be broken up into little chunks for all those small CPUS. It could be executed very slowly using just a few logic gates and enough ram to hold the state, or…

Isn't that the Connection Machine architecture?

I would not complain about getting my own personal Connection Machine.

So long as Tamiko Thiel does the design.

Re: I want a good parallel computer

#29

Post by deviantbit »

Retr0id wrote: post #26

Something that frustrates me a little is that my system (apple silicon) has unified memory, which in theory should negate the need to shuffle data between CPU and GPU. But, iiuc, the GPU programming APIs at my disposal all require me to pretend the memory is not unified - which makes sense because they want to be portable across different hardware configurations. But it would make my life a lot easier if I could just…

Unified memory doesn't mean unified address space. It frustrates me when no one understands unified memory.

Re: I want a good parallel computer

#30

Post by Grosvenor »

grg0 wrote: post #19

The issue is that programming a discrete GPU feels like programming a printer over a COM port, just with higher bandwidths. It's an entirely moronic programming model to be using in 2025. - You need to compile shader source/bytecode at runtime; you can't just "run" a program. - On NUMA/discrete, the GPU cannot just manipulate the data structures the CPU already has; gotta copy the whole thing over. And you better des…

What I want is a Linear Algebra interface - As Gilbert Strang taught it. I'll "program" in LinAlg, and a JIT can compile it to whatever wonky way your HW requires.

I'm not willing to even know about the HW at all, the higher level my code the more opportunities for the JIT to optimize my code.

What I really want is something like Mathematica that can JIT to GPU.

As another commenter mentioned all the API's assume you're a discrete GPU off the end of a slow bus, without shared memory. I would kill for an APU that could freely allocate memory for GPU or CPU and change ownership with the speed of a pagefault or kernel transition.

Post reply on HN