I like the retro-ish and out of trend name they've chosen: AheadComputing.
Together Compute SFCompute And so on … definitely not out of trend
Computing sounds retro, but Compute does not.
Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
121–130 of 142 posts
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#122Earlier quoted context omitted.
Wondering how you'd classify Gaudi, tenstorrent-stuff, groq, or lightmatter's photonic thing. Calling something a GPU tends to make people ask for (good, performant) support for opengl, Vulkan, direct3d... which seem like a huge waste of effort if you want to be an "AI-coprocessor".
> Wondering how you'd classify Gaudi, tenstorrent-stuff, groq, or lightmatter's photonic thing. Completely irrelevant to consumer hardware, in basically the same way as NVIDIA's Hopper (a data center GPU that doesn't do graphics). They're ML accelerators that for the foreseeable future will mostly remain discrete components and not be integrated onto Xeon/EPYC server CPUs. We've seen a handful of products where a sma…
Consumer hardware and AI inference are joined at the hip right now due to perverse historical reasons.
AI inference's big bottleneck right now is RAM and memory bandwidth, not so much compute per se.
If we redid AI inference from scratch without consumer gaming considerations then it probably wouldn't be a coprocessor at all.
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#123I tire of “Employees from Y company leave to start their own” and even “Ex-Y employees launch new W”. How many times do we have to see these stories play out to realize it doesn’t matter where they came from. These big companies employee a lot of people of varying skill, having it on your resume means almost nothing IMHO. Just look at the Humane pin full of “ex-Apple employees”, how’d that work out? And that’s only o…
There are other forces at play. Same happens with video games. People can't see how many "variables" must be tuned in order to become notified and successful.
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#124They should have named it Zilog…
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#125One of the biggest problems with CPUs is legacy. Tie yourself to any legacy, and now you're spending millions of transistors to make sure some way that made sense ages ago still works. Just as a thought experiment, consider the fact that the i80486 has 1.2 million transistors. An eight core Ryzen 9700X has around 12 billion. The difference in clock speed is roughly 80 times, and the difference in number of transistor…
CPUs can’t do that, but legacy is irrelevant. They just don’t have enough parallelism to leverage all these extra transistors. Let’s compare the 486 with a modern GPU.
Intel 80486 with 1.2M transistors delivered 0.128 flops / cycle.
nVidia 4070 Ti Super with 45.9B transistors delivers 16896 flops / cycle.
As you see, each transistor became 3.45 times more efficient at delivering these FLOPs per cycle.
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#126I hope they design, build and sell a true 256-1024+ multicore CPU with local memories that appears as an ordinary desktop computer with a unified memory space for under $1000. I've written about it at length and I'm sure that anyone who's seen my comments is sick of me sounding like a broken record. But there's truly a vast realm of uncharted territory there. I believe that transputers and reprogrammable logic chips…
The challenges that arise when you have a massively parallel system are well understood by now. It is hard to keep all processing units doing something useful rather than waiting for memory or other processing units.
Once you follow the logical steps to increase utilization/efficiency you end up with something like a GPU, and that comes with the programming challenges that we have today.
In other words, it's not like CPU architects didn't think of that. Instead, there are good reasons for the status quo.
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#127Earlier quoted context omitted.
Echoing the other comment, this isn't easier. I was on a team that did it. The ML team was overheard by media complaining that we were preventing them from achieving their goals because we had taken 2 years to build something that didn't beat the latest hardware from Nvidia, let alone keep pace with how fast their demands had grown.
I don't need it to beat the latest from nvidia, just be affordable, available, and have user servicable ram slots so "48gb" isn't such an ooo-ahh amount of memory I couldn't find any buy it now links but 512gb sticks don't seem to be fantasies, either: https://news.samsung.com/global/samsung-develops-industrys-f...
What would you want from on-gpu slots that you don't get from existing mechanisms to give your GPU access to system memory?
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#128Earlier quoted context omitted.
That doesn't really change anything. The use cases for a GPU in any given market segment don't change depending on whether you call it a GPU. And for low-power consumer devices like laptops, "matrix multiplication coprocessor for AI tasks" is at least as likely to mean NPU as GPU, and NPUs are always integrated rather than discrete.
Yes it does change something. A GPU needs to run $GAME from $CURRENT_YEAR at 60 fps despite the ten million SLoC of shit code and legacy cruft in $GAME. That's where the huge expense for the GPU manufacturer lies. Matrix multiplication is a solved probelm and we need to implement it just once in hardware. At some point matrix multiplication will be ubiquitous like floating-point is now.
You're completely ignoring that there are several distinct market segments that want hardware to do AI/ML. Matrix multiplication is not something you can implement in hardware just once.
NVIDIA's biggest weakness right now is that none of their GPUs are appropriate for any system with a lower power budget than a gaming laptop. There's a whole ecosystem of NPUs in phone and laptop SoCs targeting different tradeoffs in size, cost, and power than any of NVIDIA's offerings. These accelerators represent the biggest threat NVIDIA's CUDA monopoly has ever faced. The only response NVIDIA has at the moment is to start working with MediaTek to build laptop chips with NVIDIA GPU IP and start competing against pretty much the entire PC ecosystem.
At the same time, all the various low-power NPU architectures have differing limitations owing to their diverse histories, and approximately none of them currently shipping were designed from the beginning with LLMs in mind. On the timescale of hardware design cycles, AI is still a moving target.
So far, every laptop or phone SoC that has shipped with both an NPU and a GPU has demonstrated that there are some AI workloads where the NPU offers drastically better power efficiency. Putting a small-enough NVIDIA GPU IP block onto a laptop or phone SoC probably won't be able to break that trend.
In the datacenter space, there are also tradeoffs that mean you can't make a one-size-fits-all chip that's optimal for both training and inference.
In the face of all the above complexity, the question of whether a GPU-like architecture retains any actual graphics-specific hardware is a silly question. NVIDIA and AMD have both demonstrated that they can easily delete that stuff from their architectures to get more TFLOPs for general compute workloads using the same amount of silicon.
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#129Earlier quoted context omitted.
I don't need it to beat the latest from nvidia, just be affordable, available, and have user servicable ram slots so "48gb" isn't such an ooo-ahh amount of memory I couldn't find any buy it now links but 512gb sticks don't seem to be fantasies, either: https://news.samsung.com/global/samsung-develops-industrys-f...
What would you want from on-gpu slots that you don't get from existing mechanisms to give your GPU access to system memory?
prelude: I realized that I typed out a ton of words but in the end engineering is all about tradeoffs, so, fine if there's a way I can teach some existing GPU, or some existing PCIe TPU, to access system RAM over an existing PCIe slot, that sounds like a fine step forward. I just don't have a lot of experience in that setup to know if only certain video cards allow that or what
Bearing in mind the aforementioned "I'm not a hardware guy," my mental model of any system RAM access for GPUs is:
1. copy weights from SSD to RAM
2. trigger GPU with that RAM location
3. GPU copies weights over PCIe bus to do calculation
4. GPU copies activations over PCIe bus back to some place in RAM
5. goto 3
Contrast this with memory socket(s) on the "GPU's mainboard" where it is, what, 3mm of trace wires away from ripping the data back and forth between its RAM and its processors, only choosing to PCIe the result out to RAM. It can have its own PCIe to speak to other sibling GPGPU setups for doing multi-device inference[3]
I would entertain people saying "but what a waste having 128GB of RAM only usable for GPGPU tasks" but if all these folks are right in claiming that it's the end of software engineering as we know it, I would guess it's not going to be that idle
1: I wish I had actually made a bigger deal out of wanting a GPGPU since for this purpose I don't care at all what DirectX or Vulkan whatever it runs
2: furthermore, if the "just use system RAM" was such a hot idea, I don't think it would be 2025 and we still have graphics cards with only 8GB of RAM on them. I'm not considering the Apple architecture because they already solder RAM and mark it up so much that normal people can't afford a sane system anyway, so I give no shits how awesome their unified architecture is
3: I also should have drew more attention to the inference need, since AIUI things like the TPUs I have on my desk aren't (able to do|good at) training jobs but that's where my expertise grinds to a halt because I have no idea why that is or how to fix it
Re: Top researchers leave Intel to build startup with 'the biggest, baddest CPU'
#130Earlier quoted context omitted.
Check out the Azure HBv5 servers. High bandwidth memory on-package with 352 AMD Zen 4 cores! With 7 TB/s memory bandwidth, it’s basically an x86 GPU. This is the future of high performance computing. It used to be available only for supercomputers but it’s trickling down to cloud VMs you can rent for reasonable money. Eventually it’ll be standard for workstations under your desk.
it's kind of concerning that it's only available as a hosted product. Not good news for anyone that needs to run on-prem for confidentiality or availability reasons.
I’m guessing the unit price is about $100K or higher, and it’s a niche product typically only used for HPC. If you’re a big enough customer, I’m sure you could buy these but I suspect the minimum order size is measured in the thousands of hosts.