Nvidia to Acquire Arm for $40B

Thankfully those patents are about to expire.

By late 2000s, instruction scheduling research was largely considered done and dusted, with papers like:

https://dl.acm.org/doi/book/10.5555/923366 https://dl.acm.org/doi/10.1145/349299.349318

and many, many others (it produced so many PhDs in 90s). And, needless to say, HP and Intel hired so many excellent researchers during the heydays of Itanium. So I don't know on what basis you think there wasn't enough investment. So I have no choice but to assume you're ignorant of the actual history here, both in academics and industry.

It turns out instruction scheduling can not overcome the challenge of variable memory and cache latency, and branch prediction, because all of those are dynamic and unpredictable, for "integer" application (i.e. bulk of the code running on the CPUs of your laptop and cell phones). And, predication, which was one of the "solutions" to overcome branch misprediction penalties, turns out to be not very efficient, and is limited in its application.

For integer applications, it turns out the instruction level parallelism isn't really the issue. It's about how to generate and maintain as many outstanding cache misses at a time. VLIW turns out to be insufficient and inefficient for that. Some minor attempts are addressing that through prefetches and more elaborate markings around load/store all failed to give good results.

For HPC type workload, it turns out data parallelism and thread-level parallelism are much more efficient way to improve the performance, and also makes ILP on a single instruction stream play only a very minor role - GPUs and ML accelerators demonstrate this very clearly.

As for the security and the speculative execution, speculative execution is not going anywhere. Naturally, there are many researches around this like:

https://ieeexplore.ieee.org/abstract/document/9138997 https://dl.acm.org/doi/abs/10.1145/3352460.3358306

and while it will take a while before the real pipeline implements ideas like above thus we may continue to see some smaller and smaller vulnerabilities as the industry collectively plays whack-a-mole game, I don't see a world where the top of the line general-purpose microprocessor giving up on speculative execution, as the performance gain is simply too big.

I have yet to meet any academics or industry processor architects or compiler engineer who think VLIW / Itanium is the way to move forward.

This is not to say putting as much work to the compiler is a bad idea, as nVidia has demonstrated. But what they are doing is not VLIW.

This is horrible for Arm

Looking at the conversations almost 24 hours after posting the IP, licensing, ecosystem, political, and overall business aspects of this have been discussed to death. Oddly for Hacker News there has been little discussion of the potential technical aspects of this acquisition.

Pure speculation (of course)...

To me (from a tech standpoint) this acquisition centers around three things we already know about Nvidia:

- Nvidia is pushing to own anything and everything GPGPU/TPU related, from cloud/datacenter to edge. Nvidia has been an ARM licensee for years with their Jetson line of hardware for edge GPGPU applications:

https://developer.nvidia.com/buy-jetson

Looking at the architecture of these devices (broadly speaking) Nvidia is combining an ARM CPU with their current gen GPU hardware (complete with Tensor Cores, etc). What's often left out of this mention is that they utilize a shared memory architecture where the ARM CPU and CUDA cores share memory. Not only does this cut down on hardware costs and power usage, it increases performance.

- Nvidia has acquired Mellanox for high performance network I/O across various technologies (Ethernet and Infiniband). Nvidia is also actively working to be able to remove the host CPU from as many GPGPU tasks as possible (network I/O and data storage):

https://developer.nvidia.com/gpudirect

- Nvidia already has publicly available software in place to effectively make their CUDA compute available over the network using various APIs:

https://github.com/triton-inference-server/server

Going on just the name Triton is currently only available for inference but it provides the ability to not only directly serve GPGPU resources via network API at scale but ALSO accelerate various models with TensorRT optimization:

https://docs.nvidia.com/deeplearning/triton-inference-server...

Given these points I think this is an obvious move for Nvidia. TDP and performance is increasingly important across all of their target markets. They already have something in place for edge inference tasks powered by ARM with Jetson but looking at ARM core CPU benchmarks it's sub-optimal. Why continue to pay ARM licensing fees when you can buy the company, collect licensing fees, get talent, and (presumably) drastically improve performance and TDP for your edge GPGPU hardware?

In the cloud/datacenter, why continue to give up watts in terms of TDP and performance to sub-optimal Intel/AMP/x86_64 CPUs and their required baggage (motherboard bridges, buses, system RAM, etc) when all you really want to do is shuffle data between your GPUs, network, and storage as quickly and efficiently as possible?

Of course many applications will still require a somewhat general purpose CPU for various tasks, customer code, etc. AWS already has their own optimized ARM cores in place. aarch64 is more and more becoming a first class citizen across the entire open source ecosystem.

As platform and software as a service continues to eat the world cloud providers likely have already started migrating the underlying hardware powering these various services to ARM cores for improved performance and TDP (same product, more margin).

Various ARM cores are already showing to be quite capable for most CPU tasks but given the other architectural components in place here even the lowliest of modern ARM cores is likely to be asleep most of the time for the applications Nvidia currently cares about. Giving up licensing, die space, power, tighter integration, etc to x86_64 just seems to be foolish at this point.

Meanwhile (of course) if you still need x86_64 (or any other arch) for whatever reason you can hit a network API powered by hardware using Nvidia/Mellanox I/O, GPU, and ARM. Potentially (eventually) completely transparently using standard CUDA libraries and existing frameworks (see work like Apex):

https://github.com/NVIDIA/apex

I, for one, am excited to see what comes from this.

I would be quite surprised if LLVM ever lost the ability to compile C for the platform.