The von Neumann bottleneck is impeding AI computing?
research.ibm.com
The von Neumann bottleneck is impeding AI computing?
1–10 of 34 posts
Re: The von Neumann bottleneck is impeding AI computing?
#2The old saw from corporations that want to sell you an locked-down alternative to general-purpose computing -- now for "AI"
Re: The von Neumann bottleneck is impeding AI computing?
#3Nit,
ARM processors primarily use a modified Harvard architecture, including the raspberry pi pico.
Re: The von Neumann bottleneck is impeding AI computing?
#4Actual result: "This new process promises to increase the number of optical fibers that can be connected at the edge of a chip, a measure known as beachfront density, by six times."
Faster interconnects are always nice, but this is more like routine improvement.
Re: The von Neumann bottleneck is impeding AI computing?
#5Nit, ARM processors primarily use a modified Harvard architecture, including the raspberry pi pico.
this isn't about Harvard/VonNeuman split/no-split between i-cache and d-cache
I think this post is more about... compute in memory? if I got it right?
Re: The von Neumann bottleneck is impeding AI computing?
#6Why they don't use AI to create a new architecture?
Re: The von Neumann bottleneck is impeding AI computing?
#7Why they don't use AI to create a new architecture?
https://github.com/GAIR-NLP/ASI-Arch
This is being done, with great results so far. As models get better, architecture search and creation and refinment improves, driving a reinforcement loop. At some point in the near future the big labs will likely start seeing significant returns from methods like this, translating into better and faster AI for consumers.
Re: The von Neumann bottleneck is impeding AI computing?
#8Nit, ARM processors primarily use a modified Harvard architecture, including the raspberry pi pico.
Nit: RP2040 is a Von Neumann. There's only one AHB port on the m0.
Edit: see also ARM7TDMI, Cortex-m0/0+/1, and probably a few others. All the big stuff is modified Harvard or very rarely pure Harvard.
Re: The von Neumann bottleneck is impeding AI computing?
#9Actual result: "This new process promises to increase the number of optical fibers that can be connected at the edge of a chip, a measure known as beachfront density, by six times." Faster interconnects are always nice, but this is more like routine improvement.
"In recent inference tests run on a 3-billion-parameter LLM developed from IBM’s Granite-8B-Code-Base model, NorthPole was 47 times faster than the next most energy-efficient GPU and was 73 times more energy efficient than the next lowest latency GPU."
It's also fascinating that they are experimenting with analog memory because it pairs so well with model weights
Re: The von Neumann bottleneck is impeding AI computing?
#10Why they don't use AI to create a new architecture?
Do you want SkyNet? That's how you get SkyNet.