Waterwave Could Quench AIs' Thirst for GPU Memory
spectrum.ieee.org
Waterwave Could Quench AIs' Thirst for GPU Memory
1–6 of 6 posts
Re: Waterwave Could Quench AIs' Thirst for GPU Memory
#2Unfortunately a reality vague decision of the technique. They say you can train multiple models at once but wouldn't this require N times more memory? They also say that models that require heavy compute won't parallelize well which also seems just a bit absurd because aren't all those shaders all firing off perfectly in parallel to do all the big matrix math?
Re: Waterwave Could Quench AIs' Thirst for GPU Memory
#3Anyone have a link to the source paper?
Re: Waterwave Could Quench AIs' Thirst for GPU Memory
#4Anyone have a link to the source paper?
It's paywalled and has no citations...
https://ieeexplore.ieee.org/document/10130297
https://www.computer.org/csdl/journal/tc/5555/01/10130297/1N...
Re: Waterwave Could Quench AIs' Thirst for GPU Memory
#5The person who wrote this article doesn't understand what the problem is, and how this paper doesn't address it at all.
The paper has a better way to share one GPU between small training jobs.
The real world problem is training huge models for a long time across many GPUs.
The problem the paper solves is one that is practically irrelevant aside from some niche cases.
Re: Waterwave Could Quench AIs' Thirst for GPU Memory
#6The person who wrote this article doesn't understand what the problem is, and how this paper doesn't address it at all. The paper has a better way to share one GPU between small training jobs. The real world problem is training huge models for a long time across many GPUs. The problem the paper solves is one that is practically irrelevant aside from some niche cases.
The problem to solve is GPU utilization rate. Training a “huge model” on n GPUs will have a cyclical pattern of idle GPUs and GPUs are expensive. This approach is likely scheduling GPU access across t training jobs to insure GPU is always in use. Maximizing GPU utility is a key requirement for training as a service.
The biggest issue is that the failure mode of this approach (one fault hoses the entire pipeline of T jobs) is exactly opposing the goals of its most likely users: Training SaaS providers.