91–100 of 199 posts
Earlier quoted context omitted.
won't it just be a few racks of gpus in one one the existing giant data centers?
Super computers are much larger than a few racks of GPU's.
It's not like Facebook had to go hollow out the Moon to make this.
I can't shake the feelings that a trillion or a quadrillion parameters won't solve the fundamental shortcomings of ML models not being models of artificial intelligence. I guess there's no way of knowing until we reach AGI, but I've never heard a compelling argument for why pure ML would get us there. GPT3 seems more like an argument against that hypothesis (in my view) than for it. Even the best, most expensive mode…
a). an internal feedback loop that evaluates a possible output without actuating it, and self-modifies the parameters if the possible output is not what it's needed
b). the capability (based on a) to model own behaviours without acting on them, and to model other agents behaviours and incorporate that model into the feedback
c). the ability to switch between modelling own behaviour and other agents behaviour intentionally by the model itself - as part of the feedback loop
i.e. what I feel it's totally missing in the self-driving cars today is the capability to model OTHER traffic participants actions and intentions; an experienced and attentive human driver does this all the time, pays attention to the pedestrians on the side if they want to jump in front of the car, pays attention to where other cars are LIKELY to go, pays attention to how the bicyclist that's currently overtaken may fall, even pays attention to random soccer balls flying out of a courtyard because a kid may be chasing that. I am not seeing any driving car trying to model any agent outside its own.
I can't shake the feelings that a trillion or a quadrillion parameters won't solve the fundamental shortcomings of ML models not being models of artificial intelligence. I guess there's no way of knowing until we reach AGI, but I've never heard a compelling argument for why pure ML would get us there. GPT3 seems more like an argument against that hypothesis (in my view) than for it. Even the best, most expensive mode…
> Meta’s AI supercomputer houses 6,080 Nvidia graphics-processing units ..... By mid-summer, when the AI Research SuperCluster is fully built, it will house some 16,000 GPUs Honestly ... this is lot of GPUs ... but is it the biggest...? > Model training is done with mixed precision on the NVIDIA DGX SuperPOD-based Selene supercomputer powered by 560 DGX A100 servers networked with HDR InfiniBand in a full fat tree co…
Earlier quoted context omitted.
I've always imagined AGI (perhaps naively) as being achieved by clever usage of ML, plus some utilization of classical/symbolic AI from pre-AI winter days, plus probably some unknown elements. For what it's worth, this is my view as well. And I don't think it's particularly naive. Plenty of people have researched and/or are researching aspects of how to do this. But how to combine something like a neural network, wit…
Even if they did replicate how the brain works our brains aren’t one of these networks trained for specific things it is millions, maybe billions, of them combined.
“The experiences we’re building for the metaverse require enormous compute power…and RSC will enable new AI models that can learn from trillions of examples, understand hundreds of languages, and more,” Meta CEO Mark Zuckerberg I don't really understand how AI processing is going to make the 'experiences' any better? This seems to me like investor fluff, saying they have some insane capability that other 'VR provider…
I think there are plenty of possibilities: - 3d worlds with style transfer on the textures, like maybe there's a cafe with the visual style of Starry Night or something - NPCs with conversation models that are finetuned for each NPC's personality and saves some history for each person it talks to for continuity - Game-playing AI on NPCs that make them go around doing actual things or playing minigames with players -…
Earlier quoted context omitted.
or at least ads will be 2% less irrelevant.
Nope, you’ll just get 10x as many ads with half the duration to minimize the amount of time your brain has to determine if something irrelevant or not. Those 5 second ads don’t cut short because they’re kind - it’s all they need to repeat to have the name, jingle, or sad-face burned into your neural net.
I can't shake the feelings that a trillion or a quadrillion parameters won't solve the fundamental shortcomings of ML models not being models of artificial intelligence. I guess there's no way of knowing until we reach AGI, but I've never heard a compelling argument for why pure ML would get us there. GPT3 seems more like an argument against that hypothesis (in my view) than for it. Even the best, most expensive mode…
How far can we actually take current machine learning technologies by scaling the underlying hardware? Are we going to see some AI algorithms that are 20% better or an order of magnitude better? And what will that realistically look like to an end user? This will have cost a lot of money and maybe the news alone will push stock prices and mean its paid for itself but is it actually going to result in a substantially…
It's linear for now (check GPT-2 vs GPT-3), but we're close to the point of diminishing returns.
> Meta’s AI supercomputer houses 6,080 Nvidia graphics-processing units ..... By mid-summer, when the AI Research SuperCluster is fully built, it will house some 16,000 GPUs Honestly ... this is lot of GPUs ... but is it the biggest...? > Model training is done with mixed precision on the NVIDIA DGX SuperPOD-based Selene supercomputer powered by 560 DGX A100 servers networked with HDR InfiniBand in a full fat tree co…
Honestly, this single GPU-based install is child's play compared to Google's multiple TPU exoflop supercomputers with hyper-cube optical interconnects. Google's ML setups allow synchronous weight update on thousand+ TPUs...