Earlier quoted context omitted.
won't it just be a few racks of gpus in one one the existing giant data centers?
Super computers are much larger than a few racks of GPU's.
Meta Unveils New AI Supercomputer
111–120 of 199 posts
Re: Meta Unveils New AI Supercomputer
#112Earlier quoted context omitted.
I feel that for there are three requirements for a NN-based AGI, inspired by biology: a). an internal feedback loop that evaluates a possible output without actuating it, and self-modifies the parameters if the possible output is not what it's needed b). the capability (based on a) to model own behaviours without acting on them, and to model other agents behaviours and incorporate that model into the feedback c). the…
Cruise actually consider both social dynamics and uncertainty (i.e. what can hide behind an obstacle, or where are pedestrians/bikes/cars likely to move to). If you are interested in self-driving cars, I can highly recommend their presentation from November 2021: https://youtu.be/uJWN0K26NxQ?t=1467 For me it felt more convincing than Tesla's (a few months prior); https://www.youtube.com/watch?v=j0z4FweCy4M
Re: Meta Unveils New AI Supercomputer
#113Anybody else get the sense that we're just totally frickin doomed? Even if Yudkowsky is off about AGI (which is a big maybe!) in what possible world will this technology be used to make our individual lives better (assuming you're not a FAIR researcher?)
It's just bunch of GPUs. It could be used for anything people can imagine, good or bad
Re: Meta Unveils New AI Supercomputer
#114Earlier quoted context omitted.
Those models are surprisingly tractable. You're nowhere near as interesting and unique as you might think you are at scale. Evidence: actual work experience at building latent representations to characterize customer behavior at FAANG. It's hard to come up with something that really gets you, but it's not hard to come up with something likely to make you spend more. You're surprisingly predictable on that axis and ev…
I wouldn't think it would take putting in any hours to be a crazy outlier - just a markedly different value system.
What people don't seem to grasp is that most of the supposed alternatives to mainstream are pretty mainstream too or we wouldn't have stores like Hot Topic in the first place.
Re: Meta Unveils New AI Supercomputer
#115I can't shake the feelings that a trillion or a quadrillion parameters won't solve the fundamental shortcomings of ML models not being models of artificial intelligence. I guess there's no way of knowing until we reach AGI, but I've never heard a compelling argument for why pure ML would get us there. GPT3 seems more like an argument against that hypothesis (in my view) than for it. Even the best, most expensive mode…
I feel that for there are three requirements for a NN-based AGI, inspired by biology: a). an internal feedback loop that evaluates a possible output without actuating it, and self-modifies the parameters if the possible output is not what it's needed b). the capability (based on a) to model own behaviours without acting on them, and to model other agents behaviours and incorporate that model into the feedback c). the…
The reasoning is that given enough training data the system would know the pedestrian is going to jump out or the cyclist is going to fall just based on sheer volume of training examples. It would have seen that scenario tons of times in the image data.
Whether that will actually work is the question though
Re: Meta Unveils New AI Supercomputer
#116What is the difference between “a supercomputer” and “a bunch of racks of computers”? The actual difference between the two is quite diminished compared to years past and seems to reduce more to how a collection of computers is used and not what it is.
> What is the difference between “a supercomputer” and “a bunch of racks of computers”? In addition to the other responses, I like pointing people to this talk[1] by Jeff Hammond for a comprehensive answer to this question (you can skip to the 11:15 timestamp). [1] https://uchicago.hosted.panopto.com/Panopto/Pages/Embed.aspx...
Re: Meta Unveils New AI Supercomputer
#117I can't shake the feelings that a trillion or a quadrillion parameters won't solve the fundamental shortcomings of ML models not being models of artificial intelligence. I guess there's no way of knowing until we reach AGI, but I've never heard a compelling argument for why pure ML would get us there. GPT3 seems more like an argument against that hypothesis (in my view) than for it. Even the best, most expensive mode…
Heuristically, we came to be by a very dumb process of piling up newer generations. If my pet would communicate with me on the level of GPTx, I would be very impressed. That's why nowadays I have some scepticism for the ANN critics' arguments, though think it would be neat if they were right.
The thing that I dislike the most in these discussions is the pervasiveness of the AGI concept and the assumption of a linear scale of intelligence. Again, I can intuitively say that I'm more intelligent than my pet: but to quantify this, we'd need to use something silly like brain size, or qualitative/arbitrary things like "this being can talk". I think that human intelligence is a somewhat random point in a very multi-dimensional space, one that technology may never even have a reason to visit. But people tend to subscribe to the notion that this is the very important "point where AGI happens".
Re: Meta Unveils New AI Supercomputer
#118My feeling is this is a PR push by Facebook. All tech companies keep touting AI, especially Google but also Microsoft, Apple and Amazon. In some sense I believe these business want to control how their own success is defined. That is, they are right now convincing everyone that tech dominance is equivalent to AI dominance which is equivalent to ML dominance. In some sense this is turning into a purity test, like "which tech company is the most AI focused". I expect this kind of PR to accelerate as each company tries to prove its AI bona-fides to the market.
Re: Meta Unveils New AI Supercomputer
#119How can they possibly keep the location of something like this a secret? There have to be thousands of people involved in building and maintaining it.
Re: Meta Unveils New AI Supercomputer
#120> Meta’s AI supercomputer houses 6,080 Nvidia graphics-processing units ..... By mid-summer, when the AI Research SuperCluster is fully built, it will house some 16,000 GPUs Honestly ... this is lot of GPUs ... but is it the biggest...? > Model training is done with mixed precision on the NVIDIA DGX SuperPOD-based Selene supercomputer powered by 560 DGX A100 servers networked with HDR InfiniBand in a full fat tree co…