61–70 of 199 posts
What is the difference between “a supercomputer” and “a bunch of racks of computers”? The actual difference between the two is quite diminished compared to years past and seems to reduce more to how a collection of computers is used and not what it is.
I've always imagined AGI (perhaps naively) as being achieved by clever usage of ML, plus some utilization of classical/symbolic AI from pre-AI winter days, plus probably some unknown elements.
Earlier quoted context omitted.
Good luck building special GPUs in just two years, especially with what's happening regarding chip production right now. Not sure how they could have achieved a project of this size/scope unless they use off-the-shelf components, since the backlogs are so long and have been for some time now.
Facebook has been hiring FPGA engineers with ML experience since 2018 so I don't think this would be out of the question! But even so, Nvidia sell custom GPUs that aren't the same ones for gaming.
> But even so, Nvidia sell custom GPUs that aren't the same ones for gaming.
Interesting. Gonna be fun to observe the outrage (from distance) about how GPUs are not only used to destroy the environment for cryptocurrency profits, but now Facebook will also contribute to the world destruction for ad-money.
“The experiences we’re building for the metaverse require enormous compute power…and RSC will enable new AI models that can learn from trillions of examples, understand hundreds of languages, and more,” Meta CEO Mark Zuckerberg I don't really understand how AI processing is going to make the 'experiences' any better? This seems to me like investor fluff, saying they have some insane capability that other 'VR provider…
> understand
I know this is CEO-talk, but I sometimes wonder if these pricks really think they are inventing AI.
Earlier quoted context omitted.
Yes, I think we're in trouble. Walked over to friends' house over the weekend. It was a very nice, sunny weekend. Saw them through the window with their goggles on, rang the doorbell. They came to the door and I said, look how nice and sunny it it, we should go for a walk and they were like "oh, we had no idea it was so nice out because we were in Meta with our goggles on".
I don't see how this is a bad thing. What's wrong with preferring the metaverse to their sunny neighborhood? Honestly the matrix seems like the end game here and I'm all for it. Then it can be a sunny day in the neighborhood every day. The doom comes from the matrix being built by people who either don't have our best interests in mind, or aren't competent enough to build the system to correctly act in our best inter…
You mean like Zuckerberg and FB?
How far can we actually take current machine learning technologies by scaling the underlying hardware? Are we going to see some AI algorithms that are 20% better or an order of magnitude better? And what will that realistically look like to an end user? This will have cost a lot of money and maybe the news alone will push stock prices and mean its paid for itself but is it actually going to result in a substantially…
Could this have been the reason for chip shortage?
I used to work at a university where my professor had been in automatic speech recognition for a long time, but basically gave up on that line of research about 10 years ago because he figured that universities simply cannot compete budget wise with the big industry players. I suppose the same will soon be true for most ML-related areas of research sooner or later, at least as far as applied ML is concerned. Already,…
In the NLP space there's been a lot of work recently around reducing model sizes, since they've started to reach the point where model weights sometimes don't fit in the memory of most GPUs.
There's also projects like MarianNMT which completely abandon Python and write heavily optimized models with fast languages that can run quickly and accurately even without GPUs. I think we'll see a lot more of this, though of course there's a pretty big barrier in the sheer rarity of being good at both deep learning research and writing optimized low-level code.