Mark Zuckerberg’s new goal is creating artificial general intelligence

Post by gloosx » Fri, Jan 19, 2024, 10:14 AM UTC

Post by sublinear » Fri, Jan 19, 2024, 10:56 AM UTC

LudwigNagasena wrote: post #254

two_in_one wrote: post #152

Earlier quoted context omitted.

> why there's so much insistence from business that magic happens at scale with LLMs It's already happening. See latest Google layoffs. They are automating a lot of things. Most people don't realize it, but the change is going to be dramatic. > We're a long way from AGI This is a big question, what is AGI? LLMs are quite generic and 'intelligent'. Not human-like, but. Next is going to be incremental evolution. Till w…

> See latest Google layoffs. They are automating a lot of things. Most people don't realize it, but the change is going to be dramatic. What exactly did they automate?

Post by ethanbond » Fri, Jan 19, 2024, 1:11 PM UTC

stickfigure wrote: post #238

ethanbond wrote: post #226

Earlier quoted context omitted.

A human or group of humans deliberately operating millions of artificial intelligences (of average caliber).

So something like an army of robots? This seems like a nation-state kind of concern. I guess we could imagine small private armies, but I would imagine that governments will constrain those the same way we constrain more serious weapons. At any rate, we are a long way from having practical robots - let alone having to worry about an army of them.

Post by dwaltrip » Fri, Jan 19, 2024, 3:10 PM UTC

gardenhedge wrote: post #202

dwaltrip wrote: post #161

Earlier quoted context omitted.

LLM scaling laws are pretty well established at this point. They probably won’t hold forever but we aren’t at the breaking point yet. Some more pressing questions are: * What new capabilities emerge as the models get better and better at predicting (i.e. loss goes down)? * How much will it cost to train increasingly large models? And to run inference on them? * How difficult will it be to find or generate more and mo…

> LLM scaling laws are pretty well established at this point what are they then? I thought everyone was firmly in the "let's train with more data and see what happens" camp

Post by latchkey » Fri, Jan 19, 2024, 4:45 PM UTC

KeplerBoy wrote: post #318

latchkey wrote: post #205

Earlier quoted context omitted.

There certainly is a lot of value in the CUDA ecosystem, today. The problem is that when all the big companies are buying up hundreds of thousands of GPUs, that doesn't leave much for anyone else. Sane business people will look to decentralize their compute over time and not be reliant on a single provider. AMD will be able to take advantage of that and they've already stated that is their focus going forward. ROCm/H…

That's a big issue in AMD land imho. Everyone can pickup a 200$ GPU (talking about the RTX 3050) which will behave like a scaled down A100 and get started playing around with CUDA. You can't really do that with AMD GPUs, their cheapest officially supported GPU is the 7900 XTX and that has a different architecture than the data center ones.