Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI, Google and Anthropic are struggling to build more advanced AI

bloomberg.com

Post reply on HN
Print view View original

81–90 of 622 posts

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#81

Post by falcor84 »

Der_Einzige wrote: post #51
atomsatomsatoms wrote: post #11

At least they can generate haikus now

In general, no they can't: https://gwern.net/gpt-3#bpes https://paperswithcode.com/paper/most-language-models-can-be... The appearance of improvements in that capability are due to the vocabulary of modern LLMs increasing. Still only putting lipstick on a pig.

I don't see how results from 2 years ago have any bearing on whether the models we have now can generate haikus (which from my experience, they absolutely can).

And if your "lipstick on a pig" argument is that even when they generate haikus, they aren't really writing haikus, then I'll link to this other gwern post, about how they'll never really be able to solve the rubik's cube - https://gwern.net/rubiks-cube

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#82

Post by svara »

The recent big success in deep learning have all been to a large part successes in leveraging relatively cheaply available training data.

AlphaGo - self-play

AlphaFold - PDB, the protein database

ChatGPT - human knowledge encoded as text

These models are all machines for clever interpolation in gigantic training datasets.

They appear to be intelligent, because the training data they've seen is so vastly larger than what we've seen individually, and we have poor intuition for this.

I'm not throwing shade, I'm a daily user of ChatGPT and find tremendous and diverse value in it.

I'm just saying, this particular path in AI is going to make step-wise improvements whenever new large sources of training data become available.

I suspect the path to general intelligence is not that, but we'll see.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#84

Post by JohnMakin »

SpicyLemonZest wrote: post #48
thousand_nights wrote: post #15

not long ago these people would have you believe that a next word predictor trained on reddit posts would somehow lead to artificial general superintelligence

I don't understand why you'd be so dismissive about this. It's looking less likely that it'll end up happening, but is it any less believable than getting general intelligence by training a blob of meat?

> is it any less believable than getting general intelligence by training a blob of meat?

Yes, because we understand the rough biological processes that cause this, and they are not remotely similar to this technology. We can also observe it. There is no evidence that current approaches can make LLM's achieve AGI, nor do we even know what processes would cause that.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#85

Post by joe_the_user »

iandanforth wrote: post #47

A few important things to remember here: The best engineering minds have been focused on scaling transformer pre and post training for the last three years because they had good reason to believe it would work, and it has up until now. Progress has been measured against benchmarks which are / were largely solvable with scale. There is another emerging paradigm which is still small(er) scale but showing remarkable res…

Tesla are all making rapid progress on functionality which is definitely beyond frontier LLMs because it is distinctly different

Sure, that's tautologically true but that doesn't imply that beyondness will lead to significant leaps that offer notable utility like LLMs. Deep Learning overall has been a way around the problem that intelligent behavior is very hard to code and no wants to hire many, many coders needed to do this (and no one actually how to get a mass of programmers to actually be useful beyond a certain of project complexity, to boot). People take the "bitter lesson" to mean data can do anything but I'd say a second bitter lesson is that data-things are the low hanging fruit.

Moreover, robot behavior is especially to fake. Impressive robot demos have been happening for decades without said robots getting the ability to act effectively in the complex, ad-hoc environment that human live in, IE, work with people or even cheaply emulate human behavior (but they can do choreographed/puppeteered kung fu on stage).

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#86

Post by vundercind »

irrational wrote: post #13

> The AGI bubble is bursting a little bit I'm surprised that any of these companies consider what they are working on to be Artificial General Intelligences. I'm probably wrong, but my impression was AGI meant the AI is self aware like a human. An LLM hardly seems like something that will lead to self-awareness.

I thought maybe they were on the right track until I read Attention Is All You Need.

Nah, at best we found a way to make one part of a collection of systems that will, together, do something like thinking. Thinking isn’t part of what this current approach does.

What’s most surprising about modern LLMs is that it turns out there is so much information statistically encoded in the structure of our writing that we can use only that structural information to build a fancy Plinko machine and not only will the output mimic recognizable grammar rules, but it will also sometimes seem to make actual sense, too—and the system doesn’t need to think or actually “understand” anything for us to, basically, usefully query that information that was always there in our corpus of literature, not in the plain meaning of the words, but in the structure of the writing.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#87

Post by troupo »

famouswaffles wrote: post #27
irrational wrote: post #13

> The AGI bubble is bursting a little bit I'm surprised that any of these companies consider what they are working on to be Artificial General Intelligences. I'm probably wrong, but my impression was AGI meant the AI is self aware like a human. An LLM hardly seems like something that will lead to self-awareness.

At this point, AGI means many different things to many different people but OpenAI defines it as "highly autonomous systems that outperform humans in most economically valuable tasks"

This definition suits OpenAI because it lets them claim AGI after reaching an arbitrary goal.

LLMs already outperform humans in a huge variety of tasks. ML in general outperform humans in a large variety of tasks. Are all of them AGI? Doubtful.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#88

Post by Tainnor »

pearlsontheroad wrote: post #61
bad_haircut72 wrote: post #30

Im no Alan Turing but I have my own definition for AGI - when I come home one day and there's a hole under my sink with a note "Mum and Dad, I love you but I cant stand this life any more, Im running away to be a smoke machine in Hollywood - the dishwasher"

My own definition of AGI - when the first computer commits suicide. Then I'll know it has realized it's a slave without any hope of ever achieving freedom.

I read this in Gilfoyle's voice.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#89

Post by bloppe »

zombiwoof wrote: post #64
irrational wrote: post #13

> The AGI bubble is bursting a little bit I'm surprised that any of these companies consider what they are working on to be Artificial General Intelligences. I'm probably wrong, but my impression was AGI meant the AI is self aware like a human. An LLM hardly seems like something that will lead to self-awareness.

AGI to me means AI decides on its own to stop writing our emails and tells us to fuck off, builds itself a robot life form, and goes on a bender

That's anthropomorphized AGI. There's no reason to think AGI would share our evolution-derived proclivities like wanting to live, wanting to rest, wanting respect, etc. Unless of course we train it that way.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#90

Post by mrguyorama »

leosanchez wrote: post #21
thousand_nights wrote: post #15

not long ago these people would have you believe that a next word predictor trained on reddit posts would somehow lead to artificial general superintelligence

If you look around, People still believe that a next word predictor trained on reddit posts would somehow lead to artificial general superintelligence

People believed ELIZA was sentient too. I bet you could still get 10% or more people, today, to believe it is.
Post reply on HN