Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Auto-GPT: An Autonomous GPT-4 Experiment

github.com

Post reply on HN
Print view View original

161–170 of 178 posts

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#161

Post by roflyear »

ALittleLight wrote: post #137
roflyear wrote: post #132

Earlier quoted context omitted.

It is not clear LLMs have a "general problem solving capability" at all. That's the entire point. That's a high bar!

What do you call being able to play chess and play any other well known game and do well on a battery of standardized tests and write code in a variety of languages in a variety of problems and ask questions and write fiction prose or poetry and generally just take a shot at anything you happen to ask. I just can't take the idea that there is ambiguity as to whether these things have general problem solving skills se…

Well, I dunno. Similar to Stockfish, wolfram alpha, etc.. I suppose! (tho seems it's much worse at specific problems than these tools are at those problems).

I'm not saying it isn't impressive! Just that it very much seems to be really good at finding out what text should come next. I don't think that's general problem solving!

Giving it a SQL schema and getting valid queries out of it is super impressive, but I have no idea what it was trained on.

> I just can't take the idea that there is ambiguity as to whether these things have general problem solving skills seriously. They obviously do.

It is not obvious to me this is the case! Often I will get totally wrong answers, and I won't be able to get the correct answer out of it no matter how hard I try.

> what's something you would be able to say or do that an unrestricted ChatGPT wouldn't?

Well, I'd ask you clarifying questions, for one! GPT doesn't do this type of stuff without being forced to, and even then it fails at it.

Also if you asked me to do something like "replace the word 'a' with the word 'eleven' in all your replies to me" I won't do weird garbage stuff, like reply with:

"ok11y I will repl11ce all words with the word eleven when using the letter 'a'"

lol

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#162

Post by roflyear »

tanseydavid wrote: post #145
mjr00 wrote: post #56

Earlier quoted context omitted.

> Language models obviously have some form of intelligence right now. This is not "obvious" in any sense of the word. At best, it's highly debatable.

>> This is not "obvious" in any sense of the word. At best, it's highly debatable. Does a dog or cat have intelligence? If you answered no, then I would ask if you don't you believe that by some measure a dog or cat has more intelligence than a rock? And as a follow-on I would ask if you think GPT demonstrates more intelligence than a dog or a cat. But perhaps you believe that in every one of these examples there is…

It probably doesn't have more intelligence than a dog or a cat...

Just like chatbots 20 years ago didn't, even though they could talk, too.

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#163

Post by quonn »

Super_Jambo wrote: post #59
quonn wrote: post #34

Earlier quoted context omitted.

But not in a way that‘s more problematic than human-to-human communication.

quantity has a quality all of it's own.

But it's not. The OP said "us". So it's passing through a human action. So it's not faster than before.

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#164

Post by roflyear »

ALittleLight wrote: post #144
ModernMech wrote: post #139

Earlier quoted context omitted.

I would be able to make a long list of things while maintaining logical consistency with things earlier the list. For instance, I asked ChatGPT-4 to create a schedule for a class, and it started off okay, but by the time it got to the end of the schedule, it started listing topics already covered. Really shows how it's just going off of statistics.

This is an example of ChatGPT performing poorly, but not being unable to do the thing. Nobody would say would say ChatGPT has human level intelligence across all domains - but that it has general problem solving ability. In other words, I'm saying it has an IQ, not that it has the highest possible IQ. And, of course, there are domains where ChatGPT will do better than you. Since I don't know your skill set I don't kn…

You're just moving the goalposts.

GPT being bad this way, and being bad at "substitute words in all your responses" means it is leaking the abstraction to us. It's because of how its built and how it works. It means it isn't a general problem solving thing: it's a text prediction thing.

GPT is super impressive, I don't know how many times I need to say that, but it isn't intelligent, it doesn't understand the problem, and it doesn't seem like it ever will get there.

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#165

Post by roflyear »

tanseydavid wrote: post #146
roflyear wrote: post #133

Earlier quoted context omitted.

Is stockfish intelligent?

>> Is stockfish intelligent? It isn't general intelligence but I would argue that it is more intelligent than a new-born human being.

I think it's hard to define intelligence, and I wouldn't say (generally) that computer programs are intelligent.

If a building was on fire and you had to save a running instance of stockfish or a newborn, you'd probably pick the newborn.

But! If you do say stockfish is intelligent, sure! GPT is too!

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#166

Post by ALittleLight »

roflyear wrote: post #164
ALittleLight wrote: post #144

Earlier quoted context omitted.

This is an example of ChatGPT performing poorly, but not being unable to do the thing. Nobody would say would say ChatGPT has human level intelligence across all domains - but that it has general problem solving ability. In other words, I'm saying it has an IQ, not that it has the highest possible IQ. And, of course, there are domains where ChatGPT will do better than you. Since I don't know your skill set I don't kn…

You're just moving the goalposts. GPT being bad this way, and being bad at "substitute words in all your responses" means it is leaking the abstraction to us. It's because of how its built and how it works. It means it isn't a general problem solving thing: it's a text prediction thing. GPT is super impressive, I don't know how many times I need to say that, but it isn't intelligent, it doesn't understand the problem…

That's not moving the goalposts - it's exactly what I've said throughout this thread. GPT is better, worse, and within human ranges at different tasks - but it can do a wide range of tasks.

That GPT can solve a wide variety of problems, including problems it's never seen before, is literally the definition of intelligence and pointing out results where it underperformed is not even attempting to rebut that.

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#167

Post by roflyear »

ALittleLight wrote: post #166
roflyear wrote: post #164

Earlier quoted context omitted.

You're just moving the goalposts. GPT being bad this way, and being bad at "substitute words in all your responses" means it is leaking the abstraction to us. It's because of how its built and how it works. It means it isn't a general problem solving thing: it's a text prediction thing. GPT is super impressive, I don't know how many times I need to say that, but it isn't intelligent, it doesn't understand the problem…

That's not moving the goalposts - it's exactly what I've said throughout this thread. GPT is better, worse, and within human ranges at different tasks - but it can do a wide range of tasks. That GPT can solve a wide variety of problems, including problems it's never seen before, is literally the definition of intelligence and pointing out results where it underperformed is not even attempting to rebut that.

Sure, I would agree with that. I do not agree that it is doing anything more than predicting text. But it does it really well!

> including problems it's never seen before

Can you demonstrate this?

> is literally the definition of intelligence

I wish it was this easy! Unfortunately, it is not. GPT says the definition of intelligence is:

Intelligence is a complex and multifaceted concept that is difficult to define precisely. Broadly speaking, intelligence refers to the ability to learn, understand, reason, plan, solve problems, think abstractly, comprehend complex ideas, adapt to new situations, and learn from experience. It encompasses a range of cognitive abilities, including verbal and spatial reasoning, memory, perception, and creativity. However, there is ongoing debate among researchers and scholars about the nature of intelligence and how to measure it, and no single definition or theory of intelligence has gained widespread acceptance.

Which, is pretty good!

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#168

Post by visarga »

tanseydavid wrote: post #143
visarga wrote: post #28

Earlier quoted context omitted.

Needs a big honking datacenter or billions of compute credits and safety for 6-12 months.

Doesn't Alpaca seem to suggest that assumption is no longer true?

Alpaca rides on LLaMA. And LLaMA was trained on 1T tokens for a long time. The fine-tuning takes one hour with low rank adaptation. But pre-training a new model takes months.

Re: Auto-GPT: An Autonomous GPT-4 Experiment

#170

Post by larsonnn »

The demo already shows the problem. It picks an event, the AI picks Earth Day. Then it makes a recipe with an avocado in it.

And writes it is good for the planet. Avocados are exactly the opposite. They have an extremely high water consumption and then have to be imported from all over the world.

Contrary to the author who claims, "Auto-GPT pushes the boundaries of what is possible with AI." I don't find that.

Why can't you just use GPT-4 as it is. It is an insanely tool to simplify many things. But it's still a long way from being ready, and it's not meant to decide anything on its own. And even to reflect reasonably out of own motivation.

Post reply on HN