Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Report #4: AutoGPT And Open-source lags behind Part 2

theaireport.substack.com

Post reply on HN
Print view View original

31–37 of 37 posts

Re: AI Report #4: AutoGPT And Open-source lags behind Part 2

#31

Post by coffeebeqn »

furyofantares wrote: post #20
coffeebeqn wrote: post #2

Has anyone actually done anything of use with autogpt? I tied it for 4 tasks and it would inevitably get stuck on each and produce absolutely nothing of value. These were fairly simple like research popular topics and write articles on them etc. It would do things like Google something, the result wasn’t relevant , try again , got an error from one of the pages and then seemingly started to do something completely in…

It is likely we can do better than 1:1 human input to GPT output on current tech; but the human in the loop is doing a lot of work very easily that the LLM is very bad at, just like the LLM is doing a lot of work very easily that is otherwise laborious for the human. We can't just take the things that the LLM is bad at and humans do easily and expect to fix it with more LLM. Right now we have: Step 1: Human reasoning…

Right - programming is how you get computers to do things. AI isn’t magic

Re: AI Report #4: AutoGPT And Open-source lags behind Part 2

#32

Post by joshuanapoli »

itsuka wrote: post #28

I have never built an agent before, nor am I knowledgeable about the latest studies in this field. So what I am saying below is likely to be nonsensical. I was thinking that perhaps we have been working with abstractions that are too low-level. Instead of providing a set of tools such as API calls or text splitters, wouldn't it be more reliable to give agents templates or workflows of successful tasks, such as trimmi…

In my experience, a GPT 3.5 or 4 agent has trouble accomplishing anything if you make too many APIs available. Using a completion to narrow down the list of options makes the entire exchange very slow. There is also a compounding chance of failure with multi-stage strategies; the “agent” may get stuck responding in the “wrong” way and burn up even more time with error recovery.

Re: AI Report #4: AutoGPT And Open-source lags behind Part 2

#33

Post by dinvlad »

furyofantares wrote: post #20
coffeebeqn wrote: post #2

Has anyone actually done anything of use with autogpt? I tied it for 4 tasks and it would inevitably get stuck on each and produce absolutely nothing of value. These were fairly simple like research popular topics and write articles on them etc. It would do things like Google something, the result wasn’t relevant , try again , got an error from one of the pages and then seemingly started to do something completely in…

It is likely we can do better than 1:1 human input to GPT output on current tech; but the human in the loop is doing a lot of work very easily that the LLM is very bad at, just like the LLM is doing a lot of work very easily that is otherwise laborious for the human. We can't just take the things that the LLM is bad at and humans do easily and expect to fix it with more LLM. Right now we have: Step 1: Human reasoning…

Very well put, thanks for laying it out so clearly.

Re: AI Report #4: AutoGPT And Open-source lags behind Part 2

#34

Post by behnamoh »

ttul wrote: post #8
coffeebeqn wrote: post #2

Has anyone actually done anything of use with autogpt? I tied it for 4 tasks and it would inevitably get stuck on each and produce absolutely nothing of value. These were fairly simple like research popular topics and write articles on them etc. It would do things like Google something, the result wasn’t relevant , try again , got an error from one of the pages and then seemingly started to do something completely in…

In my experience, AutoGPT is limited primarily by the poor state of its tools. For instance, browsing web pages often does not return relevant text that a human would pick out of the same page content. GPT-4 makes very good plans of what it should do, but the tools fail to give it what a human would receive. For example, when asked to search for the top executives at company X, it rightly uses Google Search with the…

> In experiments using SerpAPI in combination with Microsoft’s guidance module, I got much farther than AutoGPT.

would love to see how you implemented this with guidance. Did you use GPT4?

Re: AI Report #4: AutoGPT And Open-source lags behind Part 2

#35

Post by ryanklee »

fullstackchris wrote: post #27

To all the AI hucksters - there was a decades long effort already tried, known as "5th generation programming languages" where they assumed the next level after high level programming languages would be to remove the human programmer completely. basically, this effort ended up failing because, well, problem solving itself is inherently complex. https://en.wikipedia.org/wiki/Fifth-generation_programming_l... seems lik…

> seems like the exact same thing happened with ChatGPT/ AutoGPT/ GPT4, and this will keep happening.

A bit early to call it, by far.

Serious development around these capabilities have only just gotten off the ground.

It also doesn't seem like LLMs are done improving.

Re: AI Report #4: AutoGPT And Open-source lags behind Part 2

#36

Post by olup »

Feels like this newsletter is actually written by an llm. It's full of repetition and very flat assertion stitched together with high intensity random connectors. "Make no mistake [something incredibly common and already expressed here]", "Our own take on this is [another super cliche platitude here]".

Re: AI Report #4: AutoGPT And Open-source lags behind Part 2

#37

Post by ilyazub »

ttul wrote: post #8
coffeebeqn wrote: post #2

Has anyone actually done anything of use with autogpt? I tied it for 4 tasks and it would inevitably get stuck on each and produce absolutely nothing of value. These were fairly simple like research popular topics and write articles on them etc. It would do things like Google something, the result wasn’t relevant , try again , got an error from one of the pages and then seemingly started to do something completely in…

In my experience, AutoGPT is limited primarily by the poor state of its tools. For instance, browsing web pages often does not return relevant text that a human would pick out of the same page content. GPT-4 makes very good plans of what it should do, but the tools fail to give it what a human would receive. For example, when asked to search for the top executives at company X, it rightly uses Google Search with the…

> The google search function is also limited. For comparison, SerpAPI masterfully scrapes Google Search using a proxy network and very intelligent parsing. In experiments using SerpAPI in combination with Microsoft’s guidance module, I got much farther than AutoGPT.

Thanks for your kind words. We are working on SerpApi integration for Auto-GPT: https://github.com/serpapi/public-roadmap/issues/905

Post reply on HN