AI Report #4: AutoGPT And Open-source lags behind Part 2

Has anyone actually done anything of use with autogpt? I tied it for 4 tasks and it would inevitably get stuck on each and produce absolutely nothing of value. These were fairly simple like research popular topics and write articles on them etc.

It would do things like Google something, the result wasn’t relevant , try again , got an error from one of the pages and then seemingly started to do something completely incoherent related to the error message

I've used GPT4, and never got it really do anything useful.

The agent loop is too frail, and too prone to lose focus, especially if you also want to combine it with some sort of persistent chat context. Hardcoding the control loop instead of letting the ai ficlgure it out was the only way tlive found to extract actual work out of gpt, and works,to an extent, with smaller models as well.

In my experience, AutoGPT is limited primarily by the poor state of its tools. For instance, browsing web pages often does not return relevant text that a human would pick out of the same page content. GPT-4 makes very good plans of what it should do, but the tools fail to give it what a human would receive.

For example, when asked to search for the top executives at company X, it rightly uses Google Search with the query “top executives at company X,” which returns a list of web pages such as the company’s About page. It then parses the About page but because of messed up page formatting, it returns nonsense data like the LinkedIn profile URL and some marketing material like a case study link, even though the executive profiles are right there.

The google search function is also limited. For comparison, SerpAPI masterfully scrapes Google Search using a proxy network and very intelligent parsing. In experiments using SerpAPI in combination with Microsoft’s guidance module, I got much farther than AutoGPT.

Fortunately, a lot of people are contributing to AutoGPT now and it is improving quickly. They are revamping the core right now and I expect it will work far better when they are done. With time, better tools will be made available to GPT-4 and progress should then be faster.

Can you help me understand autogpt? Is it just a recursive gpt where an initial prompt is given and has the ability of the output to be used to pipe to other gpt prompts. Am I missing something?

I tried doing more complex tasks using GPT4 and was initially optimistic about plugins but they have all been very disappointing.

For instance, a dream for me would be something like: "Find some rental property opportunities within 1 hour commute to New York City, that have a high rent to sale price and low taxes"

Broken down into steps it would be:

1. Find towns within 50 miles or so from within Manhattan. Take the top 100 or so by population

2. Find commute times for each one leaving at 9am monday and coming back at 6pm. Narrow down the cities to 1 hours. Unfortunately I didn't see any map plugins but maybe something like wolfram alpha can suffice or just google commute time for each town

3. Use zillow to pull typical rents and sale prices for each town. build a simple model (maybe wolfram) to model the rent and apply them to homes for sale including taxes. calculate median expected rent / median sale price

4. Remove towns that you don't have enough data on (not enough rentals or homes for sale) and return the top towns with a few examples of how much you can get

If I were building something like autogpt, I would start with an example like this and use this almost like an integration test. Theoretically all the pieces are there, but it just falls apart very quickly. I've heard these models can't yet do "planning" and i'm not sure what that means technically but I think this kind of problem requires planning so it might be a model limitation