Live data from Hacker News

ChatGPT agent: bridging research and action

openai.com

501–508 of 508 posts

Re: ChatGPT agent: bridging research and action

#504
post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

This reminds me of the story where Barclays had to buy bad assets from the Lehman bankruptcy because they only hid the rows of assets they did not want, but the receiver saw all the rows due to a mistake somewhere. The kind of 2% fault rate in Excel that could tank a big bank.

https://www.computerworld.com/article/1561181/excel-error-le...

Re: ChatGPT agent: bridging research and action

#505
post #469

Earlier quoted context omitted.

Do we even know what % of Waymo rides in SF are completely autonomous? I would not be surprised if more of them are remotely piloted than they've let on...

My understanding is they don't have the capability to have a ride be flat-out remotely piloted in real time. If the car gets stuck and puts its hazards on, a human can intervene, look at the 360 view from the cameras, and then give the car a simple high-level instruction like "turn left here" or "it's safe to proceed straight." But they can't directly drive the car continuously. And those moments where the car gives…

That's interesting to hear. It may be completely true, I don't really know. The source of my skepticism, however, is that all of the incentives are there for them to not be transparent about this, and to make the cars appear "smarter" than they really are.

Even if it's just a high level instruction set, it's possible that that occurs often enough to present scaling issues. It's also totally possible that it's not a problem, only time will tell.

What I have in mind is the Amazon stores, which were sold as being powered by AI, but were actually driven by a bunch of low-paid workers overseas watching cameras and manually entering what people were putting in their carts.

https://www.businessinsider.com/amazons-just-walk-out-actual...

Re: ChatGPT agent: bridging research and action

#506

Earlier quoted context omitted.

I opened up the app bundle of CC on macOS and CC is incredibly simple at its core! There’s about 14 tools (read, write, grep, bash, etc). The power is in the combination of the model, the tools and the system prompt/tool description prompts. It’s kind of mind blowing how well my cobbled together home brew version actually works. It doesn’t have the fancy CLI GUI but it is more or less performant as CC when running it…

Are you saying that you modified/added to the app bundle for CC?

Nope, I "borrowed" the system prompt and all the tool descriptions / tool schemas. I then re-implementing all the tools based on those descriptions and input schemas. I then used the OpenAI Agent SDK to give a model all the tools. Very conveniently the OpenAI Agent SDK supports different providers, namely the Vercel AI SDK ones, so it can be used with non-OpenAI models. The Vercel SDK has an Anthropic provider so that's how I used it to test.

This could have been somewhat trivial to implement from scratch too (not using the Agent/Vercel SDKs), but it sure made it convient! It's basically one run({model:'sonnet4', prompt: systemPrompt, tools: [tool1, tool2, tool3]}) call. Much nicer than implementing a loop and all that logic by hand.

Re: ChatGPT agent: bridging research and action

#507

Earlier quoted context omitted.

> Meanwhile an agent that you accept to get only 98% of things right is meeting expectations. Well yeah, because the agent is so much cheaper and faster than a human that you can eat the cost of the mistakes and everything that comes with them and still come out way ahead. No, of course that doesn't work in aircraft manufacturing or medicine or coding or many other scenarios that get tossed around on HN, but it does…

Definitely would work in coding. Most software companies can only dream of a 2% defect rate. Reality is probably closer to 98%, which is why we have so much organisational overhead around finding and fixing human error in software.

How does a software product with 98% defect rate look like? Even 2% seems like a lot. Like one in 50 interactions fail, or 1 in 50 data writes produce data corruption.

Re: ChatGPT agent: bridging research and action

#508

Earlier quoted context omitted.

> Humans don't even have a 99.99% success rate in breathing Will you please elaborate a little on this?

Humans cough or otherwise have to clear their airways about 1 in every 1,000 breaths, which is a 99.9% success rate.

Thank you for following up
Post reply on HN