Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT agent: bridging research and action

openai.com

Post reply on HN
Print view View original

361–370 of 508 posts

Re: ChatGPT agent: bridging research and action

#361

Post by j_timberlake »

travelalberta wrote: post #59
twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

I think this is my favorite part of the LLM hype train: the butterfly effect of dependence on an undependable stochastic system propagates errors up the chain until the whole system is worthless. "I think it got 98% of the information correct..." how do you know how much is correct without doing the whole thing properly yourself? The two options are: - Do the whole thing yourself to validate - Skim 40% of it, 'seems…

"a human making those mistakes again and again would get fired"

You must be really desperate for anti-AI arguments if this is the one you're going with. Employees make mistakes all day every day and they don't get fired. Companies don't give a shit as long as the cost of the mistakes is less than the cost of hiring someone new.

Re: ChatGPT agent: bridging research and action

#362

Post by mat_b »

enneff wrote: post #308
lobochrome wrote: post #305

Earlier quoted context omitted.

“The quip about 98% correct should be a red flag for anyone familiar with spreadsheets” I disagree. Receiving a spreadsheet from a junior means I need to check it. If this gives me infinite additional juniors I’m good. It’s this popular pattern of HN comments - expect AI to behave deterministically correct - while the whole world operates on stochastically correct all the time…

In my experience the value of junior contributors is that they will one day become senior contributors. Their work as juniors tends to require so much oversight and coaching from seniors that they are a net negative on forward progress in the short term, but the payoff is huge in the long term.

I don't see how this can be true when no one stays at a single job long enough for this to play out. You would simply be training junior employees to become senior employees for someone else.

Re: ChatGPT agent: bridging research and action

#363

Post by Metricon »

polytely wrote: post #88
tootyskooty wrote: post #78

Earlier quoted context omitted.

Honestly might be more indicative of how far behind vision is than anything. Despite the fact that CV was the first real deep learning breakthrough VLMs have been really disappointing. I'm guessing it's in part due to basic interleaved web text+image next token prediction being a weak signal to develop good image reasoning.

Is there anyone trying to solve OCR, I often think of that annas-archive blog about how we basically just have to keep shadow libraries alive long enough until the conversion from pdf to plaintext is solved. https://annas-archive.org/blog/critical-window.html I hope one of these days one of these incredibly rich LLM companies accidentally solves this or something, would be infinitely more beneficial to mankind than t…

You may want to have a look at Mistral OCR: https://mistral.ai/news/mistral-ocr

Re: ChatGPT agent: bridging research and action

#364

Post by j_timberlake »

2oMg3YWV26eKIs wrote: post #167

The security risks with this sound scary. Let's say you give it access to your email and calendar. Now it knows all of your deepest secrets. The linked article acknowledges that prompt injection is a risk for the agent: > Prompt injections are attempts by third parties to manipulate its behavior through malicious instructions that ChatGPT agent may encounter on the web while completing a task. For example, a maliciou…

Create a burner account for email/calendar, that solves most of those problems. Nobody will care if the AI leaks that you have a dentist appointment on Tuesday.

Re: ChatGPT agent: bridging research and action

#365

Post by d--b »

twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

I see it as a good reason why people aren’t going to lose their jobs that much.

It just make people quite faster at what they’re already doing.

Re: ChatGPT agent: bridging research and action

#366

Post by j_timberlake »

AgentMatrixAI wrote: post #47

I'm not so optimistic as someone that works on agents for businesses and creating tools for it. The leap from low 90s to 99% is classic last mile problem for LLM agents. The more generic and spread an agent is (can-do-it-all) the more likely it will fail and disappoint. Can't help but feel many are optimizing happy paths in their demos and hiding the true reality. Doesn't mean there isn't a place for agents but rathe…

I mostly agree with this. The goal with AI companies is not to reach 99% or 100% human-level, it's >100% (do tasks better than an average human could, or eventually an expert).

But since you can't really do that with wedding planning or whatnot, the 100% ceiling means the AI can only compete on speed and cost. And the cost will be... whatever Nvidia feels like charging per chip.

Re: ChatGPT agent: bridging research and action

#367

Post by intended »

slg wrote: post #225
Aurornis wrote: post #127

Earlier quoted context omitted.

> how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... This is where the AI hype bites people. A great use of AI in this situation would be to automate the collection and checking of data. Se…

The proper use of these systems is to treat them like an intern or new grad hire. You can give them the work that none of the mid-tier or senior people want to do, thereby speeding up the team. But you will have to review their work thoroughly because there is a good chance they have no idea what they are actually doing. If you give them mission-critical work that demands accuracy or just let them have free rein with…

For this example - Let’s replace the word “intern” with “initial-stage-experts” or something.

There’s a reason people invest their time with interns.

Re: ChatGPT agent: bridging research and action

#369

Post by arthurcolle »

dustingetz wrote: post #326
Aurornis wrote: post #127

Earlier quoted context omitted.

> how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... This is where the AI hype bites people. A great use of AI in this situation would be to automate the collection and checking of data. Se…

98% sure each commit doesn’t corrupt the database, regress a customer feature, open a security vulnerability. 50 commits later … (which is like, one day for an agentic workflow)

I would be embarrassed to be at OpenAI releasing this and pretending the last 9 months haven't happened... waxing poetically about "age of agents" - absolutely cringe and pathetic

Re: ChatGPT agent: bridging research and action

#370

Post by fudged71 »

alach11 wrote: post #14

It's very hard for me to imagine the current level of agents serving a useful purpose in my personal life. If I ask this to plan a date night with my wife this weekend, it needs to consult my calendar to pick the best night, pick a bar and restaurant we like (how would it know?), book a babysitter (can it learn who we use and text them on my behalf?), etc. This is a lot of stuff it has to get right, and it requires a…

I think you might enjoy this post about Productive Friction and the benefits: https://every.to/context-window/why-you-need-productive-fric...

The act of choosing a date spot is part of your human connection with the person, don’t automate it away!

Focus the automation on other things :)

Post reply on HN