Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT agent: bridging research and action

openai.com

Post reply on HN
Print view View original

471–480 of 508 posts

Re: ChatGPT agent: bridging research and action

#471

Post by casey2 »

Aurornis wrote: post #127
twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

> how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... This is where the AI hype bites people. A great use of AI in this situation would be to automate the collection and checking of data. Se…

In reality most people will just scan for something that is obviously wrong, check that, and call the rest "good enough". Government data is probably going to get updated later anyhow. It's just a target for a company to aim for. For many companies the cost savings is much more than having a slightly larger margin of error on some projections. For other companies they will just have to accept the several hours of saved time rather than the full day.

Re: ChatGPT agent: bridging research and action

#472

Post by financypants »

ronjakoi wrote: post #451
ivape wrote: post #196

Earlier quoted context omitted.

”The people who YOLO it with prompting cycles until the code passes tests and then submit a PR are causing problems almost as fast as they’re developing new features in non-trivial codebases.” This might as well be the new definition of “script kiddie”, and it’s the kids that are literally going to be the ones birthed into this lifestyle. The “craft” of programming may not be carried by these coming generations and p…

Too bad no one will be able to read it. Better make it a video essay.

with subtitles flashing in the center of the screen

Re: ChatGPT agent: bridging research and action

#473

Post by financypants »

stockresearcher wrote: post #312
simantel wrote: post #197

Earlier quoted context omitted.

> It feels kind of odd that almost no one is talking about self-driving now, compared to how hot of a topic it used to be Probably because it's just here now? More people take Waymo than Lyft each day in SF.

The typical Lyft vehicle is a piece of junk worth less than $20k, while the typical Waymo vehicle is a pretend luxury car with $$$ of equipment tacked on. Waymo needs to be proving 5-10x the number of daily rides as Lyft before we get excited

Their cars cost waymo money

Re: ChatGPT agent: bridging research and action

#474

Post by seunosewa »

__jonas wrote: post #332
WXLCKNO wrote: post #276

Earlier quoted context omitted.

In the system I'm building the main agent doesn't have access to tools and must call scoped down subagents who have one or two tools at most and always in the same category (so no mixed fetch and calendar tools). They must also return structured data to the main agent. I think that kind of isolation is necessary even though it's a bit more costly. However since the subagents have simple tasks I can use super cheap mo…

What isolation is there? If a compromised sub agent returns data that gets inserted into the main agents context (structured or not) then the end result is the same as if the main agent was directly interacting with the compromising resource is it not?

Perhaps a reference to the data can be inserted in prompt. thee key or filename

Re: ChatGPT agent: bridging research and action

#475

Post by casey2 »

samtp wrote: post #82
twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

This is the exact same issue that I've had trying to use LLMs for anything that needs to be precise such as multi-step data pipelines. The code it produces will look correct and produce a result that seems correct. But when you do quality checks on the end data, you'll notice that things are not adding up. So then you have to dig into all this overly verbose code to identify the 3-4 subtle flaws with how it transform…

It's more likely that, because you didn't do the work, you haven't already justified the flaws to yourself.

Re: ChatGPT agent: bridging research and action

#476

Post by casey2 »

2oMg3YWV26eKIs wrote: post #167

The security risks with this sound scary. Let's say you give it access to your email and calendar. Now it knows all of your deepest secrets. The linked article acknowledges that prompt injection is a risk for the agent: > Prompt injections are attempts by third parties to manipulate its behavior through malicious instructions that ChatGPT agent may encounter on the web while completing a task. For example, a maliciou…

You should treat all email as public

Re: ChatGPT agent: bridging research and action

#478

Post by ytpete »

thorum wrote: post #202
twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

People say this, but in my experience it’s not true. 1) The cognitive burden is much lower when the AI can correctly do 90% of the work. Yes, the remaining 10% still takes effort, but your mind has more space for it. 2) For experts who have a clear mental model of the task requirements, it’s generally less effort to fix an almost-correct solution than to invent the entire thing from scratch. The “starting cost” in me…

> The cognitive burden is much lower when the AI can correctly do 90% of the work.

It's a high cognitive burden if you don't know which 10% of the work the AI failed to do / did incorrectly, though.

I think you're picturing a percentage indicating what scope of the work the AI covered, but the parent was thinking about the accuracy of the work it did cover. But maybe what you're saying is if you pick the right 90% subset, you'll get vastly better than 98% accuracy on that scope of work? Maybe we just need to improve our intuition for where LLMs are reliable and where they're not so reliable.

Though as others have pointed out, these are just made-up numbers we're tossing around. Getting 99% accuracy on 90% of the work is very different from getting 75% accuracy on 50% of the work. The real values vary so much by problem domain and user's level of prompting skill, but it will be really interesting as studies start to emerge that might give us a better idea of the typical values in at least some domains.

Re: ChatGPT agent: bridging research and action

#479

Post by divan »

ainch wrote: post #346
divan wrote: post #218

And I'm still waiting for the simple feature – the ability to edit documents in projects. I use projects for working on different documents - articles, research, scripts, etc. And would absolutely love to write it paragraph after paragraph with the help of ChatGPT for phrasing and using the project knowledge. Or using voice mode - i.e. on a walk "Hey, where did we finish that document - let's continue. Read the last…

Have you tried the Canvas feature for collaborative writing? Agreed on voice mode - would be great to be able to narrate while doing busywork round the house.

As I understand, Canvas belongs to the chat and no to the project. If there was a way to link Canvas/artifact to the project - that would do the trick.

Re: ChatGPT agent: bridging research and action

#480

Post by vidarh »

wat10000 wrote: post #452
vidarh wrote: post #392

Earlier quoted context omitted.

Unless we exceed the turing computable - which there isn't the tiniest shred of evidence for -, nothing we do is "outside the realm of 'token generation'". There is no reason why the token stream generated needs to be treated as equivalent to an internal monologue, or need to always be used to produce language at all, and Turing complete systems are computationally equivalent (they can all compute the same set of fun…

A token stream is universal, but I don't see any reason to think that a token stream generated by an LLM can ever be universal. I mean, theoretically in an "infinite tape" model, sure. But we don't even know if it's physically possible. Given that the observable universe is finite and the information capacity of a finite space is also finite, then anything humans can do can theoretically be encoded with a lookup tabl…

Unless humans exceed the Turing computable, the human brain is the existence proof that a sufficiently complex Turing machine can be made to replicate human thought in a compact space.

That encoding a naive/basic UTM in an LLM would potentially be impractical is largely irrelevant in that case, because for any UTM you can "compress" the program by increasing the number of states or symbols, and effectively "embedding" the steps required to implement a more compact representation in the machine itself.

While it is possible using current LLM architectures might make encoding a model that can be efficient enough to be physically practical impossible, there's no reasonable basis for assuming this approach can not translate.

Post reply on HN