Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Principles for Building One-Shot AI Agents

edgebit.io

Post reply on HN
Print view View original

11–20 of 33 posts

Re: Principles for Building One-Shot AI Agents

#11

Post by robszumski »

TZubiri wrote: post #3

What is a “one-shot” AI Agent? A one-shot AI agent enables automated execution of a complex task without a human in the loop. Not at all what one-shot means in the field. Zero-shot, one-shot and many-shot means how many examples at inference time are needed to perform a task Zero shot: "convert these files from csv to json" One shot: "convert from csv to json, like "id,name,age/n1,john,20" to {id:"1",name:"tom",age:"…

Fair criticism. I was going for the colloquial usage of "you get one shot" but yeah I did read that Google paper the other day referring to these as zero-shot.

Re: Principles for Building One-Shot AI Agents

#14

Post by dullcrisp »

sebastiennight wrote: post #8
randysalami wrote: post #4

Earlier quoted context omitted.

I think we need quantum systems to ever break out of that issue. EDIT: not as to creating an agent that can do anything but creating an agent that more reliably represents and respects its reality, making it easier for us to reason and work with seriously.

Could you share the logic behind that statement? Because here I'm getting "YouTuber thumbnail vibes" at the idea of solving non-deterministic programming by selecting the one halting outcome out of a multiverse of possibilities

ELI40 “YouTuber thumbnail vibes?”

Re: Principles for Building One-Shot AI Agents

#15

Post by pmichaud »

dullcrisp wrote: post #14
sebastiennight wrote: post #8

Earlier quoted context omitted.

Could you share the logic behind that statement? Because here I'm getting "YouTuber thumbnail vibes" at the idea of solving non-deterministic programming by selecting the one halting outcome out of a multiverse of possibilities

ELI40 “YouTuber thumbnail vibes?”

I think he means just try shit until something works better.

Re: Principles for Building One-Shot AI Agents

#16

Post by daxfohl »

randysalami wrote: post #10
sebastiennight wrote: post #8

Earlier quoted context omitted.

Could you share the logic behind that statement? Because here I'm getting "YouTuber thumbnail vibes" at the idea of solving non-deterministic programming by selecting the one halting outcome out of a multiverse of possibilities

That would be some Dr. Strange stuff. I’m just saying a quantum AI agent would be more grounded when deciding when to stop based on the physical nature of their computation vs. engineering hacks we need for current classical systems that become inherently inaccurate representations of reality. I could be wrong.

Quantum computation is no different than classical, except the bit registers have the ability to superpose and entangle, which allows certain specific algorithms like integer factorization to run faster. But conceptually it's still just digital code and an instruction pointer. There's nothing more "physical" about it than classical computing.

Re: Principles for Building One-Shot AI Agents

#17

Post by bhl »

sebastiennight wrote: post #2

> A different type of hard failure is when we detect that we’ll never reach our overall goal. This requires a goal that can be programmatically verified outside of the LLM. This is the largest issue : using LLMs as a black box means for most goals, we can't rely on them to always "converge to a solution" because they might get stuck in a loop trying to figure out if they're stuck in a loop. So then we're back to writ…

Just give your tool call loop to a stronger model to check if it’s a loop.

This is what I’ve done working with smaller model: if it fails validation once, I route it to a stronger model just for that tool call.

Re: Principles for Building One-Shot AI Agents

#18

Post by canadiantim »

lerp-io wrote: post #12

u can’t one shot anything, you have to iterate many many times.

You one-shot it, then you iterate.

Sounds tautological but you want to get as far as possible with the one-shot before iterating, because one-shot is when the results have the most integrity

Re: Principles for Building One-Shot AI Agents

#19

Post by behnamoh »

bhl wrote: post #17
sebastiennight wrote: post #2

> A different type of hard failure is when we detect that we’ll never reach our overall goal. This requires a goal that can be programmatically verified outside of the LLM. This is the largest issue : using LLMs as a black box means for most goals, we can't rely on them to always "converge to a solution" because they might get stuck in a loop trying to figure out if they're stuck in a loop. So then we're back to writ…

Just give your tool call loop to a stronger model to check if it’s a loop. This is what I’ve done working with smaller model: if it fails validation once, I route it to a stronger model just for that tool call.

> if it fails validation once, I route it to a stronger model just for that tool call.

the problem the GP was referring to is that even the large model might fail to notice it's struggling to solve a task and keep trying more-or-less the same approaches until the loop is exhausted.

Re: Principles for Building One-Shot AI Agents

#20

Post by zavec »

devmor wrote: post #5
TZubiri wrote: post #3

What is a “one-shot” AI Agent? A one-shot AI agent enables automated execution of a complex task without a human in the loop. Not at all what one-shot means in the field. Zero-shot, one-shot and many-shot means how many examples at inference time are needed to perform a task Zero shot: "convert these files from csv to json" One shot: "convert from csv to json, like "id,name,age/n1,john,20" to {id:"1",name:"tom",age:"…

Given the misunderstandings and explanation of how they struggled with a long-solved ml problem, I believe this article was likely written by someone without much formal experience in AI. This is probably a case where some educational training could have saved the engineer(s) involved a lot of frustration.

As a casual ML non-practicioner, what was the long-solved ML problem they ran up against?
Post reply on HN