Principles for Building One-Shot AI Agents
1–10 of 33 posts
Re: Principles for Building One-Shot AI Agents
#2> A different type of hard failure is when we detect that we’ll never reach our overall goal. This requires a goal that can be programmatically verified outside of the LLM.
This is the largest issue : using LLMs as a black box means for most goals, we can't rely on them to always "converge to a solution" because they might get stuck in a loop trying to figure out if they're stuck in a loop.
So then we're back to writing in a hardcoded or deterministic cap on how many iterations counts as being "stuck". I'm curious how the authors solve this.
Re: Principles for Building One-Shot AI Agents
#3What is a “one-shot” AI Agent?
A one-shot AI agent enables automated execution of a complex task without a human in the loop.
Not at all what one-shot means in the field. Zero-shot, one-shot and many-shot means how many examples at inference time are needed to perform a task
Zero shot: "convert these files from csv to json"
One shot: "convert from csv to json, like "id,name,age/n1,john,20" to {id:"1",name:"tom",age:"20"}
Re: Principles for Building One-Shot AI Agents
#4> A different type of hard failure is when we detect that we’ll never reach our overall goal. This requires a goal that can be programmatically verified outside of the LLM. This is the largest issue : using LLMs as a black box means for most goals, we can't rely on them to always "converge to a solution" because they might get stuck in a loop trying to figure out if they're stuck in a loop. So then we're back to writ…
I think we need quantum systems to ever break out of that issue.
EDIT: not as to creating an agent that can do anything but creating an agent that more reliably represents and respects its reality, making it easier for us to reason and work with seriously.
Re: Principles for Building One-Shot AI Agents
#5What is a “one-shot” AI Agent? A one-shot AI agent enables automated execution of a complex task without a human in the loop. Not at all what one-shot means in the field. Zero-shot, one-shot and many-shot means how many examples at inference time are needed to perform a task Zero shot: "convert these files from csv to json" One shot: "convert from csv to json, like "id,name,age/n1,john,20" to {id:"1",name:"tom",age:"…
Given the misunderstandings and explanation of how they struggled with a long-solved ml problem, I believe this article was likely written by someone without much formal experience in AI.
This is probably a case where some educational training could have saved the engineer(s) involved a lot of frustration.
Re: Principles for Building One-Shot AI Agents
#6> A different type of hard failure is when we detect that we’ll never reach our overall goal. This requires a goal that can be programmatically verified outside of the LLM. This is the largest issue : using LLMs as a black box means for most goals, we can't rely on them to always "converge to a solution" because they might get stuck in a loop trying to figure out if they're stuck in a loop. So then we're back to writ…
I think we need quantum systems to ever break out of that issue. EDIT: not as to creating an agent that can do anything but creating an agent that more reliably represents and respects its reality, making it easier for us to reason and work with seriously.
I don’t believe quantum computers can solve the halting problem, so I don’t think that would actually help.
This issue will likely always require a monitor “outside” of the agent.
Re: Principles for Building One-Shot AI Agents
#7Earlier quoted context omitted.
I think we need quantum systems to ever break out of that issue. EDIT: not as to creating an agent that can do anything but creating an agent that more reliably represents and respects its reality, making it easier for us to reason and work with seriously.
I don’t believe quantum computers can solve the halting problem, so I don’t think that would actually help. This issue will likely always require a monitor “outside” of the agent.
I think you’re right that they can’t “solve” the halting problem but are more capable at dealing with it than classic ai agents and more physically grounded. Outside monitoring would be required but I’d imagine less so than classical systems and in physically different ways; and to be fair, humans require monitoring too if they should halt or not, haha.
Re: Principles for Building One-Shot AI Agents
#8> A different type of hard failure is when we detect that we’ll never reach our overall goal. This requires a goal that can be programmatically verified outside of the LLM. This is the largest issue : using LLMs as a black box means for most goals, we can't rely on them to always "converge to a solution" because they might get stuck in a loop trying to figure out if they're stuck in a loop. So then we're back to writ…
I think we need quantum systems to ever break out of that issue. EDIT: not as to creating an agent that can do anything but creating an agent that more reliably represents and respects its reality, making it easier for us to reason and work with seriously.
Could you share the logic behind that statement?
Because here I'm getting "YouTuber thumbnail vibes" at the idea of solving non-deterministic programming by selecting the one halting outcome out of a multiverse of possibilities
Re: Principles for Building One-Shot AI Agents
#9> A different type of hard failure is when we detect that we’ll never reach our overall goal. This requires a goal that can be programmatically verified outside of the LLM. This is the largest issue : using LLMs as a black box means for most goals, we can't rely on them to always "converge to a solution" because they might get stuck in a loop trying to figure out if they're stuck in a loop. So then we're back to writ…
[deleted]
Re: Principles for Building One-Shot AI Agents
#10Earlier quoted context omitted.
I think we need quantum systems to ever break out of that issue. EDIT: not as to creating an agent that can do anything but creating an agent that more reliably represents and respects its reality, making it easier for us to reason and work with seriously.
Could you share the logic behind that statement? Because here I'm getting "YouTuber thumbnail vibes" at the idea of solving non-deterministic programming by selecting the one halting outcome out of a multiverse of possibilities
That would be some Dr. Strange stuff. I’m just saying a quantum AI agent would be more grounded when deciding when to stop based on the physical nature of their computation vs. engineering hacks we need for current classical systems that become inherently inaccurate representations of reality. I could be wrong.