Live data from Hacker News

Agents Are Not Enough

arxiv.org

41–50 of 165 posts

Re: Agents Are Not Enough

#43
post #3

Earlier quoted context omitted.

re: LLM as UI: Given that I don't trust LLMs to be deterministic, I wouldn't trust them to make the correct API call every time I tell it to do X.

They are deterministic at 0 temperature

At zero temp there is still non-determism due to sampling and the fact that floating point addition is not commutative so you will get varying results due to parallelism.

Re: Agents Are Not Enough

#44
post #23
post #7

Earlier quoted context omitted.

I really need to write that up in one place - closest I've got is this section from my 2024 review https://simonwillison.net/2024/Dec/31/llms-in-2024/#-agents-...

didnt your summary https://gist.github.com/simonw/beaa5f90133b30724c5cc1c4008d0... pretty much cover it?

Whoa missed this! Love it.

Re: Agents Are Not Enough

#45
post #5
post #4

This paper does at least lead with its version of what "agents" means (I get very frustrated when people talk about agents without clarifying which of the many potential definitions they are using): > An agent, in the context of AI, is an autonomous entity or program that takes preferences, instructions, or other forms of inputs from a user to accomplish specific tasks on their behalf. Agents can range from simple sy…

You posted on X a while back asking for a crowdsourced definition of what an "agent" was and I regularly cite that thread as an example of the fact that this word is so blurry right now.

[dead]

Re: Agents Are Not Enough

#46
post #7
post #5

Earlier quoted context omitted.

You posted on X a while back asking for a crowdsourced definition of what an "agent" was and I regularly cite that thread as an example of the fact that this word is so blurry right now.

I really need to write that up in one place - closest I've got is this section from my 2024 review https://simonwillison.net/2024/Dec/31/llms-in-2024/#-agents-...

This write up was also fantastic and has made the rounds at our org!

Re: Agents Are Not Enough

#47

I don't get the hype about Agents. It's just calling a LLM n-times with slightly different prompts Sure, you get the ability to correct previous mistakes, it's basically a custom chain of thought - but errors compound and the results coming from agents have a pretty low success rate. Bruteforcing your way out of problems can work sometimes (as evinced by the latest o3 benchmarks) but it's expensive and rarely viable…

> It's just calling a LLM n-times with slightly different prompts

It can be, but ideally each agent’s model, prompts and tools are tailored to a particular knowledge domain. That way tasks can be broken down into subtasks which are classified and passed to the agents best suited to them.

Agree RE it being bruteforce and expensive but it does look like it can improve some aspects of LLM use.

Re: Agents Are Not Enough

#48
Math that can't be too warm and too accurate to work may have challenges being too accurate and reliable with repeating processes.

Re: Agents Are Not Enough

#50
post #7
post #5

Earlier quoted context omitted.

You posted on X a while back asking for a crowdsourced definition of what an "agent" was and I regularly cite that thread as an example of the fact that this word is so blurry right now.

I really need to write that up in one place - closest I've got is this section from my 2024 review https://simonwillison.net/2024/Dec/31/llms-in-2024/#-agents-...

I took a crack at it here that tries to bridge the gap from "autonomous" which is just software to that Agentic autonomy - https://www.aiimpactfrontier.com/p/framework-for-ai-agents
Post reply on HN