Diabolical - I love it. Impressed that the final score came up as an alert!
Agents Are Not Enough
101–110 of 165 posts
Re: Agents Are Not Enough
#102Earlier quoted context omitted.
You posted on X a while back asking for a crowdsourced definition of what an "agent" was and I regularly cite that thread as an example of the fact that this word is so blurry right now.
It's been blurry for a long time, FWIW. I have books on "Agents" dating back to the late 90's or early 2000's in which the "Intro" chapter usually has a section that tries to define what an "agent" is, and laments that there is no universally accepted definition. To illustrate: here's a paper from 1996 that tries to lay out a taxonomy of the different kinds of agents and provide some definitions: https://citeseerx.is…
The technical difference between agents then and agents now are the fuzzy parameter mapping capabilities of LLMs, if used.
Scaling agent capability requires agents that are able to auto-map various tools.
If every different tool is a new, custom integration, that must be written by a person, then we end up where we are today -- specialized agents where there exists enough demand and stability to write and maintain those integrations, but no general purpose agents.
Ultimately, parameter mapping in a sane, consistent, globally-applicable way is the key that unlocks an agentic future, or a failure that leads to its demise.
Re: Agents Are Not Enough
#103Somewhat related but here's my take on super intelligence or AGI. I have worked with CNN,GNN and other old school AI methods, but don't have the resources to build a real SOT LLM, but I do use and tinker with LLM's occasionally. If AGI or SI(super intelligence)/is possible, and that is an if...I don't think LLM's are going to be this silver bullet solution Just as we have in the real world of people who are dedicated…
I don't see why a mixture of experts couldn't be distilled into a single model and unified latent space
You could, but in many cases you wouldn't want to. You will get superior results with a fixed compute budget by relying on external tool use (where "tool" is defined liberally, and can include smaller narrow neural nets like GraphCast & AlphaGo) rather that stuffing all tools into a monolithic model.
Re: Agents Are Not Enough
#104"More Agents is all you need" https://arxiv.org/abs/2402.05120 I could not find a "Agents considered harmful" related to AI, but there is this one: "AgentHarm: A benchmark for measuring harmfulness of LLM agents" https://arxiv.org/pdf/2410.09024 This "Agents considered harmful" is not AI-related: https://www.scribd.com/document/361564026/Math-works-09
When reading anthropics blog on agents I basically took away that their advice is you shouldn't use them to solve most problems. https://www.anthropic.com/research/building-effective-agents "For many applications, however, optimizing single LLM calls with retrieval and in-context examples is usually enough."
True this was also my conclusion in
October. Most of the complexity we are building is to fight against the limitations of LLMs. If in some way we could embed all our tools in a single call and have the LLM successfully figure out which tools to call then that would be it and we wouldn’t need any of those frameworks or libraries. But it turns out the reality of agents and tool use is pretty stark and you wouldn’t know that looking at the AI influencer spamming X, Linkedin, Youtube
However The state of agents slightly changed and while we had 25% accuracy in multiturn conversations we re now at 50.
Re: Agents Are Not Enough
#105I don't get the hype about Agents. It's just calling a LLM n-times with slightly different prompts Sure, you get the ability to correct previous mistakes, it's basically a custom chain of thought - but errors compound and the results coming from agents have a pretty low success rate. Bruteforcing your way out of problems can work sometimes (as evinced by the latest o3 benchmarks) but it's expensive and rarely viable…
> It's just calling a LLM n-times with slightly different prompts That's one way of building something you could call an "agent". It's far from the only way. It's certainly possible to build agents where the LLM plays a very small role, or even one that uses no LLM at all.
Thats a workflow
Re: Agents Are Not Enough
#106I don't get the hype about Agents. It's just calling a LLM n-times with slightly different prompts Sure, you get the ability to correct previous mistakes, it's basically a custom chain of thought - but errors compound and the results coming from agents have a pretty low success rate. Bruteforcing your way out of problems can work sometimes (as evinced by the latest o3 benchmarks) but it's expensive and rarely viable…
> It's just calling a LLM n-times with slightly different prompts It can be, but ideally each agent’s model, prompts and tools are tailored to a particular knowledge domain. That way tasks can be broken down into subtasks which are classified and passed to the agents best suited to them. Agree RE it being bruteforce and expensive but it does look like it can improve some aspects of LLM use.
That is just like having a for loop per domain.
Re: Agents Are Not Enough
#107Earlier quoted context omitted.
I am not sure what you mean by LLM when you say they are professional bullshitter. While it was certainly true for model based on transformers just doing inference, recent models have progressed significantly.
> I am not sure what you mean by LLM when you say they are professional bullshitter. Not parent-poster, but an LLM is a tool for extending a document by choosing whatever statistically-seems-right based on other documents, and it does so with no consideration of worldly facts and no modeling of logical prepositions or contradictions. (Which also relates to math problems.) If it has been fed on documents with logic pu…
Tell me you haven’t used the latest models, without telling me you haven’t used the latest models?
They do hallucinate at times, but you’re missing a lot of real utility by claiming they are basically bullshit engines.
They can now use tools, and maintain internal consistency over long context windows (with both text and video). They can iterate fully autonomously on software development by building, testing, and bug fixing on real world problems producing usable & functioning code.
There’s a reason Microsoft is putting $80 billion dollars on the line to run LLMs. It’s not because they are full of shit!
Re: Agents Are Not Enough
#108Somewhat related but here's my take on super intelligence or AGI. I have worked with CNN,GNN and other old school AI methods, but don't have the resources to build a real SOT LLM, but I do use and tinker with LLM's occasionally. If AGI or SI(super intelligence)/is possible, and that is an if...I don't think LLM's are going to be this silver bullet solution Just as we have in the real world of people who are dedicated…
I don't see why a mixture of experts couldn't be distilled into a single model and unified latent space
Exactly what DeepSeek3 is doing.
Re: Agents Are Not Enough
#109IMHO, the word agent is quickly becoming meaningless. The amount of agency that sits with the program vs. the user is something that changes gradually. So we should think about these things in terms of how much agency are we willing to give away in each case and for what gain[1]. Then the ecosystem question that the paper is trying to solve will actually solve itself, because it is already the case today that in many…
> IMHO, the word agent is quickly becoming meaningless. The amount of agency that sits with the program vs. the user is something that changes gradually Yes, the term is becoming ambiguous, but that's because it's abstracting out the part of AI that is most important and activating: the ability to work both independently and per intention/need. Per the paper: "Key characteristics of agents include autonomy, programma…
> Manufacturing went to China against the wishes of most everyone involved
Certainly those in China and the executive suites of Western countries wished it, and made it happen. Arguably the western markets wanted it too when they saw the prices dropping and offerings growing.
AI isn't happening in a vacuum. Shareholders and customers are buying it.
Re: Agents Are Not Enough
#110Earlier quoted context omitted.
> I am not sure what you mean by LLM when you say they are professional bullshitter. Not parent-poster, but an LLM is a tool for extending a document by choosing whatever statistically-seems-right based on other documents, and it does so with no consideration of worldly facts and no modeling of logical prepositions or contradictions. (Which also relates to math problems.) If it has been fed on documents with logic pu…
Tell me you haven’t used the latest models, without telling me you haven’t used the latest models? They do hallucinate at times, but you’re missing a lot of real utility by claiming they are basically bullshit engines. They can now use tools, and maintain internal consistency over long context windows (with both text and video). They can iterate fully autonomously on software development by building, testing, and bug…
It's true, they're very convincing bullshitters ;)