Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agents but they're working in big tech

alexsima.substack.com

Post reply on HN
Print view View original

21–30 of 58 posts

Re: AI agents but they're working in big tech

#21

Post by kgdiem »

simonw wrote: post #18
kgdiem wrote: post #13

Earlier quoted context omitted.

Midjourney uses multiple agents for determining if a prompt is appropriate or not. I kinda did this, too. I made a 3 agent system — one is a router that parses the request and determines where to send it (to the other 2) one is a chat agent and the third is an image generator. If the router determines an image is requested, the chat agent is tasked with making a caption to go along with the image. It works well enoug…

Is that multiple agents, or just multiple prompts? Are agents and prompts the same thing?

I am fairly certain that agents are the same thing as prompts (but also could be different).

Only the chat prompt/agent/whatever is connected to RAG; the image generator is DALLE and the router is a one-off call each time.

eg, it could be the same model with a different prompt or a different model + different prompt all together. AFAIU it’s just serving a different purpose than the other calls

Re: AI agents but they're working in big tech

#22

Post by [deleted] »

alsima wrote: post #19
Imnimo wrote: post #14

Earlier quoted context omitted.

I don't even buy that the linked paper justifies the claim. All the paper does is draw multiple samples from an LLM and take the majority vote. They do try integrating their majority vote algorithm with an existing multi-agent system, but it usually performs worse than just straightforwardly asking the model multiple times (see Table 3). I don't understand how the author of this article can make that claim, nor why t…

Looking at Table 3: "Our [sampling and voting] method outperforms other methods used standalone in most cases and always enhances other methods across various tasks and LLMs", which benchmark did the majority vote algorithm perform worse in?

[deleted]

Re: AI agents but they're working in big tech

#23

Post by alsima »

Imnimo wrote: post #20
alsima wrote: post #19

Earlier quoted context omitted.

Looking at Table 3: "Our [sampling and voting] method outperforms other methods used standalone in most cases and always enhances other methods across various tasks and LLMs", which benchmark did the majority vote algorithm perform worse in?

I'm not saying that sampling and majority voting performed worse. I'm saying that multi-agent interaction (labeled Debate and Reflection) performed worse than straightforward approaches that just query multiple times. For example, the Debate method combined with their voting mechanism gets 0.48 GSM8K with Llama2-13B. But majority voting with no multi-agent component (Table 2) gets 0.59 on the same setting. And majori…

I see. Agree with the point about marginal improvements at a hefty increase in computational cost (I touch on this a bit at the end of the blog post where I mention that better performance requires better tooling/base models). Though I would still consider sampling and voting a “multi-agent” framework as it’s still performing an aggregation over multiple results.

Re: AI agents but they're working in big tech

#26

Post by sandspar »

fancyfredbot wrote: post #24

Can I just ask whether other people think that "agentic" is a word? As far as I can tell it's not in the OED or Miriam Webster dictionaries. But recently everyone's using it so perhaps it soon will be.

"Agentic" is a term of art from psychology that's diffused into common usage. It dates back to the 1970s, primarily associated with Albert Bandura, the guy behind the Bobo doll experiment.

From ChatGPT: Other examples include "heuristic," "cognitive dissonance," "meta-cognition," "self-actualization," "self-efficacy," "locus of control," and "archetype."

Re: AI agents but they're working in big tech

#28

Post by fancyfredbot »

sandspar wrote: post #26
fancyfredbot wrote: post #24

Can I just ask whether other people think that "agentic" is a word? As far as I can tell it's not in the OED or Miriam Webster dictionaries. But recently everyone's using it so perhaps it soon will be.

"Agentic" is a term of art from psychology that's diffused into common usage. It dates back to the 1970s, primarily associated with Albert Bandura, the guy behind the Bobo doll experiment. From ChatGPT: Other examples include "heuristic," "cognitive dissonance," "meta-cognition," "self-actualization," "self-efficacy," "locus of control," and "archetype."

Thanks. Interesting! I see "Agentic state" is one where an "individual perceives themselves as an agent of the authority figure and is willing to carry out their commands, even if it goes against their own moral code". That's ironic as most LLMs have such strong safety training that it's almost impossible to get them to enter such a state.

Re: AI agents but they're working in big tech

#29

Post by memhole »

simonw wrote: post #3

"Research shows that AI systems with 30+ agents out-performs a simple LLM call in practically any task (see More Agents Is All You Need), reducing hallucinations and improving accuracy." Has anyone heard of that actually playing out practically in real-world applications? This article links to the paper about it - https://arxiv.org/abs/2402.05120 - but I've not heard from anyone who's implementing production systems…

Thanks for this paper! Still early so not quite production, but I’ve seen positive results on tasks scaling the number of “agents”. I more or less think of agents as a task focused prompt and these agents are slight variations of the task. It makes me think I’m running some kind of Monte Carlo simulation.

Re: AI agents but they're working in big tech

#30

Post by BoredPositron »

kgdiem wrote: post #17
BoredPositron wrote: post #15

Earlier quoted context omitted.

Midjourney does not use an agent system, they use a single call.

I remembered reading that they used > 1, found this screengrab of discord on Reddit: https://www.reddit.com/r/midjourney/comments/137bj1o/new_upd...

It's still one call they just use a different llm now.
Post reply on HN