Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agents but they're working in big tech

alexsima.substack.com

Post reply on HN
Print view View original

31–40 of 58 posts

Re: AI agents but they're working in big tech

#31

Post by fmbb »

simonw wrote: post #3

"Research shows that AI systems with 30+ agents out-performs a simple LLM call in practically any task (see More Agents Is All You Need), reducing hallucinations and improving accuracy." Has anyone heard of that actually playing out practically in real-world applications? This article links to the paper about it - https://arxiv.org/abs/2402.05120 - but I've not heard from anyone who's implementing production systems…

Thirty recursive loops of LLMs perform better than one prompt? I should hope so!

But how much power does it need?

Re: AI agents but they're working in big tech

#32

Post by henning »

- Big tech is very different from open source

- The original SWE-bench paper only consists of solved issues when a big part of Open Source is triage, follow-up, clarification and dealing with crappy issues

- Saying " is all you need" when you are increasing your energy usage 30-fold just to fail > 50% of the time is intellectually dishonest

Re: AI agents but they're working in big tech

#33

Post by alsima »

fmbb wrote: post #31
simonw wrote: post #3

"Research shows that AI systems with 30+ agents out-performs a simple LLM call in practically any task (see More Agents Is All You Need), reducing hallucinations and improving accuracy." Has anyone heard of that actually playing out practically in real-world applications? This article links to the paper about it - https://arxiv.org/abs/2402.05120 - but I've not heard from anyone who's implementing production systems…

Thirty recursive loops of LLMs perform better than one prompt? I should hope so! But how much power does it need?

A lot...as you might imagine the costs of running the whole organization scale immensely.

Re: AI agents but they're working in big tech

#34

Post by alsima »

henning wrote: post #32

- Big tech is very different from open source - The original SWE-bench paper only consists of solved issues when a big part of Open Source is triage, follow-up, clarification and dealing with crappy issues - Saying " is all you need" when you are increasing your energy usage 30-fold just to fail > 50% of the time is intellectually dishonest

Definitely not saying multi-agents is all you need for SWE-bench haha. I touch on this at the end of the blog post, where I mention jumps in progress require better base models or tooling.

Re: AI agents but they're working in big tech

#35

Post by det2x »

It's interesting how long the word "agents"/"intelligent agents" have been around for and how long they've been hyped up for. If you go back to the 80s and 90s you will see how Microsoft was hyping up "intelligent agents" in Windows but nothing ever became of it[1].

I have yet to see an actual useful usecase for agents despite the countless posts asking for examples nobody has provided one.

[1] https://www.wired.com/1995/09/future-forward/

Re: AI agents but they're working in big tech

#36

Post by sroussey »

det2x wrote: post #35

It's interesting how long the word "agents"/"intelligent agents" have been around for and how long they've been hyped up for. If you go back to the 80s and 90s you will see how Microsoft was hyping up "intelligent agents" in Windows but nothing ever became of it[1]. I have yet to see an actual useful usecase for agents despite the countless posts asking for examples nobody has provided one. [1] https://www.wired.com/…

Or CORBA based intelligent agents in the 1990s

Re: AI agents but they're working in big tech

#38

Post by raybb »

fancyfredbot wrote: post #24

Can I just ask whether other people think that "agentic" is a word? As far as I can tell it's not in the OED or Miriam Webster dictionaries. But recently everyone's using it so perhaps it soon will be.

Seems to have been for a while.

https://en.wiktionary.org/wiki/agentic

Re: AI agents but they're working in big tech

#39

Post by connicpu »

fancyfredbot wrote: post #28
sandspar wrote: post #26

Earlier quoted context omitted.

"Agentic" is a term of art from psychology that's diffused into common usage. It dates back to the 1970s, primarily associated with Albert Bandura, the guy behind the Bobo doll experiment. From ChatGPT: Other examples include "heuristic," "cognitive dissonance," "meta-cognition," "self-actualization," "self-efficacy," "locus of control," and "archetype."

Thanks. Interesting! I see "Agentic state" is one where an "individual perceives themselves as an agent of the authority figure and is willing to carry out their commands, even if it goes against their own moral code". That's ironic as most LLMs have such strong safety training that it's almost impossible to get them to enter such a state.

In the case of LLMs, I think their training is the supreme authority.
Post reply on HN