AI agents but they're working in big tech • Hacker News Board

Re: AI agents but they're working in big tech

#51

Post by potatoman22 » Wed, Aug 07, 2024, 12:29 AM UTC

simonw wrote: post #9

alsima wrote: post #7

Earlier quoted context omitted.

I would check out this company, Swarms ( https://github.com/kyegomez/swarms ) who's working with enterprises to integrate multi-agents. But definitely a great point to focus on, the research paper mentions that the scaling of performance reduces with complexity of the task, which is definitely true for SWE

I totally believe that people are selling solutions around this idea, what I'd like to hear is genuine success stories from people who have used them in production (and aren't currently employed by a vendor).

Try it for classification tasks.

Re: AI agents but they're working in big tech

#52

Post by danenania » Wed, Aug 07, 2024, 12:53 AM UTC

aantix wrote: post #47

danenania wrote: post #16

Earlier quoted context omitted.

> Where you specify a top-level objective, it plans out those objectives, it selects a completion metric so that it knows when to finish, and iterates/reiterates over the output until completion? I built Plandex[1], which works roughly like this. The goal (so far) is not to take you from an initial prompt to a 100% working solution in one go, but to provide tools that help you iterate your way to a 90-95% solution. Y…

This looked very promising. Although, it's now prompting me to make an account when I issue `plandex new`? None of the video demos show this requirement. I think the demos should show this requirement. Or the Quickstart docs should directly link to the self-hosted instructions. "? Hey there! It looks like this is your first time using Plandex on this computer. What would you like to do? > Start an anonymous trial on…

Thanks for the feedback. The cloud option is offered as a way to get started as quickly as possible, but self-hosting is straightforward too: https://docs.plandex.ai/hosting/self-hosting

Re: AI agents but they're working in big tech

#53

Post by aantix » Wed, Aug 07, 2024, 1:13 AM UTC

danenania wrote: post #52

aantix wrote: post #47

Earlier quoted context omitted.

This looked very promising. Although, it's now prompting me to make an account when I issue `plandex new`? None of the video demos show this requirement. I think the demos should show this requirement. Or the Quickstart docs should directly link to the self-hosted instructions. "? Hey there! It looks like this is your first time using Plandex on this computer. What would you like to do? > Start an anonymous trial on…

Thanks for the feedback. The cloud option is offered as a way to get started as quickly as possible, but self-hosting is straightforward too: https://docs.plandex.ai/hosting/self-hosting

Plandex and Github's Copilot Workspace are very similar. You could draw inspiration from each other.

I tweeted about Copilot Workspace the other day.

https://twitter.com/aantix/status/1819794837375263228

Re: AI agents but they're working in big tech

#54

Post by alsima » Wed, Aug 07, 2024, 3:21 AM UTC

potatoman22 wrote: post #49

Imnimo wrote: post #20

Earlier quoted context omitted.

I'm not saying that sampling and majority voting performed worse. I'm saying that multi-agent interaction (labeled Debate and Reflection) performed worse than straightforward approaches that just query multiple times. For example, the Debate method combined with their voting mechanism gets 0.48 GSM8K with Llama2-13B. But majority voting with no multi-agent component (Table 2) gets 0.59 on the same setting. And majori…

My take: the agents are bad, the ensembling approach is good.

Potentially true as well haha

Re: AI agents but they're working in big tech

#55

Post by tmaly » Wed, Aug 07, 2024, 4:47 AM UTC

kgdiem wrote: post #13

simonw wrote: post #9

Earlier quoted context omitted.

I totally believe that people are selling solutions around this idea, what I'd like to hear is genuine success stories from people who have used them in production (and aren't currently employed by a vendor).

Midjourney uses multiple agents for determining if a prompt is appropriate or not. I kinda did this, too. I made a 3 agent system — one is a router that parses the request and determines where to send it (to the other 2) one is a chat agent and the third is an image generator. If the router determines an image is requested, the chat agent is tasked with making a caption to go along with the image. It works well enoug…

How do you jailbreak midjourney?

Re: AI agents but they're working in big tech

#56

Post by soco » Wed, Aug 07, 2024, 9:43 AM UTC

det2x wrote: post #35

It's interesting how long the word "agents"/"intelligent agents" have been around for and how long they've been hyped up for. If you go back to the 80s and 90s you will see how Microsoft was hyping up "intelligent agents" in Windows but nothing ever became of it[1]. I have yet to see an actual useful usecase for agents despite the countless posts asking for examples nobody has provided one. [1] https://www.wired.com/…

Clippy?

Re: AI agents but they're working in big tech

#57

Post by Lockal » Wed, Aug 07, 2024, 2:37 PM UTC

simonw wrote: post #3

"Research shows that AI systems with 30+ agents out-performs a simple LLM call in practically any task (see More Agents Is All You Need), reducing hallucinations and improving accuracy." Has anyone heard of that actually playing out practically in real-world applications? This article links to the paper about it - https://arxiv.org/abs/2402.05120 - but I've not heard from anyone who's implementing production systems…

Originally term "agency" was used to indicate inability of computers to act autonomously, intentionally or with purpose.

Then few developers published joke projects of automated browsers that pointlessly doomscroll random webpages saying "here are your agents".

Then few companies transformed this joke into "rag" agents - but there automated browsers don't act autonomously, they plagiarize web content according to prompt.

Then many websites started to block rag agents, and to hide it companies fell back to return data from LLMs, occasionally updating responses in the background (aka "online models").

The idea of plagiarizing content is also mixed with another idea: if LLM rewrites plagiarized content multiple times, it becomes harder to proof.

Obviously, none of involved companies will admit to plagiarizing. Instead, they will cover themselves with the idea that it leads to superintelligence. For example, if multiple neural networks repeat that 9.11 > 9.9, it will be considered more accurate[1].

[1] https://www.reddit.com/r/singularity/comments/1e4fcxm/none_o...

Re: AI agents but they're working in big tech

#58

Post by Imnimo » Thu, Aug 08, 2024, 12:15 AM UTC

potatoman22 wrote: post #49

Imnimo wrote: post #20

Earlier quoted context omitted.

I'm not saying that sampling and majority voting performed worse. I'm saying that multi-agent interaction (labeled Debate and Reflection) performed worse than straightforward approaches that just query multiple times. For example, the Debate method combined with their voting mechanism gets 0.48 GSM8K with Llama2-13B. But majority voting with no multi-agent component (Table 2) gets 0.59 on the same setting. And majori…

My take: the agents are bad, the ensembling approach is good.

Ensembling is okay, but it doesn't scale great. Like you can put in a bit more compute to draw more samples and get a better result, but you can't keep doing it without hitting a plateau, because eventually the model's majority vote stabilizes. So it's a perfectly reasonable thing to do, as long as we temper our expectations.