Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swarm, a new agent framework by OpenAI

github.com

Post reply on HN
Print view View original

81–90 of 113 posts

Re: Swarm, a new agent framework by OpenAI

#81

Post by dboreham »

llm_trw wrote: post #69
ValentinA23 wrote: post #40

Earlier quoted context omitted.

>People really don't understand how much better LLM swarms get with more agents. I never hit a point of diminishing returns on text quality Could you elaborate please ? One use for swarms is to use multiple agents/prompts in place of one single agent with one long prompt in order to increase performance by splitting one big task into many. It is very time consuming though, as it requires experimenting to determine ho…

>Could you elaborate please ? No. I've tried explaining this to supposedly smart people in both a 15 minute pitch deck and a research paper and unless they were inclined to think it from the start no amount of proof has managed to convince them. I figure it's just not possible to convince people, even with the proof in front of them, of how powerful the system is. The same way that we still have people arguing _right…

> people arguing _right now_ that all LLMs are just auto complete on steroids.

Funny because when I learned about how LLMS worked my immediate thought was "Oh, humans are just LLMs on steroids". So auto complete on steroids squared.

Re: Swarm, a new agent framework by OpenAI

#82

Post by theptip »

hubraumhugo wrote: post #49

Has anyone seen AI agents working in production at scale? It doesn't matter if you're using Swarm, langchain, or any other orchestration framework if the underlying issue is that AI agents too slow, too expensive, and too unreliable. I wrote about AI agent hype vs. reality[0] a while ago, and I don't think it has changed yet. [0] https://www.kadoa.com/blog/ai-agents-hype-vs-reality

Frankly, what you are describing is a money-printing machine. You should expect anyone who has figured out such a thing to keep it as a trade secret, until the FOSS community figures out and publishes something comparable.

I don’t think the tech is ready yet for other reasons, but absence of anyone publishing is not good evidence against.

Re: Swarm, a new agent framework by OpenAI

#83

Post by fsndz »

hubraumhugo wrote: post #49

Has anyone seen AI agents working in production at scale? It doesn't matter if you're using Swarm, langchain, or any other orchestration framework if the underlying issue is that AI agents too slow, too expensive, and too unreliable. I wrote about AI agent hype vs. reality[0] a while ago, and I don't think it has changed yet. [0] https://www.kadoa.com/blog/ai-agents-hype-vs-reality

True. In the classic form of automation, reasoning is externalized into rules. In the case of AI agents, reasoning is internalized within a language model. This is a fundamental difference. The problem is that language models are not designed to reason. They are designed to predict the next most likely word. They mimic human skills but possess no general intelligence. They are not ready to function without a human in the loop. So, what are the implications of this new form of automation that AI agents represent? https://www.lycee.ai/blog/ai-agents-automation-eng

Re: Swarm, a new agent framework by OpenAI

#84

Post by johntash »

exitb wrote: post #26
htrp wrote: post #23

Does anyone else feel like these are Google-style 20% time projects from the OpenAI team members looking to leave and trying to line up VC funding?

Doesn’t working on a venture on company time put you at an enormous disadvantage in terms of ownership?

Not just company time, but company resources and the company's github org.

But yeah, I'd assume they have no ownership themselves unless they signed something explicit?

Re: Swarm, a new agent framework by OpenAI

#85

Post by keithwhor »

arnaudsm wrote: post #30
ac130kz wrote: post #17

Looks kinda poorly written: not even a single async present, print debugging, deepcopy all over the place. Such a shame that there's nothing to replace Langchain with other than writing it all from the ground up yourself.

OpenAI's code quality leaves to be desired, which is surprising considering how well compensated their engineers are. Their recent realtime demo had so many race conditions, function calling didn't even work, and the patch suggested by the community hasn't been merged for a week. https://github.com/openai/openai-realtime-api-beta/issues/14

Hey! I was responsible for developing this.

Not speaking for OpenAI here, only myself — but this is not an official SDK — only a reference implementation. The included relay is only intended as an example. The issues here will certainly be tackled for the production release of the API :).

I’d love to build something more full-featured here and may approach it as a side project. Feel free to ping me directly if you have ideas. @keithwhor on GitHub / X dot com.

Re: Swarm, a new agent framework by OpenAI

#86

Post by darknavi »

NelsonMinar wrote: post #42
antfarm wrote: post #9

There used to be another open-source agentframework by the same name, but it was for multi-agent simulations. For a moment I thought there was a new wave of interest in a deeper understanding of complex systems by means of modelling. https://en.wikipedia.org/wiki/Swarm_(simulation) https://www.santafe.edu/research/results/working-papers/the-...

Hey, I wrote that! But it was nearly 30 years ago, it's OK for someone else to use the same name. Fun fact: Swarm was one of the very few non-NeXT/Apple uses of Objective C. We used the GNU Objective C runtime. Dynamic typing was a huge help for multiagent programming compared to C++'s static typing and lack of runtime introspection. (Again, nearly 30 years ago. Things are different now.)

> compared to C++'s static typing and lack of runtime introspection. (Again, nearly 30 years ago. Things are different now.)

C++ has added a ton of great features since (especially C++11 onward) but run-time reflection is still sorely missed.

Re: Swarm, a new agent framework by OpenAI

#87

Post by fnordpiglet »

hubraumhugo wrote: post #49

Has anyone seen AI agents working in production at scale? It doesn't matter if you're using Swarm, langchain, or any other orchestration framework if the underlying issue is that AI agents too slow, too expensive, and too unreliable. I wrote about AI agent hype vs. reality[0] a while ago, and I don't think it has changed yet. [0] https://www.kadoa.com/blog/ai-agents-hype-vs-reality

Yes we use agents in a human support agent facing application that has many sub agents used to summarize and analyze a lot of different models, prior support cases, knowledge base information, third party data sets, etc, to form an expert in a specific customer and their unique situation in detected potential fraud and other cases. The goal of the expert is to reduce the cognitive load of our support agent in analyzing some often complex situation with lots of information more rapidly and reliably. Because there is no right answer and the goal is error reduction not elimination it’s not necessary to have determinism, just do better than a human at understanding a lot of divergent information rapidly and answering various queries. Cost isn’t an issue because the decisions are high value. Speed isn’t an issue because the alternative is a human attempting to make sense of an enormous amount of information in many systems. It has dramatically improved our precision and recall over pure humans.

Re: Swarm, a new agent framework by OpenAI

#88

Post by dimitri-vs »

d4rkp4ttern wrote: post #66
xrd wrote: post #52

Earlier quoted context omitted.

I want hear more about this. I'm playing with langroid, crew.ai, and dspy and they all layer so many abstractions on top of a shifting LLM landscape. I can't believe anyone is really using them in the way their readme goals profess.

Not you in particular, but I hear this common refrain that the "LLM landscape is shifting", but what exactly is shifting? Yes new models are constantly announced, but at the end of the day, interacting with the LLMs involves making calls to an API, and the OpenAI API (and perhaps Anthropic's variant) has become fairly established, and this API will obviously not change significantly any time soon. Given that there is…

Sonnet 3.5 and other large context models made context management approaches irrelevant and will continue to do so.

o1 (and likely sonnet 3.5) made chain of through and other complex prompt engineering irrelevant.

Realtime API (and others that will soon follow) will made the best VTT > LLM > TTV irrelevant.

VLMs will likely make LLMs irrelevant. Who knows what Google has planned for Gemini 2.

The point is building these complex agents has been proven a waste of time over and over again until, at least until we see a plateau in models. It's much easier to swap in a single API call and modify one or two prompts than to rework a convoluted agentic approach. Especially when it's very clear that the same prompts can't be reused reliably between different models.

Re: Swarm, a new agent framework by OpenAI

#89

Post by irthomasthomas »

999900000999 wrote: post #78
irthomasthomas wrote: post #73

Earlier quoted context omitted.

I use my own agent all day, every day. Here is one example: https://x.com/xundecidability/status/1835085853506650269 I've been using the general agent to build specialised sub-agents. Here's an example search agent beating perplexity: https://x.com/xundecidability/status/1835059091506450493

Do you have any code to share? I'm failing to see the point in the example, unless the agents can do things on multiple threads. For example let's say we have Boss Agent. I can ask Boss agent to organize a trip for five people to the Netherlands. Boss agent can ask some basic questions, about where my Friends are traveling from, and what our budget is . Then travel agent can go and look up how we each can get there,…

That particular task didn't need parallel agents or any of the advanced features.

The prompt was: Research claude pricing with caching and then review a conversation history to calculate the cost. First, search online for pricing for anthropic api with and without caching enabled for all of the models: claude-3-haiku, claude-3-opus and claude-3.5-sonnet (sonnet 3.5). Create a json file with ALL the pricing data.

from the llm history db, fetch the response.response_json.usage for each result under conversation_id=01j7jzcbxzrspg7qz9h8xbq1ww llm_db=$(llm logs path) schema=$(sqlite3 $llm_db '.schema') example usage: { "input_tokens": 1086, "output_tokens": 1154, "cache_creation_input_tokens": 2364, "cache_read_input_tokens": 0 }

Calculate the actual costs of each prompt by using the usage object for each response based the actual token usage cached or not. Also calculate/simulate what it would have cost if the tokens where not cached. Create interactive graphs of different kinds to show the real cost of conversation, the cache usage, and a comparison to what it would have costed without caching.

Write to intermediary files along the way.

Ask me if anything is unclear.

I just gave it your task and I'll share the results tomorrow (I'm off to bed).

Re: Swarm, a new agent framework by OpenAI

#90

Post by arach »

Worth noting there is an interesting multi-agent open source project named Swarms. When I saw this on X earlier I thought maybe the team had joined OpenAI but there's no connection between these projects

> "Swarms: The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework"

[0] https://github.com/kyegomez/swarms

[1] https://docs.swarms.world/en/latest/

Post reply on HN