Live data from Hacker News

Multi AI agent systems using OpenAI's assistants API

github.com

81–87 of 87 posts

Re: Multi AI agent systems using OpenAI's assistants API

#81
post #80

I don’t understand the comment about server send events not being async friendly. What is unfriendly about this? import OpenAI from 'openai'; const openai = new OpenAI(); async function main() { const stream = await openai.chat.completions.create({ model: 'gpt-4', messages: [{ role: 'user', content: 'Say this is a test' }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.d…

> const stream = await… Is this right? Aren’t you prematurely unwrapping the promise here?

I believe that is what gets the call started so awaiting there is okay. There isn’t anything to stream at that point.

Re: Multi AI agent systems using OpenAI's assistants API

#82
post #34
post #4

I'd be interested in knowing if anyone is seriously using the assistants API, it feels like such a lock in to OpenAIs platform when your can alternatively just use completions that are much more easily interchanged.

I do and built Assistants API compat layer for Groq and Anthropic: https://github.com/supercorp-ai/supercompat I’d argue that Assistants API DX > manual completions API.

Are you using Assistants API v2 with streaming?

Re: Multi AI agent systems using OpenAI's assistants API

#84
post #34

Earlier quoted context omitted.

I do and built Assistants API compat layer for Groq and Anthropic: https://github.com/supercorp-ai/supercompat I’d argue that Assistants API DX > manual completions API.

Are you using Assistants API v2 with streaming?

Yeah, I do both in prod and in the lib. In the lib I even ported Anthropics streaming API to be OpenAI compatible. Will write the docs over the coming days if interested.

Re: Multi AI agent systems using OpenAI's assistants API

#85
post #76

Earlier quoted context omitted.

True, this is how RAG works, but this is why I prefer to use open-source LLMs for RAG: because the token costs are less opaque and I can control how many chunks I pull fromthe database to manage my costs

I believe it will get better and more efficient as we go. On a side note, OpenAI seems to release products before they are ready and they evolve as they go.

> I believe it will get better and more efficient as we go.

Yes of course. The point remains: the LLM has to process the data somehow.

If you are concerned about costs and token usage then switch to a provider that works for your problem (Flash Gemini looks very interesting..)

Re: Multi AI agent systems using OpenAI's assistants API

#86
post #79

Earlier quoted context omitted.

we do multistep programs in louie.ai via a variety of agents/tools, like "get X data from DB Y, wrangle cols A+B in Python, and then draw an interactive map + graph" The ultimate answer is fairly short if you are a senior python data scientist, like 50loc. The agents will wander and iterate until they push through. You might correct & tweak if a bit off. Importantly, this does agents opposite of the way Devin AI engi…

So the human is like a reviewer, coming in, checking things, tweaking etc, then sending it back to the machine? (At which point the cycle continues)

Yes, imagine data analysis scenarios like Excel users or Jupyter notebooks, or operational investigations like user 360's and security incidents. Just now defaulting to natural language and connected to your data silos and a variety of analytics tools & libraries.

We try to make the generated code and backing data explainable. Users are figuring out the scenario by having the AI go ahead for them, and automating much of the debug loop in typical coding and investigations, so folks can focus more on the analysis, less on syntax, schemas, libraries, and be more ambitious on each step.

Importantly, it is still kind of like making a much more accessible Jupyter notebook or editable excel/doc, vs a linear chat session. Instead of generating the whole notebook and it being buggy and you starting over (~= Devin, or notebook.io's ChatGPT plugin), you drive it forward only 1-3 cells at a time, and as it is an interactive document so you can edit those, go to the next, or non-destructively edit earlier ones. In contrast, ChatGPT's data assistant deletes cells below the current edit, which would stink in a normal data env.

There are other differences, but from a perspective of using genAI well, we budget 3-60s for genAI assisting in 1-3 steps, aiming for 10-100x productivity wins and a lot more peace of mind during it. Taking 1-3 steps forward may mean the AI takes 3-10 internally due to backtracking / CoT / etc

We could let the system take 100 turns, and have interesting experiments there such as around security investigations, but the use cases become more niche due to cascading errors => reliability.

Re: Multi AI agent systems using OpenAI's assistants API

#87
post #28
post #23

Earlier quoted context omitted.

I would love to be using Claude, but you can't get API access (beyond an initial trial period) in the EU without providing a European VAT number. They don't want personal users or people to even learn and experiment I guess.

You can use the Claude APIs via OpenRouter with a pre-paid account.

Thanks, this did the job!
Post reply on HN