Bare JS. What is this 2001?
What would you like to see instead? What would be the benefit of something more complex?
Multi AI agent systems using OpenAI's assistants API
71–80 of 87 posts
Re: Multi AI agent systems using OpenAI's assistants API
#72Assistants API is promising, but earlier versions have many issues, especially with how it calculates the costs. As per OpenAI docs, you pay for data storage, a fixed price per API call, + token usage. It sounds straightforward until you start using it. Here is how it works. When you upload attachments, in my case a very large PDF, it chunks that PDF into small parts and stores them in a vector database. It seems lik…
There isn’t really any other way for this to work. The only way for the model to answer questions on your pdf is for the information to be somewhere in the prompt.
The way they vectorized the PDF could be less efficient than simply extracting the text and dropping it into context as text. If it's a 100 MB PDF then it's probably a scanned PDF, and OpenAI is probably using an OCR model to vectorize each page directly. It seems an opaque process with room to be inefficient. So I would be interested to know if we could save on token/vector fees by preprocessing the PDF to text with our own OCR.
Re: Multi AI agent systems using OpenAI's assistants API
#73Re: Multi AI agent systems using OpenAI's assistants API
#74Re: Multi AI agent systems using OpenAI's assistants API
#75Re: Multi AI agent systems using OpenAI's assistants API
#76Earlier quoted context omitted.
> as every time you make a call, the system loads a large chunk or many chunks and sends them to the model along with your prompt, This is how RAG works. While you can come up with work-arounds like using lesser LLMs as a pre-filtering step the fact is that if you need GPT to read the doc you need GPT to read the doc.
True, this is how RAG works, but this is why I prefer to use open-source LLMs for RAG: because the token costs are less opaque and I can control how many chunks I pull fromthe database to manage my costs
I believe it will get better and more efficient as we go. On a side note, OpenAI seems to release products before they are ready and they evolve as they go.
Re: Multi AI agent systems using OpenAI's assistants API
#77Earlier quoted context omitted.
There isn’t really any other way for this to work. The only way for the model to answer questions on your pdf is for the information to be somewhere in the prompt.
The way they vectorized the PDF could be less efficient than simply extracting the text and dropping it into context as text. If it's a 100 MB PDF then it's probably a scanned PDF, and OpenAI is probably using an OCR model to vectorize each page directly. It seems an opaque process with room to be inefficient. So I would be interested to know if we could save on token/vector fees by preprocessing the PDF to text with…
No, it is not a scanned PDF but a standard textual PDF with tables, bullet points, chapters, etc. Somewhat like a manual.
Re: Multi AI agent systems using OpenAI's assistants API
#78From their linked main page:
> In my opinion, exploration of multi-agent systems is going to require a broader audience of engineers. For AI to become a true commodity, it needs to move out of the Python origins and into more popular languages like JavaScript , a major fact on why I wrote Experts.js.
I wholeheartedly agree
Re: Multi AI agent systems using OpenAI's assistants API
#79Earlier quoted context omitted.
We tried using a multi-agent system for a complex NLP-type task and we found: - Too many errors that just propogate on top of each other, if a single agent in the chain generates something even a little bit off then the whole system goes off the rails. - You often end up having to pass a massive amount of shared context to every agent which just increases the cost dramatically. Curiously enough we had an architect fr…
we do multistep programs in louie.ai via a variety of agents/tools, like "get X data from DB Y, wrangle cols A+B in Python, and then draw an interactive map + graph" The ultimate answer is fairly short if you are a senior python data scientist, like 50loc. The agents will wander and iterate until they push through. You might correct & tweak if a bit off. Importantly, this does agents opposite of the way Devin AI engi…
So the human is like a reviewer, coming in, checking things, tweaking etc, then sending it back to the machine? (At which point the cycle continues)
Re: Multi AI agent systems using OpenAI's assistants API
#80I don’t understand the comment about server send events not being async friendly. What is unfriendly about this? import OpenAI from 'openai'; const openai = new OpenAI(); async function main() { const stream = await openai.chat.completions.create({ model: 'gpt-4', messages: [{ role: 'user', content: 'Say this is a test' }], stream: true, }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.d…
> const stream = await…
Is this right? Aren’t you prematurely unwrapping the promise here?