Bare JS. What is this 2001?
LMAO. Yes, I love ESM modules. So maybe more like 2012 or 2015. Would you like to see TypeScript?
Yes this is great so much easier to work with
Multi AI agent systems using OpenAI's assistants API
31–40 of 87 posts
Re: Multi AI agent systems using OpenAI's assistants API
#32From the website linked in the readme: “A lot of research has been doing in this are and we can expect a lot more in 2024 in this space. I promise to share some clarity around where I think this industry is headed. In personal talks I have warned that multi-agent systems are complex and hard to get right. I've seen little evidence of real-world use cases too” These assistant systems fascinate me, but I just don’t hav…
[deleted]
Re: Multi AI agent systems using OpenAI's assistants API
#33Bare JS. What is this 2001?
LMAO. Yes, I love ESM modules. So maybe more like 2012 or 2015. Would you like to see TypeScript?
Not OP, but I use TypeScript because it adds a layer of safety to the codebase.
It's like having good test coverage - you can make large changes and if the tests pass (the code compiles), you can be fairly confident that you didn't mess anything up.
I've written Ruby for years, so I'm used to dynamically typed languages. But JavaScript is it's own level of special, and there's so many ways you can accidentally mess things up.
Having tests cover every single path (especially failure paths) can be very time consuming, and often hard or messy to setup (how would you mock the OpenAI module returning an error when adding metadata to a thread?), where as using something like TypeScript can make sure your code handles all paths somewhat correctly (at least as well as the types you defined).
Your code looks clean, and you appear to have good test coverage, so you do you though :-)
Re: Multi AI agent systems using OpenAI's assistants API
#34I'd be interested in knowing if anyone is seriously using the assistants API, it feels like such a lock in to OpenAIs platform when your can alternatively just use completions that are much more easily interchanged.
I do and built Assistants API compat layer for Groq and Anthropic: https://github.com/supercorp-ai/supercompat
I’d argue that Assistants API DX > manual completions API.
Re: Multi AI agent systems using OpenAI's assistants API
#35Earlier quoted context omitted.
Great points. Dont even get me started about how function calling in other LLMs costs me tokens. Something OpenAI provides OOTB. I'm also not a big fan of OpenAI's lock in. Right now I'm on a huge Claude 3 Haiku kick. That said, OpenAI does seem to get the APIs right and my hunch is the new Assistants API is going to potentially disrupt things again. Time will tell.
I would love to be using Claude, but you can't get API access (beyond an initial trial period) in the EU without providing a European VAT number. They don't want personal users or people to even learn and experiment I guess.
I'd guess it's more likely about the additional programming needed to meet GDPR compliance requirements.
Re: Multi AI agent systems using OpenAI's assistants API
#36Assistants API is promising, but earlier versions have many issues, especially with how it calculates the costs. As per OpenAI docs, you pay for data storage, a fixed price per API call, + token usage. It sounds straightforward until you start using it.
Here is how it works. When you upload attachments, in my case a very large PDF, it chunks that PDF into small parts and stores them in a vector database. It seems like the chunking part is not that great, as every time you make a call, the system loads a large chunk or many chunks and sends them to the model along with your prompt, which inflates your per request costs to 10 times more than the prompt + response tokens combined. So, be mindful of the hidden costs and monitor your usage.
Re: Multi AI agent systems using OpenAI's assistants API
#37I'd be interested in knowing if anyone is seriously using the assistants API, it feels like such a lock in to OpenAIs platform when your can alternatively just use completions that are much more easily interchanged.
I do and built Assistants API compat layer for Groq and Anthropic: https://github.com/supercorp-ai/supercompat I’d argue that Assistants API DX > manual completions API.
Aye, but your FinOps will be comolaining even with simple use.
Re: Multi AI agent systems using OpenAI's assistants API
#38Earlier quoted context omitted.
Great points. Dont even get me started about how function calling in other LLMs costs me tokens. Something OpenAI provides OOTB. I'm also not a big fan of OpenAI's lock in. Right now I'm on a huge Claude 3 Haiku kick. That said, OpenAI does seem to get the APIs right and my hunch is the new Assistants API is going to potentially disrupt things again. Time will tell.
> Dont even get me started about how function calling in other LLMs costs me tokens. Something OpenAI provides out of the box. Not sure what you mean by this.
I have some assumptions/guesses on how billing works. Gonna do a post on this on my unremarkable.ai blog, please do signup for posts there, no spam. I could be right or wrong but need to do some experiments and publish later.
Re: Multi AI agent systems using OpenAI's assistants API
#39Re: Multi AI agent systems using OpenAI's assistants API
#40Earlier quoted context omitted.
I do and built Assistants API compat layer for Groq and Anthropic: https://github.com/supercorp-ai/supercompat I’d argue that Assistants API DX > manual completions API.
Aye, but your FinOps will be comolaining even with simple use.
Assistants API use in prod used to suck because it would send full convo on each message. But last month they added an option to send truncted history so its no longer 2$ a pop thankfully. Also Grok, Haiku and Mistral is cheap