OpenAI: Streaming is now available in the Assistants API

Finally! I've been using the assistants api in building an ai mock interviewer (https://comp.lol) but the responses were painfully slow when using the latest iterations of the gpt-4 model. This will make things so much more responsive

Re: OpenAI: Streaming is now available in the Assistants API

#7

Post by cosmotic » Thu, Mar 14, 2024, 12:17 AM UTC

andher wrote: post #6

Finally! I've been using the assistants api in building an ai mock interviewer ( https://comp.lol ) but the responses were painfully slow when using the latest iterations of the gpt-4 model. This will make things so much more responsive

I'd still want to see the entire response all at once. Having it stream in while I read it would be very distracting and make it difficult for me to read.

Re: OpenAI: Streaming is now available in the Assistants API

#8

Post by m-p-3 » Thu, Mar 14, 2024, 12:17 AM UTC

I thought this was about making the OpenAI app available as a digital assistant on Android, as a replacement to Google.

Oh well..

Re: OpenAI: Streaming is now available in the Assistants API

#9

Post by XCSme » Thu, Mar 14, 2024, 12:51 AM UTC

Any way to have a consistent system prompt across queries without sending it (and using tokens) for each completion?

Re: OpenAI: Streaming is now available in the Assistants API

#10

Post by eightysixfour » Thu, Mar 14, 2024, 12:53 AM UTC

XCSme wrote: post #9

Any way to have a consistent system prompt across queries without sending it (and using tokens) for each completion?

The Assistant API handles that, it has the system prompt as part of the assistant that you interact with.