Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI: Streaming is now available in the Assistants API

platform.openai.com

Post reply on HN
Print view View original

31–40 of 89 posts

Re: OpenAI: Streaming is now available in the Assistants API

#31

Post by XCSme »

megous wrote: post #25

This website is now like 30% about this probability based autocomplete nonsense. Feels like all those bitcoin hypes and "running everything on blockchain" fad of few years ago. Now it's running everything through "large autocomplete" model. I really hope this will fade and focus will turn back to highlighting some broader actual human ingenuity in IT, rather than constant stream of "we used autocomplete for this new…

I just added this "autocomplete" in my app, and customers emailed to say they actually love it: https://docs.uxwizz.com/guides/ask-ai-new

Re: OpenAI: Streaming is now available in the Assistants API

#32

Post by slimsag »

potsandpans wrote: post #20

Openai banned my account for suspicious payment activities, and I never was able to talk to a real person. Just several layers of chat bots posing as people. I literally want to give them my money and can't. Every few weeks for shirts and giggles i send an email to them saying, "any update on this?"

Welcome to the future. You might be able to get an enterprise sales contract with human support.

Re: OpenAI: Streaming is now available in the Assistants API

#33

Post by [deleted] »

pedrovhb wrote: post #30

For all the brilliance in the AI and infra departments of OpenAI, their official Python library (which is the flagship one as I understand) feels pretty unidiomatic, designed without much thought for common patterns in the language. 2012 JavaScript called, it wants its callbacks wrapped in objects back. Why do we have a context manager named "stream" for which you call `.until_done()`? This could've been an iterator,…

[deleted]

Re: OpenAI: Streaming is now available in the Assistants API

#34

Post by dbish »

potsandpans wrote: post #20

Openai banned my account for suspicious payment activities, and I never was able to talk to a real person. Just several layers of chat bots posing as people. I literally want to give them my money and can't. Every few weeks for shirts and giggles i send an email to them saying, "any update on this?"

One of the reasons I tend to use any of their options through Azure where available. Azure support has a more straight forward (though still sometimes slow) process for account issues.

Re: OpenAI: Streaming is now available in the Assistants API

#35

Post by willsmith72 »

pedrovhb wrote: post #30

For all the brilliance in the AI and infra departments of OpenAI, their official Python library (which is the flagship one as I understand) feels pretty unidiomatic, designed without much thought for common patterns in the language. 2012 JavaScript called, it wants its callbacks wrapped in objects back. Why do we have a context manager named "stream" for which you call `.until_done()`? This could've been an iterator,…

Everything feels unidiomatic. The API design is bad, the frontends they build are horrific, reliability and availability are shocking.

And yet the AI is so good I put up with them everyday

If they ever grow into a proper product org they'll be unstoppable.

Re: OpenAI: Streaming is now available in the Assistants API

#37

Post by jilles »

pedrovhb wrote: post #30

For all the brilliance in the AI and infra departments of OpenAI, their official Python library (which is the flagship one as I understand) feels pretty unidiomatic, designed without much thought for common patterns in the language. 2012 JavaScript called, it wants its callbacks wrapped in objects back. Why do we have a context manager named "stream" for which you call `.until_done()`? This could've been an iterator,…

Probably written by GPT4

Re: OpenAI: Streaming is now available in the Assistants API

#38

Post by chaxor »

megous wrote: post #25

This website is now like 30% about this probability based autocomplete nonsense. Feels like all those bitcoin hypes and "running everything on blockchain" fad of few years ago. Now it's running everything through "large autocomplete" model. I really hope this will fade and focus will turn back to highlighting some broader actual human ingenuity in IT, rather than constant stream of "we used autocomplete for this new…

"old man yells at cloud"

Seriously though, it's not going away no matter how much anyone hates it. Emails and blogs will continue to be written with it, letters of recommendation will be/are written with it, Presidential speeches will be written with it, academic articles will be / are written with it (almost all ml and cs research is), news is written with it... It's not going to stop, but it will _probably_/_very likely_ get better.

There is no tool, no human, no method to determine if text is generated with one of these models at high F-score (only sometimes high precision, low recall domains for silly examples).

We're stuck with it. Like the English teacher and their despised spell check.

Re: OpenAI: Streaming is now available in the Assistants API

#39

Post by tristanz »

gtoubassi wrote: post #27
simonw wrote: post #24

Throwing a feature request in here just in case someone from OpenAI sees it. I'd really like it if the streaming versions of their APIs could return a token usage count at the end. The non-streaming APIs do this right now: curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" -d '{ "model": "gpt-3.5-turbo", "messages": [ { "role": "user", "c…

We do the token counting on our end literally just running tiktoken on the content chunks (although I think usually its one token per chunk). Its a bit annoying and I too expected they'd have the usage block but its one line of code if you already have tiktoken available. I've found the accounting on my side lines up well with what we see on our usage dashboard.

As an FYI, this is fine for rough usage, but it's not accurate. The OpenAI APIs inject various tokens you are unaware of into the input for things like function calling.

Re: OpenAI: Streaming is now available in the Assistants API

#40

Post by kfajdsl »

XCSme wrote: post #31
megous wrote: post #25

This website is now like 30% about this probability based autocomplete nonsense. Feels like all those bitcoin hypes and "running everything on blockchain" fad of few years ago. Now it's running everything through "large autocomplete" model. I really hope this will fade and focus will turn back to highlighting some broader actual human ingenuity in IT, rather than constant stream of "we used autocomplete for this new…

I just added this "autocomplete" in my app, and customers emailed to say they actually love it: https://docs.uxwizz.com/guides/ask-ai-new

I've gotten so used to having an LLM integrated into my editor that when I work on the occasional spreadsheet (or really anything with syntax that I only use occasionally and no integrated AI) it's pretty jarring to have to go to another tab to look up what function to use for a formula (even if that other tab is ChatGPT).
Post reply on HN