Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Function calling and other API updates

openai.com

Post reply on HN
Print view View original

1–10 of 174 posts

Re: Function calling and other API updates

#4

Post by ignoramous »

eminence32 wrote: post #2

The changes/improvements to `gpt-3.5-turbo` are very welcome, considering how hard it is to get access to gpt-4

The blog post indicates that GPT-4 (0613) will be generally available soon.

  gpt-4-32k-0613 includes an extended context length for better comprehension of larger texts.
  
  With these updates, we'll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model.

Re: Function calling and other API updates

#7

Post by minimaxir »

The big feature here is the function calls, as this is effectively a replacement for the "Tools" feature of Agents popularized by LangChain, except in theory much more efficient since it may not require an extra call to the API. In the case of LangChain which selects Tools and their functional outputs through JSON Markdown shennanigans (which often fails and causes ParsingErrors), this variant of ChatGPT appears to be finetuned for it so perhaps it'll be more reliable.

While developing a more-simple LangChain alternative (https://github.com/minimaxir/simpleaichat) I discovered a neat trick for allowing ChatGPT to select tools from a list reliably: put the list of tools into a numbered list, and force the model to return only a single number by using the logit_bias parameter: https://github.com/minimaxir/simpleaichat/blob/main/PROMPTS....

The slight price drop for ChatGPT inputs is of course welcome, since inputs are the bulk of the costs for longer conversations. A 4x context window at 2x the price is a good value too. The notes for the updated ChatGPT also say "more reliable steerability via the system message" which will also be huge if it works as advertised.

Re: Function calling and other API updates

#8

Post by Escapado »

This is great. I wonder if the price decrease comes from the competition (on the side of anthropic and from local LLMs). If so, I guess we will have to wait for a general GPT-4 competitor to come along before we see price decreases there aswell. Right now it's quiet expensive. We are incorporating it in a new product in the education space and we have to be fairly conservative in rate limiting things so that the cost won't go out of hand.

I also wonder how much of an impact the new Nvidia HGX systems will have on the medium term infra cost on running these services and whether we will see some benefits from that.

Re: Function calling and other API updates

#10

Post by misterdata »

minimaxir wrote: post #7

The big feature here is the function calls, as this is effectively a replacement for the "Tools" feature of Agents popularized by LangChain, except in theory much more efficient since it may not require an extra call to the API. In the case of LangChain which selects Tools and their functional outputs through JSON Markdown shennanigans (which often fails and causes ParsingErrors), this variant of ChatGPT appears to b…

As they are accepting a JSON schema for the function calls, it is likely they are using token biasing based on the schema (using some kind of state machine that follows along with the tokens and only allows the next token to be a valid one given the grammar/schema). I have successfully implemented this for JSON Schema (limited subset) on llama.cpp. See also e.g. this implementation: https://github.com/1rgs/jsonformer
Post reply on HN