Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Function calling and other API updates

openai.com

Post reply on HN
Print view View original

121–130 of 174 posts

Re: Function calling and other API updates

#121

Post by newhouseb »

civilitty wrote: post #23
newhouseb wrote: post #17

Earlier quoted context omitted.

As someone also building constrained decoders against JSON [1], I was hopeful to see the same but I note the following from their documentation: The model can choose to call a function; if so, the content will be a stringified JSON object adhering to your custom schema (note: the model may generate invalid JSON or hallucinate parameters). So sadly, it is just fine tuning. There's no hard biasing applied :(. You were…

Or there’s a trade off between more complex schemas and logit bias going off the rails since there’s probably little to no backtracking.

Good point. Backtracking is certainly possible but it is probably tricky to parallelize at scale if you're trying to coalesce and slam through a bunch of concurrent (unrelated) requests with minimal pre-emption.

Re: Function calling and other API updates

#123

Post by roughly »

tmaly wrote: post #122

> note: the model may generate invalid JSON or hallucinate parameters This seems like it could be a really big problem for this feature.

The Langchain package does something similar to this and has a feature to combat that kind of thing that basically feeds the invalid input back into the bot and says “you were asked for x but gave this, please correct it” - it’s shockingly effective.

Re: Function calling and other API updates

#124

Post by dumpsterdiver »

ramraj07 wrote: post #118

> With this capability also comes potential risks. We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc). Yeah thanks for the heads up Sam.

"When performing actions such as sending emails or posting comments on internet forums, the language learning model may mistakenly attempt to take control of powerful weaponry and engage with humans in physical warfare. For this reason we strongly suggest building in user confirmation prompts for such functionality."

Re: Function calling and other API updates

#125

Post by dumpsterdiver »

dumpsterdiver wrote: post #124
ramraj07 wrote: post #118

> With this capability also comes potential risks. We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc). Yeah thanks for the heads up Sam.

"When performing actions such as sending emails or posting comments on internet forums, the language learning model may mistakenly attempt to take control of powerful weaponry and engage with humans in physical warfare. For this reason we strongly suggest building in user confirmation prompts for such functionality."

Bonus points if the LLM replies sinisterly with something like, "As a language learning model I am not legally liable for any property damage, loss of life, or sudden regime changes that may occur due to receiving user input. Are you sure that you would like to send this email?"

Re: Function calling and other API updates

#126

Post by sp332 »

egonschiele wrote: post #71
bioemerl wrote: post #67

Earlier quoted context omitted.

This is exactly my problem. They are doing quite well, and closing the door behind them. Open AI isn't your friend and reserves the right to screw you down the line.

Oh, I 100% agree. I just haven't seen another model come close. I'd love to hear someone tell me why.

Lots and lots of human labor. https://gizmodo.com/chatgpt-openai-ai-contractors-15-dollars...

Re: Function calling and other API updates

#127

Post by tmaly »

I was looking at the models. I noticed the function calling for 3.5 is only with the 4,096 token length. The larger 16k context length for 3.5 does not mention it despite having a similar naming of 0613 suffix.

https://platform.openai.com/docs/models/gpt-3-5

gpt-3.5-turbo-0613 Snapshot of gpt-3.5-turbo from June 13th 2023 with function calling data. Unlike gpt-3.5-turbo, this model will not receive updates, and will be deprecated 3 months after a new version is released. 4,096 tokens Up to Sep 2021

gpt-3.5-turbo-16k-0613 Snapshot of gpt-3.5-turbo-16k from June 13th 2023. Unlike gpt-3.5-turbo-16k, this model will not receive updates, and will be deprecated 3 months after a new version is released. 16,384 tokens Up to Sep 2021

Re: Function calling and other API updates

#128

Post by muzani »

com2kid wrote: post #12

GPT3.5 has been undergoing constant improvements, this price decrease (and context length increase) is great news! The main problem I see with people using GPT3.5 is they try and ask it to "write a short story about aliens" and then they get back a crap boring response that sounds like it was written by an AI that was asleep at the wheel. Good creative prompts are long and detailed, and to get the best results you re…

Honestly, the good old davinci model has proven to be much better at writing for me. 3.5 feels overtrained.

You also have to give it a good sample of how to write, else it will write at the average quality of fiction that it has been fed.

Here's something by GPT-4: https://chat.openai.com/share/ab2fc479-f3f9-4bf9-b625-e2aab7...

The same prompt with GPT-3.5: https://chat.openai.com/share/be81167d-11eb-4f38-b1b4-b6f592...

The original plot was actually generated by davinci, which I think is the most creative of the three. 3.5 for price and speed, GPT-4 has rationality and experience, and davinci has his head up in the cloud.

Re: Function calling and other API updates

#129

Post by taf2 »

16k is tokens on 3.5 is amazing but they also complicated the pricing from a simple flat per 1k token to two separate fees for input and output … I think it’s better maybe but token based pricing is challenging to reason with and even more so to explain to customers
Post reply on HN