Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Function calling and other API updates

openai.com

Post reply on HN
Print view View original

31–40 of 174 posts

Re: Function calling and other API updates

#31

Post by ilaksh »

Escapado wrote: post #8

This is great. I wonder if the price decrease comes from the competition (on the side of anthropic and from local LLMs). If so, I guess we will have to wait for a general GPT-4 competitor to come along before we see price decreases there aswell. Right now it's quiet expensive. We are incorporating it in a new product in the education space and we have to be fairly conservative in rate limiting things so that the cost…

Right this is a very exciting release but disappointing that there was no price reduction at all or rate limit increase for gpt-4. I guess it just uses a lot of GPU and RAM.

Re: Function calling and other API updates

#33

Post by kgwgk »

wellthisisgreat wrote: post #28
sashank_1509 wrote: post #18

Earlier quoted context omitted.

I see this statement a lot and have no idea how people come to this conclusion. I have a beefy 16k$ workstation with 2 4090s and I could barely run the LLAMA 65B model at a very slow pace. Let us say we do have the model weights to GPT-4 and GPT3.5, me as the average consumer I don't know how this helps me in any way. I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even i…

> LLAMA 65B model at a very slow pace How does it compare to GPT 3.5, or 4? I mean if you ask the same questions. Is it usable at all? I tried the models that work with 4090 and they were completely useless for anything practical (code questions, etc.). Curiosities sure, but on Eliza level.

Is there a simple question / answer that you would find illuminating?

Re: Function calling and other API updates

#34

Post by TheCaptain4815 »

com2kid wrote: post #12

GPT3.5 has been undergoing constant improvements, this price decrease (and context length increase) is great news! The main problem I see with people using GPT3.5 is they try and ask it to "write a short story about aliens" and then they get back a crap boring response that sounds like it was written by an AI that was asleep at the wheel. Good creative prompts are long and detailed, and to get the best results you re…

Could you share any tips? I'm always looking to learn more on Prompting, especially with 3.5.

Re: Function calling and other API updates

#35

Post by kgwgk »

ilaksh wrote: post #31
Escapado wrote: post #8

This is great. I wonder if the price decrease comes from the competition (on the side of anthropic and from local LLMs). If so, I guess we will have to wait for a general GPT-4 competitor to come along before we see price decreases there aswell. Right now it's quiet expensive. We are incorporating it in a new product in the education space and we have to be fairly conservative in rate limiting things so that the cost…

Right this is a very exciting release but disappointing that there was no price reduction at all or rate limit increase for gpt-4. I guess it just uses a lot of GPU and RAM.

Why would they reduce the price when they have a waiting list?

Re: Function calling and other API updates

#36

Post by SkyPuncher »

civilitty wrote: post #13

Finally! I’ve been getting the shakes waiting for next OpenAI release. 16k context with 3.5-turbo is huge. It’ll make all those dime a dozen document driven assistants a lot more useful. I’m curious to see if people will figure out ways to hack functions to get more reliable structured JSON data out out of GPT without tons of examples, giving lots more context room to play with

This is awesome. We were finding a lot of frustrating with 4k context being far too short to properly chunk documents.

In a worst case scenario, you have to assume that output is going to be the same length as input. That means useful context is actually half of the total context.

Add in a bit of fixed size for chunking/overlap (maybe ~500 tokens), suddenly you're looking at only 1k to 1.5k being reliably available for input. 16k context bumps that number up to 7.5k available for input. That's massive.

Re: Function calling and other API updates

#39

Post by dragonwriter »

ilaksh wrote: post #31
Escapado wrote: post #8

This is great. I wonder if the price decrease comes from the competition (on the side of anthropic and from local LLMs). If so, I guess we will have to wait for a general GPT-4 competitor to come along before we see price decreases there aswell. Right now it's quiet expensive. We are incorporating it in a new product in the education space and we have to be fairly conservative in rate limiting things so that the cost…

Right this is a very exciting release but disappointing that there was no price reduction at all or rate limit increase for gpt-4. I guess it just uses a lot of GPU and RAM.

> Right this is a very exciting release but disappointing that there was no price reduction at all or rate limit increase for gpt-4

They are planning on reducing the pricing from $infinite to $current-listed (or, viewed another way, to increase the quota from 0 to current-listed) by clearing the waiting list.

This, obviously, doesn’t benefit (may even, competitively, hurt) those who already have GPT-4 access, but for everyone else, its a win.

Re: Function calling and other API updates

#40

Post by MuffinFlavored »

I don't understand it. You tell the gpt-4 session a function exists and supply the signature, but I don't see any URLs being defined on where the APIs live?

> 1. Call the model with functions and the user’s input

> 2. Use the model response to call your API

I don't get this part. Where do you tell it where the API lives?

Post reply on HN