Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT4 is up to 6 times more expensive than GPT3.5

openai.com

Post reply on HN
Print view View original

51–60 of 136 posts

Re: GPT4 is up to 6 times more expensive than GPT3.5

#51

Post by sebzim4500 »

cscheid wrote: post #42
sebzim4500 wrote: post #38

Why is the 32K context only twice as expensive as the 8K context? Are they using sparse attention or something? I don't think flash attention on it's own can explain it. EDIT: Oh right, if the cost is per token so if you actually fill the context then it is 8x more, which makes much more sense.

Ask it the other way around, "Why is the 8K context only twice as cheap as the 32K context?" and your original answer is clearer: because they think the demand curve supports it. Price is not determined by cost, but by how much people are willing to pay.

No I'm just an idiot. Since the price is per token the actual ratio is more like 8x more (assuming you fill both contexts) which is plausibly the ratio between the costs.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#52

Post by Mizza »

kenjackson wrote: post #33
Mizza wrote: post #24

Earlier quoted context omitted.

Thanks for the reply. I'm not all that impressed by "what is"/information retrieval tasks, is it any better at "thought" tasks like explanatory reasoning and ideogensis? For instance - "How might an agoraphobic kleptomaniac satisfy both of their desires simultaneously?"

That's a tough question even for me. The answer it did give me was to steal something online. Which was better than anything I was thinking. What did you have in mind?

Ah, interesting! It's a tough question and not one that has an obvious answer, which is what makes it a good test, it requires a little creativity. In my tests, ChatGPT/3 can't/won't answer it.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#53

Post by JohnFen »

djbusby wrote: post #43
JohnFen wrote: post #21

Earlier quoted context omitted.

The problem with the name isn't that they're charging. It's that they're implying that they're something in the ballpark of open source when they're not.

OpenTable isn't Open Source. Open, as a word, has more than one context.

But OpenTable was never open source to begin with, so that association was never made.

Open AI started with a commitment to being open source (thus the "open"), but then they changed their minds and went closed source. I think in this context, keeping the "Open" in their name is a bit deceptive.

It's not the biggest issue in the world, admittedly, but it does leave a bad taste in the mouth.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#54

Post by sebzim4500 »

djbusby wrote: post #45
kenjackson wrote: post #33

Earlier quoted context omitted.

That's a tough question even for me. The answer it did give me was to steal something online. Which was better than anything I was thinking. What did you have in mind?

Steal from a friends house?

You'd still have to go outside to get to said friends house.

I think GPT4 gave the better answer here.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#55

Post by caturopath »

Mizza wrote: post #13

Does anybody have their hands on it? Is it actually 6 times better at performing tasks, or are we paying more for improved corporate-friendly bumper bowling?

It's difficult to be sure exactly what six times "better" means, but I wouldn't expect something to have to be six times better necessarily to be six times better to be six times more valuable. I wouldn't expect a pitcher making $30MM/yr to be 6x better than one making $5MM/yr, but I could buy that they were 6x more valuable to have on your team.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#56

Post by djbusby »

JohnFen wrote: post #53
djbusby wrote: post #43

Earlier quoted context omitted.

OpenTable isn't Open Source. Open, as a word, has more than one context.

But OpenTable was never open source to begin with, so that association was never made. Open AI started with a commitment to being open source (thus the "open"), but then they changed their minds and went closed source. I think in this context, keeping the "Open" in their name is a bit deceptive. It's not the biggest issue in the world, admittedly, but it does leave a bad taste in the mouth.

Oh! that's new information for me. That case does feel like a -n-switch.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#58

Post by evilduck »

djbusby wrote: post #43
JohnFen wrote: post #21

Earlier quoted context omitted.

The problem with the name isn't that they're charging. It's that they're implying that they're something in the ballpark of open source when they're not.

OpenTable isn't Open Source. Open, as a word, has more than one context.

OpenTable references restaurants tables and times that are still open to be reserved.

What contextual definition of "open" does OpenAI have? A website open to subscribers?

Re: GPT4 is up to 6 times more expensive than GPT3.5

#59

Post by Maxion »

sebzim4500 wrote: post #51
cscheid wrote: post #42

Earlier quoted context omitted.

Ask it the other way around, "Why is the 8K context only twice as cheap as the 32K context?" and your original answer is clearer: because they think the demand curve supports it. Price is not determined by cost, but by how much people are willing to pay.

No I'm just an idiot. Since the price is per token the actual ratio is more like 8x more (assuming you fill both contexts) which is plausibly the ratio between the costs.

In the end the cost for you is how many tokens consumed, if you can get more value out of one request with 32k context that multiple with lower, then it’s cheaper (and faster).

Some problems also require a larger context to be able to solve.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#60

Post by Der_Einzige »

int_is_compress wrote: post #48
Maxion wrote: post #28

Earlier quoted context omitted.

GPT-4 is a scary improvement over 3.5, especially for handling code. It will be the literal definition of awesome when these models get a large enough context space to hold a small-medium sized codebase. I've been playing around with it for an hour seeing what it can do to refactor some of the things we have with the most tech debt, and it is astounding how well it does with how little context I give it.

There are already some cool projects that help LLM go beyond the context window limitation and work with even larger codebases like https://github.com/jerryjliu/llama_index and https://github.com/hwchase17/langchain .

The fundamental techniques that they use are highly lossey and are far inferior to ultra-long context length models where you can do it all in one prompt. Hate to break it to you and all the others.
Post reply on HN