Live data from Hacker News

GPT4 is up to 6 times more expensive than GPT3.5

openai.com

101–110 of 136 posts

Re: GPT4 is up to 6 times more expensive than GPT3.5

#101
post #95
post #28

Earlier quoted context omitted.

GPT-4 is a scary improvement over 3.5, especially for handling code. It will be the literal definition of awesome when these models get a large enough context space to hold a small-medium sized codebase. I've been playing around with it for an hour seeing what it can do to refactor some of the things we have with the most tech debt, and it is astounding how well it does with how little context I give it.

The only thing holding this back now is lack of enough context. That’s the big nut to crack. How do you hold enough information in memory at once?

GPT4 supports 32k tokens, which I guesstimate would be ~25k code tokens and perhaps ~2.5k lines of code. So already enough to work on a whole module of code at the same time. If you're using small microservices, it might already be good enough.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#102
post #49

Earlier quoted context omitted.

I’m not even kidding when I say I just did 6h or so of work in around 45 minutes

I'm worried it's going to decimate the already wobbly jr market - who are the sr devs of tomorrow to wrangle the LLM output after it replaces the jr's job of coding all day?

Certainly it will.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#103
If this is public I can only imagine what Google has internally. Does DeepMind compete in this space or is it fundamentally different than a llm like Bard?

With their massive codebase and already deep investment in AI/ML, I'm pretty sure Google and likely MS already have the ability to do massive refactoring, validate it using tests, reiterate, train, rinse and repeat.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#104

Earlier quoted context omitted.

The ChatGPT API is using a smaller model than the original "GPT 3.5". The original, presumably larger, model is used by Legacy ChatGPT.

Wait is that true? I couldn't find anything that verifies this. Do you have a source? If it is in fact true, doesn't it fall into "false advertisement" category? That is, OpenAI using ChatGPT for demonstration purposes but then charging for an API that pretends to be ChatGPT but is in fact based on a much smaller model?

The normal ChatGPT interface also switched to the cheaper model, and lots of people complained when they switched (though not enough to make a big media splash). With ChatGPT Pro you can select between the new and the "Legacy" model (and now GPT4 as a third option)

Re: GPT4 is up to 6 times more expensive than GPT3.5

#105
post #103

If this is public I can only imagine what Google has internally. Does DeepMind compete in this space or is it fundamentally different than a llm like Bard? With their massive codebase and already deep investment in AI/ML, I'm pretty sure Google and likely MS already have the ability to do massive refactoring, validate it using tests, reiterate, train, rinse and repeat.

> do massive refactoring, validate it using tests

Google do have damn good mass code change tools, for human, without AI.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#106
post #87

Earlier quoted context omitted.

Keep in mind OpenAI is able to lose money on GPT*. Inference for the same predictive power will indeed improve (Facebook's recent model is a step in the right direction) but efficiency of running on given hardware likely won't follow moore's law for long.

Are there truely top tier ML optimized chips? I would imagine the hardware space still having room to grow if just because most of the brainpower having been focused on x86, etc.

If you believe Nvidia market speak, they took AI and used it to optimize their A100 chips when creating the H100 line, so we're already using artificial brainpower on the problem.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#107

Designer leather jackets boutiques world wide are rejoicing.

I remember as a young gamer, asking my Dad whether Nvidia or Intel was a bigger company. It seemed obvious to me, since people always bragged about their GPUs online, but never the CPU, that Nvidia is the more important company. At that time, Nvidia was worth like 1% of Intel. Today, regarding online discussions, nothing has changed. People still brag about GPUs, never the CPU. But now Nvidia is worth 5 times as much…

I feel like this is comparing apples and oranges a bit, depending on when you were a kid. Even within the GPU market, Nvidia wasn't a huge player for a while. It's not just that GPUs are more important today, but that Nvidia went from being a small player in the GPU market to essentially being the GPU market. At the same time, Intel's presence in the CPU market has never been lower. Most "computers" are now smartphones running ARM processors, data centers are using AArch64 more and more, there's a slowdown in businesses and consumers buying computers, and AMD is taking a huge share of the X64 market.

Likewise, Nvidia is priced at 102x earnings while Intel is priced at 14x earnings. Some of that might be that people are anticipating the GPU market to be where all the money is in the future, but a decent amount of that is pessimism about Intel's place in the market.

You're not even comparing the CPU market to the GPU market. You're comparing what was possibly the 3rd ranked GPU marker when you were a kid and Intel when you were a kid dominating with 90-95% of CPU sales with Nvidia being 90% of the GPU market's profits today and Intel being more of a bit player in the CPU market.

Some of Nvidia's size today is because they beat all the other graphics card companies (yes, ATI/AMD is still around, but Nvidia's marketshare is above 85%). Some of that comes from GPUs becoming a much bigger factor in many things like AI. But when comparing Intel and Nvidia today, Intel is still getting double the revenue and profits of Nvidia. This is despite the fact that most CPUs have moved away from their architecture, despite the fact that they've had at least half a decade of terrible engineering performance, despite AMD starting to kick their butt and taking serious marketshare on their architecture, etc.

I don't want to say that GPUs aren't important, but Nvidia being worth more than Intel probably says more about Nvidia executing well while Intel stumbles than anything else. If you were asking your father this in 1998, Nvidia was just releasing the RIVA TNT, 3dfx was still king, and Intel was so dominant it looked like they'd be unstoppable forever. Now you're looking at an Intel that doesn't even look like the king of their own architecture, has been having foundry issues for years, and an Nvidia that no other GPU maker has been able to touch. If another GPU maker beats Nvidia and Nvidia becomes smaller than Intel, does that mean that GPUs aren't important any more? No.

The problem isn't your conclusion that GPUs are important today, but the logic getting there. People bragging about things isn't necessarily a great barometer for future success. Lots of things are trendy and lots of companies come and go. Everyone was talking about Pogs and Beanie Babies at one point. That hasn't made them last. People were talking about smartphone/handheld devices since the 80s, but almost all of those companies aren't in that market today (Palm, Handspring, Blackberry, Microsoft). Instead, Apple won that market. Heck, you could have asked your father if 3dfx were bigger than Intel and seen them go bankrupt a few years later. Where's Matrox and S3? This is an anecdote that just feels like survivorship bias. Yep, you asked about the GPU maker that won the market and the CPU maker that would face big declines (and still Intel's revenue/profits are double Nvidia's).

Re: GPT4 is up to 6 times more expensive than GPT3.5

#108

Earlier quoted context omitted.

OpenTable references restaurants tables and times that are still open to be reserved. What contextual definition of "open" does OpenAI have? A website open to subscribers?

An open API you can build products off of. Consider the alternative. They could not let anyone use their models and just release AI products (their own search engine, their own version of Teams, their own self driving car software, their own photoshop competitor, etc).

This is revisionist history. OpenAI very much started as a (somewhat) open source (for whatever definition of "open" makes sense for LLMs) initiative.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#109

Earlier quoted context omitted.

The title compares GPT3.5 (not ChatGPT) with GPT4. Although the question is: If ChatGPT can do almost anything GPT3.5 can do, then why is it priced almost 10x less? Is it because GPT3.5 is less "censored"?

If I were to speculate, OpenAI is doing two types of optimizations: 1) Cost 2) Function I suspect ChatGPT is 90% a cost-optimized version of GPT3, and 10% a function-enhanced one. GPT3 was the first major public wow-able model, and I suspect it was not running on any sort of optimized infrastructure.

yeah there is no way they'd want that kind of traffic unless the expense knobs were turned down.

you might have 10 million queries on this thing per day. if it's just 1c/query that's already $100k/day.

Re: GPT4 is up to 6 times more expensive than GPT3.5

#110
post #50

Earlier quoted context omitted.

I think he means it was 30x more than gpt-3.5 is today.

Yeah but that's also 30x times worse. ChatGPT is much worse-performing than davinci-003.

No, something’s off, and I’m interested in finding out what
Post reply on HN