Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4o mini: advancing cost-efficient intelligence

openai.com

Post reply on HN
Print view View original

31–40 of 83 posts

Re: GPT-4o mini: advancing cost-efficient intelligence

#31

Post by laborcontract »

skybrian wrote: post #22
razodactyl wrote: post #12

Here's something interesting to think about: In ML we do a lot of bootstrapping. If a model is 51% wrong on a binary problem you flip the answer and train a 51% correct model then work your way up from there. Small models are trained from synthetic and live data curated and generated by the more advanced models. If I end up with an advanced smaller model capable of running alongside a greater number of similar models…

I’m a little skeptical of processes that seem to create more information than you had to start with. For a game like chess or Go, it makes sense, because winning strategies are implicit in the rules of the game, but it takes a lot of computation to discover the consequences. Similarly for math where theorems are non-obvious consequences of axioms. And computer code can be similar to math. But how does that work for a…

The larger models generate high quality textbook-like synthetic data which is used to develop the model's reasoning skills. Microsoft's Phi series is a demonstration of this. These models do not have the ability to absorb and retain a lot of factual knowledge due to the low parameter count. However, they do have the ability to reason as well as larger models, which means these models perform best when most of the factual stuff is provided in context.

Re: GPT-4o mini: advancing cost-efficient intelligence

#32

Post by GaggiX »

>In pre-training, we filter out(opens in a new window) information that we do not want our models to learn from or output, such as hate speech, adult content, sites that primarily aggregate personal information, and spam.

Great so now the model would be unable to recognize this type of content, do not use it for moderation.

Re: GPT-4o mini: advancing cost-efficient intelligence

#33

Post by hayksaakian »

petercooper wrote: post #15
k2xl wrote: post #8

This is great - Though I am confused on two things: 1. How is it possible that GPT-4o mini outperforms 3.5 turbo but 3.5 turbo is more expensive? Like why would someone use a worse model and pay more? 2. Why is the GPT4o vision and GPT4o-mini vision cost the same?

I might be wrong, but I've inferred from OpenAI's pricing behavior that they use it to encourage people to migrate to more efficient models. The 3.5 Turbo pricing is maintained to encourage you to stop using it. Look at davinci-002's pricing, for example - it's very high for something that's relatively ancient.

exactly. the only people who would use 3.5 now are people who MUST use it due to some specification, contract or requirement.

You can charge a premium to people who aren't allowed to change their mind.

Re: GPT-4o mini: advancing cost-efficient intelligence

#34

Post by mechagodzilla »

quotemstr wrote: post #5
minimaxir wrote: post #3

GPT-4o mini is $0.15/1M input tokens, $0.60/1M output tokens. In comparison, Claude Haiku is $0.25/1M input tokens, $1.25/1M output tokens. There's no way this price-race-to-the-bottom is sustainable.

> There's no way this price-race-to-the-bottom is sustainable. Why not?

Well each new generation of model costs like 10x the previous one to train, and its value (and thus ability to generate a return) diminishes extremely rapidly. The only source of improved economics is the rapidly evaporating Moore's Law (and any opex savings are swamped by the crazy high capex if you're using chips from Nvidia).

Re: GPT-4o mini: advancing cost-efficient intelligence

#35

Post by pants2 »

laborcontract wrote: post #27
razodactyl wrote: post #12

Here's something interesting to think about: In ML we do a lot of bootstrapping. If a model is 51% wrong on a binary problem you flip the answer and train a 51% correct model then work your way up from there. Small models are trained from synthetic and live data curated and generated by the more advanced models. If I end up with an advanced smaller model capable of running alongside a greater number of similar models…

Sounds like you're describing mixture of experts, the architecture being used in openai's gpt-4 and mistral's mixtral series of models.

Not really, MoE is trained all at once and the 'experts' don't have pre-defined specializations. They end up being more like "punctuation expert" and "pronoun expert" than "math expert" and "french expert"

Re: GPT-4o mini: advancing cost-efficient intelligence

#36

Post by delichon »

wrs wrote: post #20

The big news for me here is the 16k output token limit. The models keep increasing the input limit to outrageous amounts, but output has been stuck at 4k. I did a project to summarize complex PDF invoices (not “unstructured” data, but “idiosyncratically structured” data, as each vendor has a completely different format). GPT-4o did an amazing job at the extraction of line items, but I had to do a heuristic layer on t…

If all that AI could do was to turn less than structured data into structured data, it would still be the biggest deal in computation since the transistor.

Re: GPT-4o mini: advancing cost-efficient intelligence

#37

Post by laborcontract »

k2xl wrote: post #8

This is great - Though I am confused on two things: 1. How is it possible that GPT-4o mini outperforms 3.5 turbo but 3.5 turbo is more expensive? Like why would someone use a worse model and pay more? 2. Why is the GPT4o vision and GPT4o-mini vision cost the same?

regarding 1, they have a strong understanding of the tasks/queries their users are performing and they are pruning the model accordingly. It's like playing jenga but with neurons.

Re: GPT-4o mini: advancing cost-efficient intelligence

#38

Post by freediver »

Based on PyLLMs benchmark. [1]

Slightly better than Haiku and slightly slower. Much cheaper.

OpenAIProvider('gpt-4o-mini') Total Cost: 0.00385 | Aggregated speed: 105.72 tok/sec | Accuracy: 51.85%

AnthropicProvider('claude-3-haiku-20240307') Total Cost: 0.00735 | Aggregated speed: 117.53 tok/sec | Accuracy: 48.15%

[1] https://github.com/kagisearch/pyllms

Re: GPT-4o mini: advancing cost-efficient intelligence

#39

Post by laborcontract »

pants2 wrote: post #35
laborcontract wrote: post #27

Earlier quoted context omitted.

Sounds like you're describing mixture of experts, the architecture being used in openai's gpt-4 and mistral's mixtral series of models.

Not really, MoE is trained all at once and the 'experts' don't have pre-defined specializations. They end up being more like "punctuation expert" and "pronoun expert" than "math expert" and "french expert"

Haven't tried any yet, but it sounds like parent may be interested in an LLM router. https://github.com/lm-sys/RouteLLM

Re: GPT-4o mini: advancing cost-efficient intelligence

#40

Post by Workaccount2 »

minimaxir wrote: post #3

GPT-4o mini is $0.15/1M input tokens, $0.60/1M output tokens. In comparison, Claude Haiku is $0.25/1M input tokens, $1.25/1M output tokens. There's no way this price-race-to-the-bottom is sustainable.

They're building a beautiful garden with rich soil and generous watering. In fact it is so wonderful that you'd love to grow your product there. A product with deep roots and symbiotic neighbors.

Just be careful when they start building the walls. And they will build those walls.

Post reply on HN