Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4o mini: advancing cost-efficient intelligence

openai.com

Post reply on HN
Print view View original

41–50 of 83 posts

Re: GPT-4o mini: advancing cost-efficient intelligence

#41

Post by jascha_eng »

delichon wrote: post #36
wrs wrote: post #20

The big news for me here is the 16k output token limit. The models keep increasing the input limit to outrageous amounts, but output has been stuck at 4k. I did a project to summarize complex PDF invoices (not “unstructured” data, but “idiosyncratically structured” data, as each vendor has a completely different format). GPT-4o did an amazing job at the extraction of line items, but I had to do a heuristic layer on t…

If all that AI could do was to turn less than structured data into structured data, it would still be the biggest deal in computation since the transistor.

But only if it could do it with reasonable accuracy. The problem is that AI is one of the few technologies that doesn't just fail to do it's job but it fails and you might never notice until the error is already very costly if it hallucinated something crazy.

Re: GPT-4o mini: advancing cost-efficient intelligence

#42

Post by pants2 »

This is awesome. I ran a query against a knowledge base that used to cost around $0.13 with 4o, now the cost doesn't even round to 1 cent, and the response is nearly as good.

I expect to make heavy use of this in my research-oriented agents, such as extracting relevant information from webpages to present to larger models.

Re: GPT-4o mini: advancing cost-efficient intelligence

#43

Post by monkeydust »

jascha_eng wrote: post #41
delichon wrote: post #36

Earlier quoted context omitted.

If all that AI could do was to turn less than structured data into structured data, it would still be the biggest deal in computation since the transistor.

But only if it could do it with reasonable accuracy. The problem is that AI is one of the few technologies that doesn't just fail to do it's job but it fails and you might never notice until the error is already very costly if it hallucinated something crazy.

Surely this is still a massive problem for any real world enterprise use case unless you throw a human in the loop (which kills the productivity benefit) or you stamp a massive disclaimer on the output

Re: GPT-4o mini: advancing cost-efficient intelligence

#44

Post by pants2 »

tedsanders wrote: post #21
minimaxir wrote: post #3

GPT-4o mini is $0.15/1M input tokens, $0.60/1M output tokens. In comparison, Claude Haiku is $0.25/1M input tokens, $1.25/1M output tokens. There's no way this price-race-to-the-bottom is sustainable.

Yeah, to put these prices in perspective: when tokens get this cheap, $1M buys you more than a trillion output tokens. To earn appreciable revenue at this price, an LLM company needs to be regularly generating multiple internets worth of text. On the one hand, generating multiple internets of text seems outlandish. But on the other hand, we're now approaching the point where you can start building LLMs into software…

I don't expect organizations to need to generate 1T output tokens, but 1T input tokens is common. Consider developers at a large company running queries with their entire codebase as context. Or lawyers plugging in the entire tax code to ask questions about. Each of them running dozens of queries per day on multi-millions of context input, it's going to add up quick.

Re: GPT-4o mini: advancing cost-efficient intelligence

#45

Post by [deleted] »

thatsnotmepls wrote: post #26
OutOfHere wrote: post #23

Earlier quoted context omitted.

> Take a loss on every sale and make up for it with volume! If you take a loss on every sale, it is impossible to make up for it with volume. The result will be a loss magnified by the volume.

Guess you missed the sarcasm.

[deleted]

Re: GPT-4o mini: advancing cost-efficient intelligence

#46

Post by joseda-hg »

One of the weirdest side efects of 4o vs 4, was single character "hallucinations" where a completely correct answer would be wrong specifically by a single character

I don't think I've seen anyone comment on it, but it was noticeable, specially when 4o was just released Has anyone noticed anything similar?

Re: GPT-4o mini: advancing cost-efficient intelligence

#47

Post by getcrunk »

How does this compare to sonnet 3.5? I’m seeing comparisons to haiku.

Very happy with the price. But it’s its slotting between 4o proper and 3.5 where is it in relation to 4? 4 was “just” good enough for my purposes

Edit: seems not too far off gpt 4o and sonnet 3.5 are very close and this mini is just a few percent below that

Re: GPT-4o mini: advancing cost-efficient intelligence

#48

Post by alexwebb2 »

joseda-hg wrote: post #46

One of the weirdest side efects of 4o vs 4, was single character "hallucinations" where a completely correct answer would be wrong specifically by a single character I don't think I've seen anyone comment on it, but it was noticeable, specially when 4o was just released Has anyone noticed anything similar?

Interesting. They switched to a new tokenizer for 4o and 4o-mini, so this might have the same issue.

Re: GPT-4o mini: advancing cost-efficient intelligence

#49

Post by GaggiX »

jascha_eng wrote: post #41
delichon wrote: post #36

Earlier quoted context omitted.

If all that AI could do was to turn less than structured data into structured data, it would still be the biggest deal in computation since the transistor.

But only if it could do it with reasonable accuracy. The problem is that AI is one of the few technologies that doesn't just fail to do it's job but it fails and you might never notice until the error is already very costly if it hallucinated something crazy.

>AI is one of the few technologies that doesn't just fail to do it's job but it fails and you might never notice until the error is already very costly if it hallucinated something crazy.

Because this is what is used to deal with non-formal and unstructured data, if you build something that would be always accurate to the task, then you would have solved it formally.

Re: GPT-4o mini: advancing cost-efficient intelligence

#50

Post by sauwan »

freediver wrote: post #38

Based on PyLLMs benchmark. [1] Slightly better than Haiku and slightly slower. Much cheaper. OpenAIProvider('gpt-4o-mini') Total Cost: 0.00385 | Aggregated speed: 105.72 tok/sec | Accuracy: 51.85% AnthropicProvider('claude-3-haiku-20240307') Total Cost: 0.00735 | Aggregated speed: 117.53 tok/sec | Accuracy: 48.15% [1] https://github.com/kagisearch/pyllms

How long before Anthropic releases Claude-3.5-Haiku at the same price with significantly better performance? OpenAI in trouble...
Post reply on HN