Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4o mini: advancing cost-efficient intelligence

openai.com

Post reply on HN
Print view View original

71–80 of 83 posts

Re: GPT-4o mini: advancing cost-efficient intelligence

#71

Post by tempaccount420 »

maeil wrote: post #70

So far ever since the initial release of gpt 3.5 turbo every ""upgrade"" has mostly been an actual downgrade. I have a battery of tasks that the initial 3.5 turbo (Nov 2022) was able to perform but the newer ones very consistently fail at, regardless of prompting. I've been moving tasks from 3.5-turbo to Llama3-70b for this reason. Very curious to see whether this time it'll be an actual upgrade instead of a downgrad…

The original GPT-4 was an upgrade IMO. GPT-4 Turbo and GPT-4o were downgrades. GPT-4o seems especially bad (on text-to-text).

Re: GPT-4o mini: advancing cost-efficient intelligence

#72

Post by raxxorraxor »

delichon wrote: post #36
wrs wrote: post #20

The big news for me here is the 16k output token limit. The models keep increasing the input limit to outrageous amounts, but output has been stuck at 4k. I did a project to summarize complex PDF invoices (not “unstructured” data, but “idiosyncratically structured” data, as each vendor has a completely different format). GPT-4o did an amazing job at the extraction of line items, but I had to do a heuristic layer on t…

If all that AI could do was to turn less than structured data into structured data, it would still be the biggest deal in computation since the transistor.

Giving an LLM any task involving numbers is quite a gamble. Still, I guess structuring content is exactly where I assume many practical applications lie, perhaps just as a preprocessor. You just need a way to validate the results...

Re: GPT-4o mini: advancing cost-efficient intelligence

#73

Post by rfoo »

mechagodzilla wrote: post #34
quotemstr wrote: post #5

Earlier quoted context omitted.

> There's no way this price-race-to-the-bottom is sustainable. Why not?

Well each new generation of model costs like 10x the previous one to train, and its value (and thus ability to generate a return) diminishes extremely rapidly. The only source of improved economics is the rapidly evaporating Moore's Law (and any opex savings are swamped by the crazy high capex if you're using chips from Nvidia).

> rapidly evaporating Moore's Law

Algorithm (no, I don't mean Mamba etc, you can still use decoder-only transformers with some special attention layers) and engineering side there's still at least 10x improvement possible. Compared to what TensorRT-LLM is able to achieve now.

My concern is, this is only possible because of scale, so local LLMs are going to be dead in the water.

Re: GPT-4o mini: advancing cost-efficient intelligence

#74

Post by sanmon3186 »

wrs wrote: post #20

The big news for me here is the 16k output token limit. The models keep increasing the input limit to outrageous amounts, but output has been stuck at 4k. I did a project to summarize complex PDF invoices (not “unstructured” data, but “idiosyncratically structured” data, as each vendor has a completely different format). GPT-4o did an amazing job at the extraction of line items, but I had to do a heuristic layer on t…

>I had to do a heuristic layer on top to break up the PDFs into small chunks so the output didn’t overflow

How do you stitch the outputs of all chunks without losing the overall context?

Re: GPT-4o mini: advancing cost-efficient intelligence

#75

Post by maeil »

tempaccount420 wrote: post #71
maeil wrote: post #70

So far ever since the initial release of gpt 3.5 turbo every ""upgrade"" has mostly been an actual downgrade. I have a battery of tasks that the initial 3.5 turbo (Nov 2022) was able to perform but the newer ones very consistently fail at, regardless of prompting. I've been moving tasks from 3.5-turbo to Llama3-70b for this reason. Very curious to see whether this time it'll be an actual upgrade instead of a downgrad…

The original GPT-4 was an upgrade IMO. GPT-4 Turbo and GPT-4o were downgrades. GPT-4o seems especially bad (on text-to-text).

Yup! OpenAI's best public English-language text model to date is GPT-4, which came out more than a year ago, March '23.

But this hasn't just held for GPT-4, it's also the case for GPT-3.5 turbo, where I'd say the difference is even bigger! 0301 was the strongest (March 2023). Then we got 0613 (June 2023) and 1106 (November 2023), both significantly worse than 0301.

It's always fun to see on e.g. Reddit, ChatGPT users discussing whether GPT is getting worse or not, with clear "for" and "against" camps. To any production user that has done 1:1 comparisons, it's clear as day. Par for the course for Altman to go for this approach though, it's clear he'll do anything it takes. Taking a page out of the Tesla "FSM in 20XX " playbook of blatant lying to sell a product.

Note: For vision input, things have in fact been getting better. 4-o clearly beats the initial gpt-4-vision.

Re: GPT-4o mini: advancing cost-efficient intelligence

#76

Post by oehpr »

kristianp wrote: post #65

@dang: This post isn't on the 1st or 2nd page of hacker news. Did it trip some automated controversy detection code for too many comments in the first hour? Edit: it says 181 points, 6 hours ago, and eyeballing the 1st page it should be in the top 5 right now.

It's really clear that hacker news puts its thumb on the scale of pretty much everything in a pointedly opaque way. It's really easy to see this in action if you go down to the bottom of comments section and you'll notice a bunch of examples of comments that have negative total votes and are older sitting above comments that have positive votes and are newer. Makes me wonder, is hacker news applying global weights to users? If I post on a page, is there some metric I don't get to see that just says "this person starts with an effective -2 votes"?

I have completely lost patience with it. I no longer use the hacker news front page. Try using the hacker news search instead: https://hn.algolia.com/?query=*&dateRange=last24h

This is just the top in the last 24 hours, or you can switch it to last week to catch up. Plus the search is pretty nice and very fast so if you're looking for something specific it's convenient. This sort's explicitly in order of votes and nothing else. It's a lot better.

I'd tolerate all this rank fiddling better if it was transparent as to why things were being sorted the way they are. But that's not going to happen. Make the best of it you can.

Re: GPT-4o mini: advancing cost-efficient intelligence

#77

Post by kristianp »

oehpr wrote: post #76
kristianp wrote: post #65

@dang: This post isn't on the 1st or 2nd page of hacker news. Did it trip some automated controversy detection code for too many comments in the first hour? Edit: it says 181 points, 6 hours ago, and eyeballing the 1st page it should be in the top 5 right now.

It's really clear that hacker news puts its thumb on the scale of pretty much everything in a pointedly opaque way. It's really easy to see this in action if you go down to the bottom of comments section and you'll notice a bunch of examples of comments that have negative total votes and are older sitting above comments that have positive votes and are newer. Makes me wonder, is hacker news applying global weights to…

Normally things work quite well, with manual interventions by moderators explained in thread. However something seems to have gone wrong this time. Usually a new model from openai attracts more than 73 comments! I'm missing the depth of discussion and analysis that usually occurs here.

Re: GPT-4o mini: advancing cost-efficient intelligence

#79

Post by dllthomas »

ben_w wrote: post #57
Sohcahtoa82 wrote: post #28

Earlier quoted context omitted.

It's a joke. Sadly, the origin is unknown, but it's a joke that's well over 10 years old.

I believe it originates in the original dot.com bubble.

I'm pretty sure I heard it in an econ class, which would have been around y2k. From the way it was presented I had the sense that it was already well known.
Post reply on HN