Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.4

openai.com

Post reply on HN
Print view View original

51–60 of 868 posts

Re: GPT-5.4

#51

Post by ZeroCool2u »

highfrequency wrote: post #38
ZeroCool2u wrote: post #24

Bit concerning that we see in some cases significantly worse results when enabling thinking. Especially for Math, but also in the browser agent benchmark. Not sure if this is more concerning for the test time compute paradigm or the underlying model itself. Maybe I'm misunderstanding something though? I'm assuming 5.4 and 5.4 Thinking are the same underlying model and that's not just marketing.

Can you be more specific about which math results you are talking about? Looks like significant improvement on FrontierMath esp for the Pro model (most inference time compute).

Frontier Math, GPQA Diamond, and Browsecomp are the benchmarks I noticed this on.

Re: GPT-5.4

#53

Post by dpoloncsak »

denysvitali wrote: post #6

Article: https://openai.com/index/introducing-gpt-5-4/ gpt-5.4 Input: $2.50 /M tokens Cached: $0.25 /M tokens Output: $15 /M tokens --- gpt-5.4-pro Input: $30 /M tokens Output: $180 /M tokens Wtf

[flagged]

Re: GPT-5.4

#54

Post by embedding-shape »

simianwords wrote: post #35
minimaxir wrote: post #8

The marquee feature is obviously the 1M context window, compared to the ~200k other models support with maybe an extra cost for generations beyond >200k tokens. Per the pricing page, there is no additional cost for tokens beyond 200k: https://openai.com/api/pricing/ Also per pricing, GPT-5.4 ($2.50/M input, $15/M output) is much cheaper than Opus 4.6 ($5/M input, $25/M output) and Opus has a penalty for its beta >200…

Why would some one use codex instead?

Why would someone use Claude Code instead? Or any other harness? Or why only use one?

My own tooling throws off requests to multiple agents at the same time, then I compare which one is best, and continue from there. Most of the time Codex ends up with the best end results though, but my hunch is that at one point that'll change, hence I continue using multiple at the same time.

Re: GPT-5.4

#55

Post by csnweb »

ZeroCool2u wrote: post #51
highfrequency wrote: post #38

Earlier quoted context omitted.

Can you be more specific about which math results you are talking about? Looks like significant improvement on FrontierMath esp for the Pro model (most inference time compute).

Frontier Math, GPQA Diamond, and Browsecomp are the benchmarks I noticed this on.

Are you may be comparing the pro model to the non pro model with thinking? Granted it’s a bit confusing but the pro model is 10 times more expensive and probably much larger as well.

Re: GPT-5.4

#57

Post by embedding-shape »

steve1977 wrote: post #42
mattas wrote: post #5

"GPT‑5.4 interprets screenshots of a browser interface and interacts with UI elements through coordinate-based clicking to send emails and schedule a calendar event." They show an example of 5.4 clicking around in Gmail to send an email. I still think this is the wrong interface to be interacting with the internet. Why not use Gmail APIs? No need to do any screenshot interpretation or coordinate-based clicking.

One could argue that LLMs learning programming languages made for humans (i.e. most of them) is using the wrong interface as well. Why not use machine code?

Why would human language by the wrong interface when they're literally language models? Why would machine code be better when there is probably magnitude less of training material with machine code?

You can also test this yourself easily, fire up two agents, ask one to use PL meant for humans, and one to write straight up machine code (or assembly even), and see which results you like best.

Re: GPT-5.4

#59

Post by tedsanders »

minimaxir wrote: post #8

The marquee feature is obviously the 1M context window, compared to the ~200k other models support with maybe an extra cost for generations beyond >200k tokens. Per the pricing page, there is no additional cost for tokens beyond 200k: https://openai.com/api/pricing/ Also per pricing, GPT-5.4 ($2.50/M input, $15/M output) is much cheaper than Opus 4.6 ($5/M input, $25/M output) and Opus has a penalty for its beta >200…

Yeah, long context vs compaction is always an interesting tradeoff. More information isn't always better for LLMs, as each token adds distraction, cost, and latency. There's no single optimum for all use cases.

For Codex, we're making 1M context experimentally available, but we're not making it the default experience for everyone, as from our testing we think that shorter context plus compaction works best for most people. If anyone here wants to try out 1M, you can do so by overriding `model_context_window` and `model_auto_compact_token_limit`.

Curious to hear if people have use cases where they find 1M works much better!

(I work at OpenAI.)

Re: GPT-5.4

#60

Post by BoredPositron »

steve1977 wrote: post #42
mattas wrote: post #5

"GPT‑5.4 interprets screenshots of a browser interface and interacts with UI elements through coordinate-based clicking to send emails and schedule a calendar event." They show an example of 5.4 clicking around in Gmail to send an email. I still think this is the wrong interface to be interacting with the internet. Why not use Gmail APIs? No need to do any screenshot interpretation or coordinate-based clicking.

One could argue that LLMs learning programming languages made for humans (i.e. most of them) is using the wrong interface as well. Why not use machine code?

because they are inherently text based as is code?
Post reply on HN