Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.5

openai.com

Post reply on HN
Print view View original

81–90 of 1001 posts

Re: GPT-5.5

#81

Post by conradkay »

k2xl wrote: post #44

Surprised to see SWE-Bench Pro only a slight improvement (57.7% -> 58.6%) while Opus 4.7 hit 64.3%. I wonder what Anthropic is doing to achieve higher scores on this - and also what makes this test particular hard to do well in compared to Terminal Bench (which 5.5 seemed to have a big jump in)

Was 4.7 distilled off Mythos (which got 77.8%)? Interesting how mythos got 82% on terminal-bench 2.0 compared to 82.7% for GPT-5.5.

Also notice how they state just for SWE-Bench Pro: "*Anthropic reported signs of memorization on a subset of problems"

Re: GPT-5.5

#83

Post by 2001zhaozhao »

cbg0 wrote: post #73
jstummbillig wrote: post #60

Earlier quoted context omitted.

You are paying per token, but what you care about is token efficiency. If token efficiency has improved by as much as they claim it did (i.e. you need less tokens to complete a task successfully) all seems well.

If it uses half the tokens to complete a task, then doubling the cost is perfectly fine. But is that actually true?

This happens with every new model release though. The model makes less mistakes and spends less time fixing them, resulting in a token usage reduction for the same difficulty of task. Almost any task other than straight boilerplate will benefit from this.

In the same vein, I would guess that Opus 4.7 is probably cheaper for most tasks than 4.6, even though the tokenizer uses more tokens for the same length of string.

Re: GPT-5.5

#84

Post by 100ms »

losvedir wrote: post #77

> It excels at ... researching online How does this work exactly? Is there like a "search online" tool that the harness is expected to provide? Or does the OpenAI infra do that as part of serving the response? I've been working on building my own agent, just for fun, and I conceptually get using a command line, listing files, reading them, etc, but am sort of stumped how I'm supposed to do the web search piece of it.…

It's literally a distinct model with a different optimisation goal compared to normal chat. There's a ton of public information around how they work and how they're trained

Re: GPT-5.5

#85

Post by Uehreka »

thimabi wrote: post #52

Will we also see a GPT-5.5-Codex version of this model? Or will the same version of it be served both in the web app and in Codex?

After 5.1, we haven’t seen a -codex-max model, presumably because the benefits of the special training gpt-5.1-codex-max got to improve long context work filtered into gpt-5.2-codex, making the variant no longer necessary (my personal experience accords with this). I’ve been using gpt-5.4 in Codex since it came out, it’s been great. I’ve never back-to-back tested a version against its -codex variant to figure out what the qualitative difference is (this would take a long time to get a really solid answer), but I wouldn’t be surprised if at some point the general-purpose model no longer needs whatever extra training the -codex model gets and they just stop releasing them.

I thought it was weird that for almost the entire 5.3 generation we only had a -codex model, I presume in that case they were seeing the massive AI coding wave this winter and were laser focused on just that for a couple months. Maybe someday someone will actually explain all of this.

Re: GPT-5.5

#86

Post by an0malous »

sosodev wrote: post #66

I hope the industry starts competing more on highest scores with lowest tokens like this. It's a win for everybody. It means the model is more intelligent, is more efficient to inference, and costs less for the end user. So much bench-maxxing is just giving the model a ton of tokens so it can inefficiently explore the solution space.

The premise of the trillion dollars in AI investments is not that it’ll be as good as it currently is but cheaper. It’s AGI or bust at this point.

Re: GPT-5.5

#87

Post by redox99 »

k2xl wrote: post #46

ARC-AGI 3 is missing on this list - given that the SOTA before 5.5 <1% if I recall, I wonder if this didn't make meaningful progress.

It's a silly benchmark anyways.

Re: GPT-5.5

#88

Post by pietz »

pixel_popping wrote: post #26
tedsanders wrote: post #19

Just as a heads up, even though GPT-5.5 is releasing today, the rollout in ChatGPT and Codex will be gradual over many hours so that we can make sure service remains stable for everyone (same as our previous launches). You may not see it right away, and if you don't, try again later in the day. We usually start with Pro/Enterprise accounts and then work our way down to Plus. We know it's slightly annoying to have to…

can't wait! Thanks guys. PS: when you drop a new model, it would be smart to reset weekly or at least session limits :)

OpenAI has been very generous with limit resets. Please don't turn this into a weird expectation to happen whenever something unrelated happens. It would piss me off if I were in their place and I really don't want them to stop.

Re: GPT-5.5

#90

Post by aiaiai177 »

cynicalpeace wrote: post #17

It's possible that "smarter" AI won't lead to more productivity in the economy. Why? Because software and "information technology" generally didn't increase productivity over the past 30 years. This has been long known as Solow's productivity paradox. There's lots of theories as to why this is observed, one of them being "mismeasurement" of productivity data. But my favorite theory is that information technology is m…

Downvoted by the AI Nazis. They are running a tight ship before the IPOs.
Post reply on HN