Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.6

openai.com

Post reply on HN
Print view View original

901–910 of 1001 posts

Re: GPT-5.6

#901

Post by infecto »

camillomiller wrote: post #899
wkjagt wrote: post #893

I just watched the video on their launch page and I am really not sure how I feel about it. On one side, it's cool that these people get to start businesses and stuff using ChatGPT (assuming these are true stories), but how much of the business is really them? And how much does this business rely on a chat bot always being present as a kind of know it all employee? Maybe I'm just being naive or old fashioned (haven't…

I would add to this that, to me and to many friends in their 30-40s, using AI models to achieve something we used our brains to achieve feels... empty? wrong? soulless? Sure, a lot of menial work can be relegated to the models, and it's ok, most of the time, but you finish a day of work with the inability to shake a precise new feeling: that you haven't really achieved something, even if you shipped more than on an a…

Inversely I am building things I either never would have gotten to and a rate that would have taken infinitely longer. I get excited every day because instead of so much of my day spent writing the boiler plate I can spend most of the time around the architecture.

Re: GPT-5.6

#902

Post by melenaboija »

eig wrote: post #38

Funny to see that they did not include Fable 5 in their GeneBench and LifeSciBench comparisons because "it does not answer advanced biology questions and refuses the majority of questions in this eval". Winner by default!

The other day I asked Fable about fasting for 16 hours, and it flagged my question.

Pathetic situation, this one, where we are supposedly building a superintelligence while at the same time thinking that fasting is a biological weapon.

Re: GPT-5.6

#903

Post by ed_elliott_asc »

camillomiller wrote: post #899
wkjagt wrote: post #893

I just watched the video on their launch page and I am really not sure how I feel about it. On one side, it's cool that these people get to start businesses and stuff using ChatGPT (assuming these are true stories), but how much of the business is really them? And how much does this business rely on a chat bot always being present as a kind of know it all employee? Maybe I'm just being naive or old fashioned (haven't…

I would add to this that, to me and to many friends in their 30-40s, using AI models to achieve something we used our brains to achieve feels... empty? wrong? soulless? Sure, a lot of menial work can be relegated to the models, and it's ok, most of the time, but you finish a day of work with the inability to shake a precise new feeling: that you haven't really achieved something, even if you shipped more than on an a…

I don’t get this feeling, I feel like I’ve achieved so much more than I have ever achieved, everything is polished and I’m happy with it.

I’m still developing, I’m just doing more than I ever did by directing Codex.

The way I see it is the same as I saw the leap from writing code in a text editor, to using an ide with intellisense, to using the jetbrains ide’s, to using mcp’s, to now directing AI - at all of those steps I wrote code, each step less and less but still it has the same output which is it is my work - even writing in a text editor I wrote less Java (until enterprise architects got involved :) ) than C++, and than assembly.

Re: GPT-5.6

#905

Post by pimeys »

I've been testing Sol/Terra/Luna now since yesterday, running complex evals on all of them and I feel a bit... mixed on how they perform.

The eval is an agent that runs a set of tools and a prompt we can tune separately for different models. The OpenAI version of the prompt was specifically tuned based on their guide[0]. Then we let Opus to run another agent that acts as a user, trying to solve a problem (anonymized and taken from production). The problem is complex and we don't expect it to be solved by these agents, but we measure how the agents operate when faced with a vague problem:

- Opus 4.8 and GLM 5.2 both identified a constraint sooner and stopped so the user can fix an issue first that the agent cannot solve.

- Sol tried hard to solve the issue with different tools, burning tokens, until finally reached to the same conclusion with Opus and GLM. It was two times more expensive compared to Opus and six times more expensive to GLM for this task.

- Terra went even further and started calling tools that would not solve the issue, burning tokens and failing.

- Luna repeated the same failing tool call until it hit the round limit, and burned more money than Opus.

I'm kind of puzzled with the new GPT. Like, yes Sol is OK for programming, but I was expecting to get a cheap agentic model for non-programming tasks, one that can detect if things go awry and correct. Terra is too expensive and Luna not really fit for the task. Sonnet 5 is a bit better but more expensive than Opus 4.8, which is still the best in my evals. GLM 5.2 is extremely good if you can define the task and the tools clearly for it, and costs pennies!

[0] https://developers.openai.com/api/docs/guides/latest-model

Re: GPT-5.6

#906

Post by 10xDev »

bevekspldnw wrote: post #331
10xDev wrote: post #234

Earlier quoted context omitted.

Seeing the dramatic differences in scores just going from high to xhigh is just another demonstration of the bitter lesson: Just keep scaling search and learning. We are probably going to need a lot more GPUs.

These aren’t raw base models they are the result of a ton of RLHF and various adjustments. Bitter lesson wildly overstated in this context.

Not sure where I implied they are "raw base models" and not sure what "various adjustments" means here or how "ton of RLHF" contradicts anything. If we look at research for open source models, "adjustments" usually come in the form of efficiency gains which directly contributes to the ability to scale or synthetic data pipelines to increase the dataset and increasing the context window.

Re: GPT-5.6

#907

Post by visarga »

froh wrote: post #443
bevekspldnw wrote: post #331

Earlier quoted context omitted.

These aren’t raw base models they are the result of a ton of RLHF and various adjustments. Bitter lesson wildly overstated in this context.

rlhf = reinforcement learning from human feedback (had to look it up)

I think it's more RLVR (reinforcement learning from verified rewards). The RLHF is just to align models to human preferences, meaning to behave nice.

Re: GPT-5.6

#908

Post by alberth »

GPT-5.6 is the first model where I’ve actually frustrated to use it.

I’m explicitly telling it to do something extremely specific and it’s just not listening to me.

Eg, I gave it an image to update. The image is sized 400x200 pixels. It then generates a new image at 300x300. I explicitly state to be 400x200 in size and it won’t listen.

Re: GPT-5.6

#909

Post by emrehan »

I am looking to rent an apartment in a new residential tower. I have asked Fable and Sol to scrap the listings from various sources, deduplicate them and present them as a web application. Just using the cowork/(ex-)codex application interfaces.

Fable had issues with the sourcing and organizing images, and shoot itself at foot looking for shortcuts as usual. As I was getting it fix these back and forth, I copied my prompt and gave it to Sol.

Sol has surpassed my expectations by far. With a one shot simple prompt on a complex task, it gave me a working web app with everything I want with minor issues to track and fix.

Post reply on HN