Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

351–360 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#351

Post by chaosist »

benterix wrote: post #290
bambax wrote: post #281

Near the end, the quote from OpenAI researcher Jason Wei seems damning to me: > Results on AIME and GPQA are really strong, but that doesn’t necessarily translate to something that a user can feel. Even as someone working in science, it’s not easy to find the slice of prompts where GPT-4o fails, o1 does well, and I can grade the answer. But when you do find such prompts, o1 feels totally magical. We all need to find…

> But why? Why would we do that? Because OpenAI needs a steady influx of money, big money. In order to do so, they have to convince the people who are giving them money that they are the best. An objective way to achieve this is by benchmarking. But once you enter this game, you start optimizing for benchmarks. At the same time, in the real world, Anthropic is following them in huge leaps and for many users Claude 3.…

Agree completely.

From a user perspective too, I was a subscriber from the first day of gpt4 until about a month ago. I thought about subscribing for the month to check this out but I am tired of the OpenAI experience.

Where is Sora? Where is the version of chatgpt that responds in real time to your voice? Remember the gpt4 demo that you would draw a website on a napkin?

How about Q* lol. Strawberry/Q*/o1, "it is super dangerous, be very careful!"

Quietly, Anthropic has just kicked their ass without all the hype and I am about to go work in sonnet instead even bothering to check o1 out.

Re: Notes on OpenAI's new o1 chain-of-thought models

#352

Post by ImHereToVote »

achow wrote: post #167
pedrosorio wrote: post #110

Earlier quoted context omitted.

> It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output This is not an apt description of the system that insists the doctor is the mother of the boy involved in a car accident when elementary understanding of English and very little logic show that answer to be obviously wrong. https://x.com/colin_fraser/status/183…

What I'm not able to comprehend is why people are not seeing the answer as brilliant! Any ordinary mortal (like me) would have jumped to the conclusion that answer is "Father" and would have walked away patting on my back, without realising that I was biased by statistics. Whereas o1, at the very outset smelled out that it is a riddle - why would anyone out of blue ask such question. So, it started its chain of thoug…

The doctor is obviously a parent of the boy. The language tricks simply emulate the ambiance of reasoning. Similarly to a political system emulating the ambiance of democracy.

Re: Notes on OpenAI's new o1 chain-of-thought models

#353

Post by famouswaffles »

fragmede wrote: post #337
ssl-3 wrote: post #283

Earlier quoted context omitted.

The farmer riddle isn't quite right as you presented it. One of the parts that makes it interesting is that the boat can't carry everything at one time[1]. It can't happen in one trip; something must be left behind. It solved the correct version fine: https://chatgpt.com/share/66e3f9bb-632c-8005-9c95-142424e396... 1: https://en.wikipedia.org/wiki/Wolf,_goat_and_cabbage_problem

You misunderstand the situation. If I give ChatGPT-4 the original farmer riddle, it "solves" it just fine, but it's assumed that it isn't actually solving it. That is, it's not thinking or doing any logical reasoning, or anything resembling that to come to a solution to the problem, but that it's simply regurgitating the problem's solution since it appears in the training data. Giving ChatGPT-4 the modified farmers r…

GPT-4 will often get the modified question of you change its "shape" enough. It's clearly overfit to that question so making the modified question not look like the one from training. Sometimes changing the names is enough.

Re: Notes on OpenAI's new o1 chain-of-thought models

#354

Post by flanked-evergl »

vingt_regards wrote: post #336
flanked-evergl wrote: post #321

Earlier quoted context omitted.

> The failure is in how you're using it. People, for the most part, know what they know and don't know. I am not uncertain that the distance between the earth and the sun varies, but I'm certain that I don't know the distance from the earth to the sun, at least not with better precision than about a light week. This is going to have to be fixed somehow to progress past where we are now with LLMs. Maybe expecting an L…

> the distance from the earth to the sun, at least not with better precision than about a light week The sun is eight light minutes away.

Thanks, I was not sure if it was light hours or minutes away, but I knew for sure it's not light weeks (emphasis on plural here) away. I will probably forget again in a couple of years.

Re: Notes on OpenAI's new o1 chain-of-thought models

#355

Post by steveBK123 »

raverbashing wrote: post #228
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

> A good intern will ask clarifying questions, tell me “I don’t know” Your expectations are bigger than mine (Though some will get stuck in "clarifying questions" and helplessness and not proceed neither)

Indeed. My expectation of a good intern is to produce nothing I will put in production, but show aptitude worth hiring them for. It's a 10 week extended interview with lots of social events, team building, tech talks, presentations, etc.

Which is why I've liked the LLM analogy of "unlimited free interns".. I just think some people read that the exact opposite way I do (not very useful).

Re: Notes on OpenAI's new o1 chain-of-thought models

#356

Post by mergisi »

I've been working on a o1-preview and recently hit some limitations with OpenAI's cap. But I’ve made progress—added all the steps, details, and code on GitHub https://github.com/mergisi/openai-o1-coded-personal-blog . The result isn't bad at all; just a few more CSS tweaks to improve it. Check it out and let me know what you think! How does it compare to tools like Claude Sonnet 3.5?

Re: Notes on OpenAI's new o1 chain-of-thought models

#357

Post by KoolKat23 »

MPSimmons wrote: post #106
layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

This is a great description.

Re: Notes on OpenAI's new o1 chain-of-thought models

#358

Post by Culonavirus »

zaptrem wrote: post #190
lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

This is the bitter lesson/just put it in the model. They're trying to figure out more ways of converting compute to intelligence now that they're running out of text data: https://images.ctfassets.net/kftzwdyauwt9/7rMY55vLbGTlTiP9Gd...

A cynical way to look at it is that we're pretty close to the ultimate limits of what LLMs can do and now the stake holders are looking at novel ways of using what they have instead of pouring everything into novel models. We're several years into the AI revolution (some call it a bubble) and Nvidia is still pretty much the only company that makes bank on it. Other than that it's all investment driven "growth". And at some point investors are gonna start asking questions...

Re: Notes on OpenAI's new o1 chain-of-thought models

#359

Post by rdtsc »

accountnum wrote: post #325
rdtsc wrote: post #253

Earlier quoted context omitted.

Fluency is a necessary but not the only prerequisite. To be able to answer a trick question, it’s first necessary to understand the question.

No, it's necessary to either know that it's a trick question or to have a feeling that it is based on context. The entire point of a question like that is to trick your understanding. You're tricking the model because it has seen this specific trick question a million times and shortcuts to its memorized solution. Ask it literally any other question, it can be as subtle as you want it to be, and the model will pick u…

> I can trick people who have never heard of the trick with the 7 wives and 7 bags and so on. That doesn't mean they didn't understand

They could fail because they didn’t understand the language. Didn’t have a good memory to memorize all the steps, or couldn’t reason through it. We could pose more questions to probe which reason is more plausible.

Re: Notes on OpenAI's new o1 chain-of-thought models

#360

Post by K0balt »

croes wrote: post #310
fragmede wrote: post #276

Earlier quoted context omitted.

https://chatgpt.com/share/66e3f9e1-2cb4-8009-83ce-090068b163... Keep up, that was last week's gotcha, with the old model.

My point is the previous "intelligent" failed at simple task, the new one will also fail on simple tasks. That's ok for humans but not for machines.

‘That's ok for humans but not for machines.’

This is a really interesting bias. I mean, I understand, I feel that way too… but if you think about it, it might be telling us something about intelligence itself.

We want to make machines that act more like humans: we did that, and we are now upset that they are just as flaky and unreliable as drunk uncle bob. I have encountered plenty of people that aren’t as good at being accurate or even as interesting to talk to as a 70b model. Sure, LLMs make mistakes most humans would not, but humans also make mistakes most LLMs would not.

(I am not trying to equate humans and LLMs, just to be clear) (also, why isn’t equivelate a word?)

It turns out we want machines that are extremely reliable, cooperative , responsible and knowledgeable. We yearn to be obsolete.

We want machines that are better than us.

The definition of AGI has drifted from meaning. “able to broadly solve problems the (class of which) system designers did not anticipate” to “must be usefully intelligent at the same level as a bright, well educated person”.

Where along the line did we suddenly forget that dog level intelligence was a far out of reach goal until suddenly it wasn’t?

Post reply on HN