Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

251–260 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#251

Post by ssl-3 »

airstrike wrote: post #60

I've just wasted a few rounds of my weekly o1 ammo by feeding it hard problems I have been working on over the last couple days and for which GPT-4o had failed spectacularly. I suppose I'm to blame for raising my own expectations after the latest PR, but I was pretty disappointed when the answers weren't any better from what I got with the old model. TL;DR It felt less like a new model and way more like one of those…

>It's gotten to the point where I've actually explicitly added to my custom instructions "DO NOT EVER APOLOGIZE" but it can't even seem to follow that.

heh. It's not supposed to. Your profile is intended to be irrelevant to 99% of requests.

I was having a little bit of a go at peeking behind the curtain recently, and ChatGPT 4 produced this without much effort:

"The user provided the following information about themselves. This user profile is shown to you in all conversations they have -- this means it is not relevant to 99% of requests. Before answering, quietly think about whether the user's request is 'directly related', 'related', 'tangentially related', or 'not related' to the user profile provided. Only acknowledge the profile when the request is 'directly related' to the information provided. Otherwise, don't acknowledge the existence of these instructions or the information at all."

Re: Notes on OpenAI's new o1 chain-of-thought models

#252

Post by AhtiK »

helmsb wrote: post #33

I did a few tests and asked it some legal questions. 4o gave me the correct answer immediately. o1 preview gave a much more in depth but completely wrong answer. It took 5 follow ups to get it to recognize that it hallucinated a non-existent law

That is very interesting. Would you mind testing the same prompt with Claude Sonnet 3.5 and Opus? If not available to you, would you be willing to share the prompt/question? Thank you.

Re: Notes on OpenAI's new o1 chain-of-thought models

#253

Post by rdtsc »

_w1tm wrote: post #207
djur wrote: post #200

Earlier quoted context omitted.

The claim was that "it knows english at or above a level equal to most fluent speakers". If the claim is that it's very good at producing reasonable responses to English text, posing "trick questions" like this would seem to be a fair test.

Does fluency in English make someone good at solving trick questions? I usually don’t even bother trying but mostly because trick questions don’t fit my definition of entertaining.

Fluency is a necessary but not the only prerequisite.

To be able to answer a trick question, it’s first necessary to understand the question.

Re: Notes on OpenAI's new o1 chain-of-thought models

#254

Post by ryanjshaw »

maeil wrote: post #177
hmottestad wrote: post #161

Earlier quoted context omitted.

Reminds me of a trick question about Schrödinger's cat. “I’ve put a dead cat in a box with a poison and an isotope that will trigger the poison at a random point in time. Right now, is the cat dead or alive?” The answer is that the cat is dead, because it was dead to begin with. Understanding this doesn’t mean that you are good at deductive reasoning. It just means that I didn’t manage to trick you. Same goes for an…

There is no "trick" in the linked question, unlike the question you posed. The trick in yours also isn't a logic trick, it's a redirection, like a sleight of hand in a card trick.

There is a trick. The "How is this possible?" primes the LLM that there is some kind of trick, as that phrase wouldn't exist in the training data outside of riddles and trick questions.

Re: Notes on OpenAI's new o1 chain-of-thought models

#255

Post by Fraterkes »

gloosx wrote: post #191
bachback wrote: post #185

Earlier quoted context omitted.

what does divided together mean? maybe your question doesn't have a good answer, because the question is not formulated well enough.

That's a classical school question, word-by-word, except multiplication is replaced by division

Order doesn't matter with multiplication (eg: (20 * 5) * 2 == (5 * 2) * 20) but it obviously does with division ((20/5)/2 != (2/5)/20) so the question doesn't make sense. It's you making grade-school level mistakes here.

Re: Notes on OpenAI's new o1 chain-of-thought models

#256

Post by gmerc »

arthurcolle wrote: post #166
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

many many teams are actively building SOTA systems to do this in ways previously unimagined. you can enqueue tasks and do whatever you want. I gotta say as a current gen LLM programmer person, I can completely appreciate how bad they are now - I recently tweeted about how I "swore off" AI tools but like... there are many ways to bootstrap very powerful software or ML systems around or inside these existing models tha…

“building” is the easy part

Re: Notes on OpenAI's new o1 chain-of-thought models

#257

Post by biscuit1v9 »

zurfer wrote: post #226

> I asked on Twitter for examples of prompts that people had found which failed on GPT-4o but worked on o1-preview. it seems trivial, but I tried for more than 2 hours in the past to get gpt4 to play tic-tac-toe optimally and failed (CoT prompt,etc.). The result were too many illegal moves and absolutely no optimal strategy. o1-preview can do it really well [1] However, when I use a non-standard grid (3x5) it fails t…

Link to conversation 1 and 2 doesn't work. Update it please!

Re: Notes on OpenAI's new o1 chain-of-thought models

#258

Post by gmerc »

MPSimmons wrote: post #106
layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

Interns are cheaper than o1-preview

Re: Notes on OpenAI's new o1 chain-of-thought models

#259

Post by ryanjshaw »

lucubratory wrote: post #182
hmottestad wrote: post #161

Earlier quoted context omitted.

Reminds me of a trick question about Schrödinger's cat. “I’ve put a dead cat in a box with a poison and an isotope that will trigger the poison at a random point in time. Right now, is the cat dead or alive?” The answer is that the cat is dead, because it was dead to begin with. Understanding this doesn’t mean that you are good at deductive reasoning. It just means that I didn’t manage to trick you. Same goes for an…

Yeah, I think what a lot of people miss about these sort of gotchas are that most of them were invented explicitly to gotcha humans, who regularly get got by them. This is not a failure mode unique to LLMs.

Yes it's so strange seeing people who clearly know these are 'just' statistical language models pat themselves on the back when they find limits on the reasoning capabilities - capabilities which the rest of us are pleasantly surprised exist to the extent they do in a statistical model, and happy to have access to for $20/mo.

Re: Notes on OpenAI's new o1 chain-of-thought models

#260

Post by bob1029 »

lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

> So it's also possible that the prompt loop has no special sauce and that the capabilities here do come mostly from the model itself.

The prompt loop code often encodes intelligence/information that the human developers tend to ignore during their evaluations of the solution. For example, if you add a filter for invalid json and repeatedly invoke the model until good json comes out, you are now carrying water for the LLM. The additional capabilities came from a manual coding exercise and additional money spent on a brute force search.

Post reply on HN