Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

421–430 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#421

Post by andrew_eu »

slig wrote: post #347
andrew_eu wrote: post #280

I thought with this chain-of-thought approach the model might be better suited to solve a logic puzzle, e.g. ZebraPuzzles [0]. It produced a ton of "reasoning" tokens but hallucinated more than half of the solution with names/fields that weren't available. Not a systematic evaluation, but it seems like a degradation from 4o-mini. Perhaps it does better with code reasoning problems though -- these logic puzzles are es…

Hey, I run ZebraPuzzles.com, thanks for mentioning it! Right now I'm trying to improve the puzzles so that people can't "cheat" using LLMs so easily ;-).

It's fantastic! Thanks for the great work.

Re: Notes on OpenAI's new o1 chain-of-thought models

#422

Post by neonsunset »

mattmanser wrote: post #150
COAGULOPATH wrote: post #54

Earlier quoted context omitted.

Yes, this only helps multi-step reasoning. The model still has problems with general knowledge and deep facts. There's no way you can "reason" a correct answer to "list the tracklisting of some obscure 1991 demo by a band not on Wikipedia." You either know or you don't. I usually test new models with questions like "what are the levels in [semi-famous PC game from the 90s]?" The release version of GPT-4 could get abo…

It's actually much worse than that and you're inadvertently down playing how bad it is. It doesn't even know mildly obsecure facts that are on the internet. For example last night I was trying to do something with C# generics and it confidently told me I could use pattern matching on the type in a switch statwmnt, and threw out some convincing looking code. You can't, it's impossible. It wàa completely wrong. When I…

> For example last night I was trying to do something with C# generics and it confidently told me I could use pattern matching on the type in a switch statwmnt, and threw out some convincing looking code.

Just use it on an instance instead

  var res = thing switch {
    OtherThing ot => …,
    int num => …,
    string s => …,
    _ => …
  };

Re: Notes on OpenAI's new o1 chain-of-thought models

#423

Post by andrew_eu »

energy123 wrote: post #338
andrew_eu wrote: post #280

I thought with this chain-of-thought approach the model might be better suited to solve a logic puzzle, e.g. ZebraPuzzles [0]. It produced a ton of "reasoning" tokens but hallucinated more than half of the solution with names/fields that weren't available. Not a systematic evaluation, but it seems like a degradation from 4o-mini. Perhaps it does better with code reasoning problems though -- these logic puzzles are es…

o1-mini does better than any other model on zebra puzzles. Maybe you got unlucky on one question? https://www.reddit.com/r/LocalLLaMA/comments/1ffjb4q/prelimi...

Entirely possible. I did not try to test systematically or quantitatively, but it's been a recurring easy "demo" case I've used with releases since 3.5-turbo.

The super verbose chain-of-reasoning that o1 does seems very well suited to logic puzzles as well, so I expected it to do reasonably well. As with many other LLM topics, though, the framing of the evaluation (or the templating of the prompt) can impact the results enormously.

Re: Notes on OpenAI's new o1 chain-of-thought models

#424

Post by KoolKat23 »

lukev wrote: post #85
fnordpiglet wrote: post #73

Earlier quoted context omitted.

One aspect that’s not achievable is they discuss hiding the chain of thought in its raw form because the chains are allowed to be unaligned. This allows the model to operate without any artifacts from alignment and apply them in the post processing, more or less. This requires effectively root and you would need the unaligned weights.

Ok but this presses on a latent question: what do we mean by alignment? Practically it's come to mean just sanitization... "don't say something nasty or embarrassing to users." But that doesn't apply here, the reasoning tokens are effectively just a debug log. If alignment means "conducting reasoning in alignment with human values", then misalignment in the reasoning phase could potentially be obfuscated and sanitize…

Glass half full and the good faith argument.

It's a compromise.

OpenAI will now have access to vaste amounts of unaligned output so they can actually study it's thinking.

Whereas the current checks and balances meant the request was rejected and the data providing this insight was not created in the first place.

Re: Notes on OpenAI's new o1 chain-of-thought models

#425

Post by ksplicer »

bambax wrote: post #281

Near the end, the quote from OpenAI researcher Jason Wei seems damning to me: > Results on AIME and GPQA are really strong, but that doesn’t necessarily translate to something that a user can feel. Even as someone working in science, it’s not easy to find the slice of prompts where GPT-4o fails, o1 does well, and I can grade the answer. But when you do find such prompts, o1 feels totally magical. We all need to find…

This is something we've been grappeling with on my team. Many of the researchers in the org want to try all these reasoning techniques to increase performance, and my team keeps pushing back that we don't actually need that extra performance- we just want to decrease latency and cost.

Re: Notes on OpenAI's new o1 chain-of-thought models

#426

Post by swader999 »

idiotsecant wrote: post #391
berniedurfee wrote: post #381

Earlier quoted context omitted.

I’m starting to think this is an unsolvable problem with LLMs. The very act of “reasoning” requires one to know that they don’t know something. LLMs are giant word Plinko machines. A million monkeys on a million typewriters. LLMs are not interns. LLMs are assumption machines. None of the million monkeys or the collective million monkeys are “reasoning” or are capable of knowing. LLMs are a neat parlor trick and are s…

I think LLMs are definitely on the path to AGI in the same way that the ball bearing was on the path to the internal combustion engine. I think its quite likely that LLMs will perform important functions within the system of an eventual AGI.

This may be accurate. I wonder if there's enough energy in the world for this endeavour.

Re: Notes on OpenAI's new o1 chain-of-thought models

#427

Post by kurtoid »

energy123 wrote: post #173
ironhaven wrote: post #133

So is o1 nicknamed “strawberry” because it was designed to solve the “how many many times does the letter R appear in strawberry” problem.

No, that was a coincidence according to an employee there

Coincidence or not, they seem to be poking fun at it: https://openai.com/index/learning-to-reason-with-llms/#chain...

(end of the cipher example)

Re: Notes on OpenAI's new o1 chain-of-thought models

#428

Post by wavemode »

bentonkek wrote: post #405
lewhoo wrote: post #307

Earlier quoted context omitted.

> which parts of intelligence do you think are not representable as conditional probability distributions Maybe I'm wrong here but a lot of our brilliance comes from acting against the statistical consensus. What I mean is, Nicolaus Copernicus probably consumed a lot of knowledge on how the Earth is the center of the universe etc. and probably nothing contradicting that notion. Can a LLM do that ?

Copernicus was an exception, not the rule. Would you say everyone else who lived at the time was not 'really' intelligent?

That's an illogical counterargument. The absence of published research output does not imply the absence of intelligent brain patterns. What if someone was intelligent but just wasn't interested in astronomy?

Re: Notes on OpenAI's new o1 chain-of-thought models

#429

Post by bnralt »

KoolKat23 wrote: post #363
pedrosorio wrote: post #110

Earlier quoted context omitted.

> It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output This is not an apt description of the system that insists the doctor is the mother of the boy involved in a car accident when elementary understanding of English and very little logic show that answer to be obviously wrong. https://x.com/colin_fraser/status/183…

I'm noticing a strange common theme in all these riddles, it's being asked and getting wrong. They're all badly worded questions. The model knows something is up and reads into it too much. In this case it's tautology, you would usually say "a mother and her son...". I think it may answer correctly if you start off asking "Please solve the below riddle:" There was another example yesterday which it solved correctly a…

> They're all badly worded questions. The model knows something is up and reads into it too much. The model knows something is up and reads into it too much. In this case it's tautology, you would usually say "a mother and her son...".

How is "a woman and her son" badly worded? The meaning is clear and blatently obvious to any English speaker.

Re: Notes on OpenAI's new o1 chain-of-thought models

#430

Post by lossolo »

lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

O1 seems like a variant of RLRF https://arxiv.org/abs/2403.14238
Post reply on HN