Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

261–270 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#261

Post by ryanjshaw »

geysersam wrote: post #197
achow wrote: post #167

Earlier quoted context omitted.

What I'm not able to comprehend is why people are not seeing the answer as brilliant! Any ordinary mortal (like me) would have jumped to the conclusion that answer is "Father" and would have walked away patting on my back, without realising that I was biased by statistics. Whereas o1, at the very outset smelled out that it is a riddle - why would anyone out of blue ask such question. So, it started its chain of thoug…

Come on. Of course chatgpt has read that riddle and the answer 1000 times already.

Why does it exist 1000 times in the training if there isn't some trick to it, i.e. some subset of humans had to have answered it incorrectly for the meme to replicate that extensively in our collective knowledge.

And remember the LLM has already read a billion other things, and now needs to figure out - is this one of them tricky situations, or the straightforward ones? It also has to realize all the humans on forums and facebook answering the problem incorrectly are bad data.

Might seem simple to you, but it's not.

Re: Notes on OpenAI's new o1 chain-of-thought models

#262

Post by jcheng »

ssl-3 wrote: post #234
valval wrote: post #186

Earlier quoted context omitted.

Really it just does what you tell it to. Have you tried telling it “ask me clarifying questions about all the APIs you need to solve this problem”? Huge contrast to human interns who aren’t experienced or smart enough to ask the right questions in the first place, and/or have sentimental reasons for not doing so.

Sure, but to what end? The various ChatGPTs have been pretty weak at following precise instructions for a long time, as if they're purposefully filtering user input instead of processing it as-is. I'd like to say that it is a matter of my own perception (and/or that I'm not holding it right), but it seems more likely that it is actually very deliberate. As a tangential example of this concept, ChatGPT 4 rather unexpe…

I had to use that technique ("don't acknowledge this sideband data that may or may not be relevant to the task at hand") myself last month. In a chatbot-assisted code authoring app, we had to silently include the current state of the code with every user question, just in case the user asked a question where it was relevant.

Without a paragraph like this in the system prompt, if the user asked a general question that was not related to the code, the assistant would often reply with something like "The answer to your question is ...whatever... . I also see that you've sent me some code. Let me know if you have specific questions about it!"

(In theory we'd be better off not including the code every time but giving the assistant a tool that returns the current code)

Re: Notes on OpenAI's new o1 chain-of-thought models

#263

Post by [deleted] »

gloosx wrote: post #244
c-fe wrote: post #208

Earlier quoted context omitted.

With multiplication the question makes sense due to the commutative property but division does not have that so the question becomes ambiguous... And now I see that the model even points this out.

There is no ambiguity, the problem is that three numbers, divided together, without the order specified, must be equal to their sum. You can find solutions for a / b / c, or b / c / a, or c / a / b, any combination of them and the solution will be correct according to the problem description. Besides, what's does it even has to do with it concluding with confidence: "The fundamental issue is that division tends to ma…

[deleted]

Re: Notes on OpenAI's new o1 chain-of-thought models

#264

Post by yreg »

eigenket wrote: post #238
roomey wrote: post #223

Earlier quoted context omitted.

Why couldn't the doctor be the boys mother? There is no indication of the sex of the doctor, and families that consist of two mothers do actually exist and probably doesn't even count as that unusual.

"When the doctor sees the boy he says" Indicates the gender of the father.

A mother can have a male gender.

I wonder if this interpretation is a result of attempts to make the model more inclusive than the corpus text, resulting in a guess that's unlikely, but not strictly impossible.

Re: Notes on OpenAI's new o1 chain-of-thought models

#265

Post by oersted »

freediver wrote: post #141

Not seeing major advance in quality with o1, but seeing major negative impact on cost and latency. Kagi LLM benchmarking project: https://help.kagi.com/kagi/ai/llm-benchmark.html

Kagi is most likely evaluating it mainly on deriving an answer for the user from search result snippets. Indeed, GPT-4o is plenty good at this already, and o1 would only perform better on particular types of hard requests, while being so much slower.

If you look at Appendix A in the o1 post [1], this becomes quite clear. There's a huge jump in performance in "puzzle" tasks like competitive maths or programming. But the difference on everything else is much less significant, and this evaluation is still focused on reasoning tasks.

The human preference chart [1] also clearly shows that it doesn't feel that much better to use, hence the overall reaction.

Everyone is complaining about exaggerated marketing, and it's true, but if you take the time to read what they wrote beyond the shallow ads, they are being somewhat honest about what this is.

[1] https://openai.com/index/learning-to-reason-with-llms/

Re: Notes on OpenAI's new o1 chain-of-thought models

#266

Post by ssl-3 »

jcheng wrote: post #262
ssl-3 wrote: post #234

Earlier quoted context omitted.

Sure, but to what end? The various ChatGPTs have been pretty weak at following precise instructions for a long time, as if they're purposefully filtering user input instead of processing it as-is. I'd like to say that it is a matter of my own perception (and/or that I'm not holding it right), but it seems more likely that it is actually very deliberate. As a tangential example of this concept, ChatGPT 4 rather unexpe…

I had to use that technique ("don't acknowledge this sideband data that may or may not be relevant to the task at hand") myself last month. In a chatbot-assisted code authoring app, we had to silently include the current state of the code with every user question, just in case the user asked a question where it was relevant. Without a paragraph like this in the system prompt, if the user asked a general question that…

I understand what you're saying, but the lack of acknowledgement isn't the problem I'm complaining about.

The problem is the instructed lack of relevance for 99% of requests.

If your sideband data included an instruction that said "This sideband data is shown to you in every request -- this means that it is not relevant to 99% of requests," then: I'd like to suggest that the for vast majority of the time, your sideband data doesn't exist at all.

Re: Notes on OpenAI's new o1 chain-of-thought models

#267

Post by kaoD »

c-fe wrote: post #208
gloosx wrote: post #191

Earlier quoted context omitted.

That's a classical school question, word-by-word, except multiplication is replaced by division

With multiplication the question makes sense due to the commutative property but division does not have that so the question becomes ambiguous... And now I see that the model even points this out.

Does the commutative property change anything here? A, B and C are not constrained in any way to each other, so they can be in whatever order you want anyways...

Moreover, addition is commutative so it doesn't matter what order the division is in since a/b/c = a+b+c = c+a+b = ...

So I'd say that the model pointing this out is actually a mistake and it managed to trick you. Classic LLM stuff: spit out wrong stuff in a convincing manner.

Re: Notes on OpenAI's new o1 chain-of-thought models

#268

Post by eigenket »

yreg wrote: post #264
eigenket wrote: post #238

Earlier quoted context omitted.

"When the doctor sees the boy he says" Indicates the gender of the father.

A mother can have a male gender. I wonder if this interpretation is a result of attempts to make the model more inclusive than the corpus text, resulting in a guess that's unlikely, but not strictly impossible.

I think its more likely this is just an easy way to trick this model. It's seen lots of riddles, so when it's sees something that looks like a riddle but isn't one it gets confused.

Re: Notes on OpenAI's new o1 chain-of-thought models

#269

Post by oersted »

thenameless7741 wrote: post #24

> No system prompt support—the models use the existing chat completion API but you can only send user and assistant messages. > No streaming support, tool usage, batch calls or image inputs either. I think it's worth adding a note explaining that many of these limitations are due to the beta status of the API. max_tokens is the only parameter I've seen deprecated in the API docs. From https://platform.openai.com/docs…

I wonder if it supports Structured Output / JSON Mode. That would make a big difference to programmatic use. I guess I will try it later when I have time.

Re: Notes on OpenAI's new o1 chain-of-thought models

#270

Post by croes »

MPSimmons wrote: post #106
layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

Intelligent?

Just ask ChatGPT

How many Rs are in strawberry?

Post reply on HN