Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

181–190 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#181

Post by 8n4vidtmkvmk »

ttul wrote: post #48
mrtesthah wrote: post #25

Earlier quoted context omitted.

It begs the question of whether we can supply a function to be called (e.g., one that compiles and runs code) to evaluate intermediate CoT results

It seems OpenAI has decided to keep the CoT results a secret. If they were to allow the model to call out to tools to help fill in the CoT steps, then this might reveal what the model is thinking - something they do not want the outside world to know about. I could imagine OpenAI might allow their own vetted tools to be used, but perhaps it will be a while (if ever) before developers are allowed to hook up their own…

It's only a matter of time. When some other company releases the tool, they likely will too.

Re: Notes on OpenAI's new o1 chain-of-thought models

#182

Post by lucubratory »

hmottestad wrote: post #161
pedrosorio wrote: post #110

Earlier quoted context omitted.

> It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output This is not an apt description of the system that insists the doctor is the mother of the boy involved in a car accident when elementary understanding of English and very little logic show that answer to be obviously wrong. https://x.com/colin_fraser/status/183…

Reminds me of a trick question about Schrödinger's cat. “I’ve put a dead cat in a box with a poison and an isotope that will trigger the poison at a random point in time. Right now, is the cat dead or alive?” The answer is that the cat is dead, because it was dead to begin with. Understanding this doesn’t mean that you are good at deductive reasoning. It just means that I didn’t manage to trick you. Same goes for an…

Yeah, I think what a lot of people miss about these sort of gotchas are that most of them were invented explicitly to gotcha humans, who regularly get got by them. This is not a failure mode unique to LLMs.

Re: Notes on OpenAI's new o1 chain-of-thought models

#183

Post by throwaway314155 »

simonw wrote: post #156
dkga wrote: post #143

Judging by this thread, surely a part of OpenAI’s business model is to release models with somewhat grey-area outlandish claims, then sit and wait for people to test it out paying top dollars for tokens.

They’re not charging anything for o1 right now - it’s running as a free beta. EDIT: Actually I’m not sure I’m correct about that - I’ve seen screenshots that indicate the API model is being charged for here: https://twitter.com/deepaiwriter/status/1834460643228897512 The value OpenAI get here is that people effectively run a massively parallel brute force attack against the new models to figure out exactly what they…

> The value OpenAI get here is that people effectively run a massively parallel brute force attack against the new models to figure out exactly what they can and can’t do.

I'm pretty sure the value they get is the money you pay.

Re: Notes on OpenAI's new o1 chain-of-thought models

#184

Post by maeil »

anon291 wrote: post #111
pedrosorio wrote: post #110

Earlier quoted context omitted.

> It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output This is not an apt description of the system that insists the doctor is the mother of the boy involved in a car accident when elementary understanding of English and very little logic show that answer to be obviously wrong. https://x.com/colin_fraser/status/183…

Keep in mind that the system always chooses randomly so there is always a possibility it commits to the wrong output. I don't know why openAi won't allow determinism but it doesn't, even with temperature set to zero

Nondeterminism provides an excuse for errors, determinism doesn't.

Nondeterminism scores worse with human raters, because it makes output sound even more robotic and less human.

Re: Notes on OpenAI's new o1 chain-of-thought models

#185

Post by bachback »

gloosx wrote: post #172

Just leaving here an example of the "smart chain of thought", which took the model about 20s of thinking. The problem I asked was – "what 3 numbers give the same result when divided and added together". Seems the current state of art is that basic school problems are too hard yet even for this new chain-of-marketing. Let's think through this step-by-step: We're looking for three numbers a, b, and c where: (a / b) / c…

what does divided together mean? maybe your question doesn't have a good answer, because the question is not formulated well enough.

Re: Notes on OpenAI's new o1 chain-of-thought models

#186

Post by valval »

williamdclt wrote: post #164
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

Really it just does what you tell it to. Have you tried telling it “ask me clarifying questions about all the APIs you need to solve this problem”?

Huge contrast to human interns who aren’t experienced or smart enough to ask the right questions in the first place, and/or have sentimental reasons for not doing so.

Re: Notes on OpenAI's new o1 chain-of-thought models

#187

Post by brna-2 »

fragmede wrote: post #151
pedrosorio wrote: post #110

Earlier quoted context omitted.

> It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output This is not an apt description of the system that insists the doctor is the mother of the boy involved in a car accident when elementary understanding of English and very little logic show that answer to be obviously wrong. https://x.com/colin_fraser/status/183…

what's weird is it gets it right when I try it. https://chatgpt.com/share/66e3601f-4bec-8009-ac0c-57bfa4f059...

Waat, got it on second try:

This is possible because the doctor is the boy's other parent—his father or, more likely given the surprise, his mother. The riddle plays on the assumption that doctors are typically male, but the doctor in this case is the boy's mother. The twist highlights gender stereotypes, encouraging us to question assumptions about roles in society.

Re: Notes on OpenAI's new o1 chain-of-thought models

#188

Post by troupo »

layer8 wrote: post #23
tptacek wrote: post #20

Earlier quoted context omitted.

I've had the opposite experience with some coding samples. After reading Nick Carlini's post, I've gotten into the habit of powering through coding problems with GPT (where previously I'd just laugh and immediately give up) by just presenting it the errors in its code and asking it to fix them. o1 seems to be effectively screening for some of those errors (I assume it's just some, but I've noticed that the o1 things…

My experience is likely colored by the fact that I tend to turn to LLMs for problems I have trouble solving by myself. I typically don't use them for the low-hanging fruits. That's the frustrating thing. LLMs don't materially reduce the set of problems where I'm running against a wall or have trouble finding information.

> That's the frustrating thing. LLMs don't materially reduce the set of problems where I'm running against a wall or have trouble finding information.

As you step outside regular Stack Overflow questions for top-3 languages, you run into limitations of these predictive models.

There's no "reasoning" behind them. They are still, largely, bullshit machines.

Re: Notes on OpenAI's new o1 chain-of-thought models

#189

Post by troupo »

spaceman_2020 wrote: post #162
layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

I don’t really see this as a massive problem. Its code. If it doesn’t run, you ask it to reconsider, give some more info if necessary, and it usually gets it right. The system doesn’t become useless if it takes 2 tries instead of 1 to get it right Still saves an incredible amount of time vs doing it yourself

I haven't found a single instance where it saved me any significant amount of time. In all cases I still had to rewrite the whole thing myself, or abandon endeavor.

And a few times the amount of time I spent trying to coax a correct answer out of AI trumped any potential savings I could've had

Re: Notes on OpenAI's new o1 chain-of-thought models

#190

Post by zaptrem »

lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

This is the bitter lesson/just put it in the model. They're trying to figure out more ways of converting compute to intelligence now that they're running out of text data: https://images.ctfassets.net/kftzwdyauwt9/7rMY55vLbGTlTiP9Gd...
Post reply on HN