Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

471–480 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#471

Post by 0xdeadbeefbabe »

williamdclt wrote: post #164
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

A lot of interns are overconfident though

Re: Notes on OpenAI's new o1 chain-of-thought models

#472

Post by Jensson »

TeMPOraL wrote: post #392
KoolKat23 wrote: post #363

Earlier quoted context omitted.

I'm noticing a strange common theme in all these riddles, it's being asked and getting wrong. They're all badly worded questions. The model knows something is up and reads into it too much. In this case it's tautology, you would usually say "a mother and her son...". I think it may answer correctly if you start off asking "Please solve the below riddle:" There was another example yesterday which it solved correctly a…

Yup. The models fail on gotcha questions asked without warning, especially when evaluated on the first snap answer. Much like approximately all humans.

> especially when evaluated on the first snap answer

The whole point of o1 is that it wasn't "the first snap answer", it wrote half a page internally before giving the same wrong answer.

Re: Notes on OpenAI's new o1 chain-of-thought models

#473

Post by 0xdeadbeefbabe »

awb wrote: post #465
berniedurfee wrote: post #381

Earlier quoted context omitted.

I’m starting to think this is an unsolvable problem with LLMs. The very act of “reasoning” requires one to know that they don’t know something. LLMs are giant word Plinko machines. A million monkeys on a million typewriters. LLMs are not interns. LLMs are assumption machines. None of the million monkeys or the collective million monkeys are “reasoning” or are capable of knowing. LLMs are a neat parlor trick and are s…

It probably depends on your problem space. In creative writing, I wonder if its even perceptible if the LLM is creating content at the boundaries of its knowledge base. But for programming or other falsifiable (and rapidly changing) disciplines it is noticeable and a problem. Maybe some evaluation of the sample size would be helpful? If the LLM has less than X samples of an input word or phrase it could include a cau…

> It probably depends on your problem space

Makes me wonder if the medical doctors can ever blame the LLM over other factors for killing their patients.

Re: Notes on OpenAI's new o1 chain-of-thought models

#474

Post by mediaman »

lucianbr wrote: post #294
mediaman wrote: post #63

Earlier quoted context omitted.

The simple answer is: I don't care. I'll statistically figure out what the typical total cost per call is from experience, and that's what matters. Who cares if they lie about it, if the model's cost per call fits my budget? If it starts costing $1 per call, and that's too high, then I just won't use it commercially. Whether it was $1 because they inflated the token count or because it just actually took a lot of tok…

The thing is it might increase in cost after you've decided to use it commercially, and have invested a lot of time and resources in it. Now it's very hard to move to something else, but very easy for OpenAI to increase your cost arbitrarily. The statistics you made are not binding for them.

The API returns how many tokens were used in reasoning, so it would be easy to see any average change in reasoning token consumption. And token prices in general have been extremely deflationary over the last 18 months.

Re: Notes on OpenAI's new o1 chain-of-thought models

#475

Post by Jensson »

yreg wrote: post #264
eigenket wrote: post #238

Earlier quoted context omitted.

"When the doctor sees the boy he says" Indicates the gender of the father.

A mother can have a male gender. I wonder if this interpretation is a result of attempts to make the model more inclusive than the corpus text, resulting in a guess that's unlikely, but not strictly impossible.

> A mother can have a male gender.

Then it would be a father, misgendering him as a mother is not nice.

Re: Notes on OpenAI's new o1 chain-of-thought models

#476

Post by SubiculumCode »

ChicagoDave wrote: post #435
SubiculumCode wrote: post #214

Earlier quoted context omitted.

At what point are we better described as tools?

Humans can be a lot of things. AI can only do what it knows and what it’s been programmed to do.

Please do something that you don't know.

Re: Notes on OpenAI's new o1 chain-of-thought models

#477

Post by TeMPOraL »

EagnaIonat wrote: post #376
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern. You are falling into the trap that everyone does. In anthropomorphising it. It doesn't understand anything you say. It just statistically knows what a likely response would be. Treat it as text completion and you can get more accurate answers.

> You are falling into the trap that everyone does. In anthropomorphising it. It doesn't understand anything you say.

And an intern does?

Anthropomorphising LLMs isn't entirely incorrect: they're trained to complete text like a human would, in completely general setting, so by anthropomorphising them you're aligning your expectations with the models' training goals.

Re: Notes on OpenAI's new o1 chain-of-thought models

#478

Post by mattnewton »

gloosx wrote: post #193
mattnewton wrote: post #176

Earlier quoted context omitted.

Claude 3.5 seems to get it, (thought the method is not what I would use?) - ——— Let's think through this step-by-step: We're looking for three numbers a, b, and c where: (a / b) / c = a + b + c ———— Certainly, let's approach this problem step-by-step: 1) First, let's write out our equation: (a / b) / c = a + b + c 2) To simplify the left side, we can rewrite it as: a / (b * c) = a + b + c 3) Now, to solve this, we ne…

It's crazy that it just tries to bruteforce it by picking numbers, and in your case it took more steps before concluding a success/failure, which seems quite to be random to me, or at least dependent on something. What's clear is that it doesn't have any idea about mathematical deduction and induction – a real chain-of-thought which kids learn in 5th grade.

Lots of people don’t either. I think it probably just needs more 5th grade math problems in the rlhf corpus :)

Re: Notes on OpenAI's new o1 chain-of-thought models

#480

Post by dpatrick86 »

simonw wrote: post #160
ChicagoDave wrote: post #137

It’s still just a tool. It does not reason. It has some add-on logic the simulates it. We’re no closer to “AI” today than we were 20 years ago.

Personally I think “add-on logic that simulates reasoning” is a pretty good match for the “artificial” part of “artificial intelligence”. I’ve been tryin out the alternative term “initiation intelligence” recently, mainly to work around the baggage that’s become attached to the term AI.

Artificial is fine and playing word games for pedants is a trap.
Post reply on HN