Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

461–470 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#461

Post by ActionHank »

bambax wrote: post #281

Near the end, the quote from OpenAI researcher Jason Wei seems damning to me: > Results on AIME and GPQA are really strong, but that doesn’t necessarily translate to something that a user can feel. Even as someone working in science, it’s not easy to find the slice of prompts where GPT-4o fails, o1 does well, and I can grade the answer. But when you do find such prompts, o1 feels totally magical. We all need to find…

The stupidest thing about ai and automation is that they are trying to target it at large corporations looking to cut down on jobs or 10x productivity when all anyone actually wants is a robot to do their laundry and dishes.

Re: Notes on OpenAI's new o1 chain-of-thought models

#462

Post by empath75 »

pedrosorio wrote: post #110
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output This is not an apt description of the system that insists the doctor is the mother of the boy involved in a car accident when elementary understanding of English and very little logic show that answer to be obviously wrong. https://x.com/colin_fraser/status/183…

The reason why that question is a famous question is that _many humans get it wrong_.

Re: Notes on OpenAI's new o1 chain-of-thought models

#463

Post by empath75 »

layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

You should not be asking it questions that require it to already know detailed information about apis and libraries. It is not good at that, and it will never be good at that. If you need it to write code that uses a particular library or api, include the relevant documentation and examples.

It's your right to dismiss it, if you want, but if you want to get some value out of it, you should play to it's strengths and not look for things that it fails at as a gotcha.

Re: Notes on OpenAI's new o1 chain-of-thought models

#464

Post by codyvoda »

ActionHank wrote: post #461
bambax wrote: post #281

Near the end, the quote from OpenAI researcher Jason Wei seems damning to me: > Results on AIME and GPQA are really strong, but that doesn’t necessarily translate to something that a user can feel. Even as someone working in science, it’s not easy to find the slice of prompts where GPT-4o fails, o1 does well, and I can grade the answer. But when you do find such prompts, o1 feels totally magical. We all need to find…

The stupidest thing about ai and automation is that they are trying to target it at large corporations looking to cut down on jobs or 10x productivity when all anyone actually wants is a robot to do their laundry and dishes.

these are almost entirely unrelated problems

Re: Notes on OpenAI's new o1 chain-of-thought models

#465

Post by awb »

berniedurfee wrote: post #381
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

I’m starting to think this is an unsolvable problem with LLMs. The very act of “reasoning” requires one to know that they don’t know something. LLMs are giant word Plinko machines. A million monkeys on a million typewriters. LLMs are not interns. LLMs are assumption machines. None of the million monkeys or the collective million monkeys are “reasoning” or are capable of knowing. LLMs are a neat parlor trick and are s…

It probably depends on your problem space. In creative writing, I wonder if its even perceptible if the LLM is creating content at the boundaries of its knowledge base. But for programming or other falsifiable (and rapidly changing) disciplines it is noticeable and a problem.

Maybe some evaluation of the sample size would be helpful? If the LLM has less than X samples of an input word or phrase it could include a cautionary note in its output, or even respond with some variant of “I don’t know”.

Re: Notes on OpenAI's new o1 chain-of-thought models

#466

Post by acedTrex »

MPSimmons wrote: post #106
layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern

So mostly useless then?

Re: Notes on OpenAI's new o1 chain-of-thought models

#467

Post by HarHarVeryFunny »

joelburget wrote: post #453
HarHarVeryFunny wrote: post #451

I think Rich Sutton's bitter lesson will prove to apply here, and what we really need to advance machine learning capabilities are more general and powerful models capable of learning for themselves - better able to extract and use knowledge from the firehose of data available from the real world (ultimately via some form of closed-loop deployment where they can act and incrementally learn from their own actions). Wh…

o1 is an application of the Bitter Less. To quote Sutton: "The two methods that seem to scale arbitrarily in this way are search and learning." (emphasis mine -- in the original Sutton also emphasized learning ). OpenAI and others have previously pushed the learning side, while neglecting search. Now that gains from adding compute at training time have started to level off, they're adding compute at inference time.

I think the key part of the bitter lesson is that (scalable) ability to learn from data should be favored over built-in biases.

There are at least three major built-in biases in GPT-O1:

- specific reasoning heuristics hard coded in the RL decision making

- the architectural split between pre-trained LLM and what appears to be a symbolic agent calling it

- the reliance on one-time SGD driven learning (common to all these pre-trained transformers)

IMO search (reasoning) should be an emergent behavior of a predictive architecture capable of continual learning - chained what-if prediction.

Re: Notes on OpenAI's new o1 chain-of-thought models

#468

Post by Workaccount2 »

ChicagoDave wrote: post #137

It’s still just a tool. It does not reason. It has some add-on logic the simulates it. We’re no closer to “AI” today than we were 20 years ago.

It's funny (and sad) when you can tell someone is old because they are still holding onto an epiphany or belief they solidified 20 years ago, but because those 20 years flew by, they never realized how outdated that belief became.

I catch this happening to myself more and more as I get older, where I realize something I confidently state as true might be totally out of date, because, oh wow, holy shit how did 10 years go by since I was last deep into that topic!?

Re: Notes on OpenAI's new o1 chain-of-thought models

#469

Post by TeMPOraL »

kasdfasH wrote: post #332
grey-area wrote: post #199

Earlier quoted context omitted.

The 'riddle': A woman and her son are in a car accident. The woman is sadly killed. The boy is rushed to hospital. When the doctor sees the boy he says "I can't operate on this child, he is my son". How is this possible? GPT Answer: The doctor is the boy's mother Real Answer: Boy = Son, Woman = Mother (and her son), Doctor = Father (he says...he is my son) This is not in fact a riddle (though presented as one) and th…

The original riddle is of course: "A father and his son are in a car accident [...] When the boy is in hospital, the surgeon says: This is my child, I cannot operate on him". In the original riddle the answer is that the surgeon is female and the boy's mother. The riddle was supposed to point out gender stereotypes. So, as usual, ChatGPT fails to answer the modified riddle and gives the plagiarized stock answer and e…

> So, as usual, ChatGPT fails to answer the modified riddle and gives the plagiarized stock answer and explanation to the original one. No intelligence here.

Or, fails in the same way any human would, when giving a snap answer to a riddle told to them on the fly - typically, a person would recognize a familiar riddle half of the first sentence in, and stop listening carefully, not expecting the other party to give them a modified version.

It's something we drill into kids in school, and often into adults too: read carefully. Because we're all prone to pattern-matching the general shape to something we've seen before and zoning out.

Re: Notes on OpenAI's new o1 chain-of-thought models

#470

Post by riku_iki »

freediver wrote: post #141

Not seeing major advance in quality with o1, but seeing major negative impact on cost and latency. Kagi LLM benchmarking project: https://help.kagi.com/kagi/ai/llm-benchmark.html

interesting that Gemini performs extremely poor in those benchmarks.
Post reply on HN