Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

91–100 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#91

Post by lukev »

29athrowaway wrote: post #88
lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

Well, if LLMs are system 1, this difference would be building towards system 2. https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow

Yes indeed, and personally if we have AGI I believe it will arise from multiple LLMs working in tandem with other types of machine learning, databases for "memory", more traditional compute functions, and a connectivity layer between them all.

But to my knowledge, that's not the kind of research OpenAI is doing. They seem mostly focused on training bigger and better models and seeking AGI through emergence in those.

Re: Notes on OpenAI's new o1 chain-of-thought models

#92

Post by motoboi »

layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

Just pass a link to a GitHub issue and ask for a response or even a webpage to summarize and will see the beautiful hallucinations it will come up to as the model is not web browsing yet.

Re: Notes on OpenAI's new o1 chain-of-thought models

#93

Post by achierius »

ldjkfkdsjnv wrote: post #15
OutOfHere wrote: post #12

Earlier quoted context omitted.

> A properly written requirements spec, and an engineer, can do the work of 5. I do not think this will scale. GPT o1 is presumably good for bootstrapping a project using tools that the engineer is not familiar with. The model will struggle to update a sizable codebase, however, with dependencies between the files. Secondly, no matter the size of the codebase and no matter the model used, the engineer still has to re…

I respectfully, but completely disagree. Right now with sonnet 3.5 + cursor ide, I'm not writing that much of my own code at my FAANG job. I am generating a ton, passing in documentation from internal libraries, iterating on the result. Most of the time, I just accept its changes. This is going to rapidly happen. All we need are a few more model releases, not even a step function improvement

Not everyone has the same experience with the replaceability of their job role as you do. I've tried pretty hard and it just doesn't work for me. Admittedly I'm in compilers which makes it a bit harder, but just in general there are a lot of engineers who are in the same relative position.

Re: Notes on OpenAI's new o1 chain-of-thought models

#94

Post by chuckledog »

salomonk_mur wrote: post #59
lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

The innovation lies in making the whole loop available to an end user immediately, without them being a programmer. My grandma can build games using ChatGPT now.

Ada Lovelace is my grandma

Re: Notes on OpenAI's new o1 chain-of-thought models

#95

Post by _fs »

layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

o1-preview != o1.

In public coding AI comparison tests, results showed 4o scoring around 35%, o1-preview scoring ~50% and o1 scoring ~85%.

o1 is not yet released, but has been run through many comparison tests with public results posted.

Re: Notes on OpenAI's new o1 chain-of-thought models

#96

Post by feralderyl »

layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

I think this model is a precursor model that is designed for agentic behavior. I expect very soon OpenAI to allow this model tool use that will allow it to verify its code creations and whatever else it claims through use of various tools like a search engine, a virtual machine instance with code execution capabilities, api calling and other advanced tool use.

Re: Notes on OpenAI's new o1 chain-of-thought models

#97

Post by TechDebtDevin »

la64710 wrote: post #79

Please please please stop saying thought. This has nothing to do with the word thought. When we say the word thought it means something. Please don’t use the same word for whatever AI is doing and trivialize the word. Invent a new word if needed but for Pete’s sake be accurate and truthful.

Okay, what is a thought then?

Re: Notes on OpenAI's new o1 chain-of-thought models

#99

Post by fnordpiglet »

lukev wrote: post #85
fnordpiglet wrote: post #73

Earlier quoted context omitted.

One aspect that’s not achievable is they discuss hiding the chain of thought in its raw form because the chains are allowed to be unaligned. This allows the model to operate without any artifacts from alignment and apply them in the post processing, more or less. This requires effectively root and you would need the unaligned weights.

Ok but this presses on a latent question: what do we mean by alignment? Practically it's come to mean just sanitization... "don't say something nasty or embarrassing to users." But that doesn't apply here, the reasoning tokens are effectively just a debug log. If alignment means "conducting reasoning in alignment with human values", then misalignment in the reasoning phase could potentially be obfuscated and sanitize…

The challenge is alignment ends up changing the models in ways that aren’t representative of the actual training set and as I understand it this generally lowers the performance even for aligned things. Further the decision to summarize the chains of thought includes the answers that wouldn’t pass alignment themselves without removal. From what I read the final output is aligned but could have considered unaligned COT. In fact because they’re in the context they’re necessarily changing the final output even if the final output complies with the alignment. There are a few other “only root could do this,” which says yes anyone could implement these without secret sauce as long as they have a raw frontier model.

Re: Notes on OpenAI's new o1 chain-of-thought models

#100

Post by inerte »

OutOfHere wrote: post #37
jumploops wrote: post #3

> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards. As a developer, this is highly concerning, as it makes it much harder to debug where/how the “reasoning” went wrong. The pricing is also silly, because I’m paying for tokens I can’t see. As a user, I don’t really care. LLMs are already magic boxes and I usually only care abo…

> As a user, I don’t really care. Tell me: Just how is it fair for a user to pay for the reasoning tokens without actually seeing them? If they are not shared, the service can bill you anything they want for them!

OpenAI could have also figured out the average number of extra output tokens, and put a markup in overall API costs. As a user, I wouldn’t care either, because the price would mostly be the same.
Post reply on HN