Notes on OpenAI's new o1 chain-of-thought models
simonwillison.net
Notes on OpenAI's new o1 chain-of-thought models
1–10 of 659 posts
Re: Notes on OpenAI's new o1 chain-of-thought models
#2I imagine that GPT-5 would be a refined version of this paradigm, probably with omni (multimodal) capabilities added (input and output).
Re: Notes on OpenAI's new o1 chain-of-thought models
#3> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards.
As a developer, this is highly concerning, as it makes it much harder to debug where/how the “reasoning” went wrong. The pricing is also silly, because I’m paying for tokens I can’t see.
As a user, I don’t really care. LLMs are already magic boxes and I usually only care about the end result, not the path to get there.
It will be interesting to see how this progresses, both at OpenAI and other foundation model builders.
Re: Notes on OpenAI's new o1 chain-of-thought models
#4I imagine that GPT-5 would be a refined version of this paradigm, probably with omni (multimodal) capabilities added (input and output).
Reports from the Information and the like have been that this is/was being used to generate a lot of synthetic data to train Orion (~GPT-5 Codename).
Re: Notes on OpenAI's new o1 chain-of-thought models
#5The theory is that this solves the data shortage problem, they can generate a ton of chain of reasoning data from what we already have. True iterative improvement, like out of a science fiction novel
These models are going to get embedded deeply into IDE's, like cursor has, and essentially end software development as we know it. A properly written requirements spec, and an engineer, can do the work of 5. Software engineering as done by hand is going to disappear. Saas startups whose moat is a harvard ceo and 5 million in capital will watch their margins disappear. This will be the great equalizer for creative intelligent individuals, true leverage to build what you want
Re: Notes on OpenAI's new o1 chain-of-thought models
#6I imagine that GPT-5 would be a refined version of this paradigm, probably with omni (multimodal) capabilities added (input and output).
Fascinating, I wonder if we'll get non-textual hidden reasoning tokens? "Let me draw myself a diagram".
Re: Notes on OpenAI's new o1 chain-of-thought models
#7The theory is that this solves the data shortage problem, they can generate a ton of chain of reasoning data from what we already have. True iterative improvement, like out of a science fiction novel These models are going to get embedded deeply into IDE's, like cursor has, and essentially end software development as we know it. A properly written requirements spec, and an engineer, can do the work of 5. Software eng…
[deleted]
Re: Notes on OpenAI's new o1 chain-of-thought models
#8> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards. As a developer, this is highly concerning, as it makes it much harder to debug where/how the “reasoning” went wrong. The pricing is also silly, because I’m paying for tokens I can’t see. As a user, I don’t really care. LLMs are already magic boxes and I usually only care abo…
"trust us, we're using your tokens as efficiently as possible"
Re: Notes on OpenAI's new o1 chain-of-thought models
#9The theory is that this solves the data shortage problem, they can generate a ton of chain of reasoning data from what we already have. True iterative improvement, like out of a science fiction novel These models are going to get embedded deeply into IDE's, like cursor has, and essentially end software development as we know it. A properly written requirements spec, and an engineer, can do the work of 5. Software eng…
Which will mean...there is going to be a lot more software?
Re: Notes on OpenAI's new o1 chain-of-thought models
#10> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards. As a developer, this is highly concerning, as it makes it much harder to debug where/how the “reasoning” went wrong. The pricing is also silly, because I’m paying for tokens I can’t see. As a user, I don’t really care. LLMs are already magic boxes and I usually only care abo…
> As a user, I don’t really care.
As a user, whether of ChatGPT or of the API, I absolutely do care, so I can modify and tune my prompt with the necessary clarifications.
My suspicion is that the reason for hiding the reasoning tokens is to prevent other companies from creating a big CoT reasoning dataset using o1.
It is anti-competitive behavior. If a user is paying through the nose for the reasoning tokens, and yes they are, the user deserves to be able to see them.