"chain of thought" is just search, right? Wouldn't it make sense to tailor the search with heuristics relevant to the problem at hand?
OpenAI threatens to revoke o1 access for asking it about its chain of thought
21–30 of 323 posts
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#22Earlier quoted context omitted.
Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.
> there is no secret sauce and they're afraid someone trains a model on the output OpenAI is fundraising. The "stop us before we shoot Grandma" shtick has a proven track record: investors will fund something that sounds dangerous, because dangerous means powerful.
Counterpoint, a place like Civit.AI is at least as dangerous, yet it's nowhere near as well funded.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#23Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
My bet: they use formal methods (like an interpreter running code to validate, or a proof checker) in a loop. This would explain: a) their improvement being mostly on the "reasoning, math, code" categories and b) why they wouldn't want to show this (its not really a model, but an "agent").
I think it could be some of both. By giving access to the chain of thought one would able to see what the agent is correcting/adjusting for, allowing you to compile a library of vectors the agent is aware of and gaps which could be exploitable. Why expose the fact that you’re working to correct for a certain political bias and not another?
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#24Earlier quoted context omitted.
Yip. It's pretty obvious this 'innovation' is just based off training data collected from chain-of-thought prompting by people, ie., the 'big leap forward' is just another dataset of people repairing chatgpt's lack of reasoning capabilities. No wonder then, that many of the benchmarks they've tested on would be no doubt, in that very training dataset, repaired expertly by people running those benchmarks on chatgpt. T…
> the 'big leap forward' is just another dataset Yeah, that’s called machine learning.
You may want to file a complaint with OpenAI then, in their latest interface they call sampling from these prior conversations they've recorded, "thinking".
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#25Earlier quoted context omitted.
Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.
> there is no secret sauce and they're afraid someone trains a model on the output OpenAI is fundraising. The "stop us before we shoot Grandma" shtick has a proven track record: investors will fund something that sounds dangerous, because dangerous means powerful.
It seems ridiculous but I think it may have some credence. Perhaps it is because of sci-fi associating "dystopian" with "futuristic" technology, or because there is additional advertisement provided by third parties fearmongering (which may be a reasonable response to new scary tech?)
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#26Earlier quoted context omitted.
Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.
> there is no secret sauce and they're afraid someone trains a model on the output OpenAI is fundraising. The "stop us before we shoot Grandma" shtick has a proven track record: investors will fund something that sounds dangerous, because dangerous means powerful.
This is correct. Most people hear about AI from two sources, AI companies and journalists. Both have an incentive to make it sound more powerful than it is.
On the other hand this thing got 83% on a test I got 47% on...
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#27Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
Maybe they just have some people in a call center replying.
Pay no attention to the man behind the mechanical turk!
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#28Earlier quoted context omitted.
Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.
> there is no secret sauce and they're afraid someone trains a model on the output OpenAI is fundraising. The "stop us before we shoot Grandma" shtick has a proven track record: investors will fund something that sounds dangerous, because dangerous means powerful.
Millenarism is a seductive idea.
If you're among the last of your kind then you're very important, in a sense you're immortal. Living your life quietly and being forgotten is apparently scarier than dying in a blaze of glory defending mankind against the rise of the LLMs.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#29Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
> for this to work the model must have freedom to express its thoughts in unaltered form, so we cannot train any policy compliance or user preferences onto the chain of thought.
Which makes it sound like they really don't want it to become public what the model is 'thinking'
The internal chain of thought steps might contain things that would be problematic to the company if activists or politicians found out that the company's model was saying them.
Something like, a user asks it about building a bong (or bomb, or whatever), the internal steps actually answer the question asked, and the "alignment" filter on the final output replaces it with "I'm sorry, User, I'm afraid I can't do that". And if someone shared those internal steps with the wrong activists, the company would get all the negative attention they're trying to avoid by censoring the final output.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#30Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
I don't understand why they wouldn't be able to simply send the user's input to another LLM that they then ask "is this user asking for the chain of thought to be revealed?", and if not, then go about business as usual.