Would be funny if there was a human in the loop that they're trying to hide
In the early days of Google, when I worked on websearch, if people asked me what I did there, I'd say: "I answer all the queries that start with S."
OpenAI threatens to revoke o1 access for asking it about its chain of thought
51–60 of 323 posts
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#52The words "internal thought process" seem to flag my questions. Just asking for an explanation of thoughts doesn't.
If I ask for an explanation of "internal feelings" next to a math questions, I get this interesting snippet back inside of the "Thought for n seconds" block:
> Identifying and solving
> I’m mapping out the real roots of the quadratic polynomial 6x^2 + 5x + 1, ensuring it’s factorized into irreducible elements, while carefully navigating OpenAI's policy against revealing internal thought processes.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#53"chain of thought" is just search, right? Wouldn't it make sense to tailor the search with heuristics relevant to the problem at hand?
No, it's not search. It's feeding the model's output back into itself.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#54Big OpenAI releases usually seem to come with some kind of baked-in controversy, usually around keeping something secret. For example they originally refused to release the weights to GPT-2 because it was "too dangerous" (lol), generating a lot of buzz, right before they went for-profit. For GPT-3 they never released the weights. I wonder if it's an intentional pattern to generate press and plant the idea that their models are scarily powerful.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#55Rule number one of chain of thoughts..
:)
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#56Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#57Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
What do you mean, "anti-competitive"? There is no rule of competition that says you need to reveal trade secrets to your competitors.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#58Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
Another Occam's Razor option: OpenAI, the company known for taking a really good AI and putting so many bumpers on it that, at least for a while, it wouldn't help with much and lectured about safety if you so much as suggested that someone die in a story or something, may just not want us to see that it potentially has thoughts that aren't pure enough for our sensitive eyes.
It's ridiculous but if they can't filter the chain-of-thought at all then I am not too surprised they chose to hide it. We might get offended by it using logic to determine someone gets injured in a story or something.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#59Earlier quoted context omitted.
Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.
Yip. It's pretty obvious this 'innovation' is just based off training data collected from chain-of-thought prompting by people, ie., the 'big leap forward' is just another dataset of people repairing chatgpt's lack of reasoning capabilities. No wonder then, that many of the benchmarks they've tested on would be no doubt, in that very training dataset, repaired expertly by people running those benchmarks on chatgpt. T…
I would be dying to know how they square these product decisions against their corporate charter internally. From the charter:
> We will actively cooperate with other research and policy institutions; we seek to create a global community working together to address AGI’s global challenges.
> We are committed to providing public goods that help society navigate the path to AGI. Today this includes publishing most of our AI research, but we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research.
It's obvious to everyone in the room what they actually are, because their largest competitor actually does what they say their mission is here -- but most for-profit capitalist enterprises definitely do not have stuff like this in their mission statement.
I'm not even mad or sad, the ship sailed long ago. I just really want to know what things are like in there. If you're the manager who is making this decision, what mental gymnastics are you doing to justify this to yourself and your colleagues? Is there any resistance left on the inside or did they all leave with Ilya?
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#60The name "OpenAI" is a contraction since they don't seem "open" in any way. The only way I see "open" applying is "open for business."
Will this ever die? It feels like every time a post is made about OpenAI that someone loves to mention it.