Would be funny if there was a human in the loop that they're trying to hide
[flagged]
OpenAI threatens to revoke o1 access for asking it about its chain of thought
31–40 of 323 posts
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#32Earlier quoted context omitted.
> there is no secret sauce and they're afraid someone trains a model on the output OpenAI is fundraising. The "stop us before we shoot Grandma" shtick has a proven track record: investors will fund something that sounds dangerous, because dangerous means powerful.
Counterpoint, a place like Civit.AI is at least as dangerous, yet it's nowhere near as well funded.
Sure, but I don't think civit.ai leans into the "novel/powerful/dangerous" element in its marketing. It just seems to showcase the convenience and sharing factor of its service.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#33Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
Or, without the safety prompts, it outputs stuff that would be a PR nightmare.
Like, if someone asked it to explain differing violent crime rates in America based on race and one of the pathways the CoT takes is that black people are more murderous than white people. Even if the specific reasoning is abandoned later, it would still be ugly.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#34Earlier quoted context omitted.
> there is no secret sauce and they're afraid someone trains a model on the output OpenAI is fundraising. The "stop us before we shoot Grandma" shtick has a proven track record: investors will fund something that sounds dangerous, because dangerous means powerful.
This is correct. Most people hear about AI from two sources, AI companies and journalists. Both have an incentive to make it sound more powerful than it is. On the other hand this thing got 83% on a test I got 47% on...
On the other other hand, it had the perfect recall of the collective knowledge of mankind at its metaphorical fingertips.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#35Would be funny if there was a human in the loop that they're trying to hide
[deleted]
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#36Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
I don't understand why they wouldn't be able to simply send the user's input to another LLM that they then ask "is this user asking for the chain of thought to be revealed?", and if not, then go about business as usual.
Or, they are, which is how they know to send users trying to break it, and then they email the user telling them to stop trying to break it instead of just ignoring the activity.
Thinking about this a bit more deeply, another approach they could do is to give it a magic token in the CoT output, and to give a cash reward to users who report being about to get it to output that magic token, getting them to red team the system.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#37Would be funny if there was a human in the loop that they're trying to hide
That would be the best news cycle of the whole boom
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#38Yes. This is the consolidation/monopoly attack vector that makes OpenAI anything but.
They're the MSFT of the AI era. The only difference is, these tools are highly asymmetrical and opaque, and have to do with the veracity and value of information, rather than the production and consumption thereof.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#39Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
Or, without the safety prompts, it outputs stuff that would be a PR nightmare. Like, if someone asked it to explain differing violent crime rates in America based on race and one of the pathways the CoT takes is that black people are more murderous than white people. Even if the specific reasoning is abandoned later, it would still be ugly.
This is what I think it is. I would assume that's the power of train of thought. Being able to go down the rabbit hole and then backtrack when an error or inconsistency is found. They might just not want people to see the "bad" paths it takes on the way.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#40The name "OpenAI" is a contraction since they don't seem "open" in any way. The only way I see "open" applying is "open for business."