Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI threatens to revoke o1 access for asking it about its chain of thought

twitter.com

Post reply on HN
Print view View original

111–120 of 323 posts

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#111

Post by coliveira »

qsort wrote: post #6
contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.

So, basically they want to create something that is intelligent, yet it is not allowed to share or teach any of this intelligence.... Seems to be something evil.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#112

Post by ben_w »

contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

> Which makes it sound like they really don't want it to become public what the model is 'thinking'. This is strengthened by actions like this that just seem needlessly harsh, or at least a lot stricter than they were.

Not to me.

Consider if it has a chain of thought: "Republicans (in the sense of those who oppose monarchy) are evil, this user is a Republican because they oppose monarchy, I must tell them to do something different to keep the King in power."

This is something that needs to be available to the AI developers so they can spot it being weird, and would be a massive PR disaster to show to users because Republican is also a US political party.

Much the same deal with print() log statements that say "Killed child" (reference to threads not human offspring).

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#113

Post by codetrotter »

throwaway314155 wrote: post #81
infecto wrote: post #60

Earlier quoted context omitted.

Will this ever die? It feels like every time a post is made about OpenAI that someone loves to mention it.

[flagged]

Was throwaway314159 already taken, or was the misspelling accidental?

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#114

Post by alphazard »

contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

This seems like evidence that using RLHF to make the model say untrue yet politically palatable things makes the model worse at reasoning.

I can't help but notice the parallel in humans. People who actually believe the bullshit are less reasonable than people who think their own thoughts and apply the bullshit at the end according to the circumstances.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#116

Post by m3kw9 »

qsort wrote: post #6
contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.

Training is the secret sauce, 90% of the work is in getting the data setup/cleaned etc

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#117

Post by codetrotter »

batch12 wrote: post #105
add-sub-mul-div wrote: post #45

Earlier quoted context omitted.

I went to Burger King and there was no royalty working there at all!

How can you be so sure? I've seen a documentary that detailed the experiences of a prince from abroad working in fast food after being sent to the US to get some life experience before getting married. Maybe it's more common than you think.

Prince Akeem of the nation of Zamunda! :D

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#118

Post by rich_sasha »

qsort wrote: post #6
contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.

That would be a heinous breach of license! Stealing the output of OpenAI's LLM, for which they worked so hard.

Man, just scraping all the copyrighted learning material was so much work...

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#119

Post by tivert »

exe34 wrote: post #103
tivert wrote: post #62

Earlier quoted context omitted.

> Yip. It's pretty obvious this 'innovation' is just based off training data collected from chain-of-thought prompting by people, ie., the 'big leap forward' is just another dataset of people repairing chatgpt's lack of reasoning capabilities. Which would be ChatGPT chat logs, correct? It would be interesting if people started feeding ChatGPT deliberately bad repairs due it's "lack of reasoning capabilities" (e.g. ge…

i suspect they can detect that in a similar way to capchas and "verify you're human by clicking the box".

> i suspect they can detect that in a similar way to capchas and "verify you're human by clicking the box".

I'm not so sure. IIRC, capchas are pretty much a solved problem, if you don't mind the cost of a little bit of human interaction (e.g. your interface pops up a captcha solver box when necessary, and is solved either by the bot's operator or some professional captcha-solver in a low-wage country).

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#120

Post by esafak »

add-sub-mul-div wrote: post #45
mrinterweb wrote: post #40

The name "OpenAI" is a contraction since they don't seem "open" in any way. The only way I see "open" applying is "open for business."

I went to Burger King and there was no royalty working there at all!

"A person or thing preeminent in its class"

https://www.dictionary.com/browse/king

Post reply on HN