Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI threatens to revoke o1 access for asking it about its chain of thought

twitter.com

Post reply on HN
Print view View original

11–20 of 323 posts

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#11

Post by huevosabio »

contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

My bet: they use formal methods (like an interpreter running code to validate, or a proof checker) in a loop.

This would explain: a) their improvement being mostly on the "reasoning, math, code" categories and b) why they wouldn't want to show this (its not really a model, but an "agent").

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#12

Post by bugglebeetle »

mjburgess wrote: post #8
qsort wrote: post #6

Earlier quoted context omitted.

Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.

Yip. It's pretty obvious this 'innovation' is just based off training data collected from chain-of-thought prompting by people, ie., the 'big leap forward' is just another dataset of people repairing chatgpt's lack of reasoning capabilities. No wonder then, that many of the benchmarks they've tested on would be no doubt, in that very training dataset, repaired expertly by people running those benchmarks on chatgpt. T…

> the 'big leap forward' is just another dataset

Yeah, that’s called machine learning.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#14

Post by andix »

huevosabio wrote: post #11
contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

My bet: they use formal methods (like an interpreter running code to validate, or a proof checker) in a loop. This would explain: a) their improvement being mostly on the "reasoning, math, code" categories and b) why they wouldn't want to show this (its not really a model, but an "agent").

My understanding was from the beginning that it’s an agent approach (a self prompting feedback loop).

They might’ve tuned the model to perform better with an agent workload than their regular chat model.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#15

Post by arthurcolle »

contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

> Honestly with all the hubbub about superintelligence you'd almost think o1 is secretly plotting the demise of humanity but is not yet smart enough to completely hide it.

Yeah, using the GPT-4 unaligned base model to generate the candidates and then hiding the raw CoT coupled with magic superintelligence in the sky talk is definitely giving https://www.reddit.com/media?url=https%3A%2F%2Fi.redd.it%2Fb... vibes

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#16

Post by JumpCrisscross »

qsort wrote: post #6
contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.

> there is no secret sauce and they're afraid someone trains a model on the output

OpenAI is fundraising. The "stop us before we shoot Grandma" shtick has a proven track record: investors will fund something that sounds dangerous, because dangerous means powerful.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#17

Post by SecretDreams »

contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

> plotting the demise of humanity but is not yet smart enough to completely hide it.

I feel like if my demise is imminent, I'd prefer it to be hidden. In that sense, sounds like o1 is a failure!

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#19

Post by skeledrew »

jsheard wrote: post #10
mrcwinn wrote: post #7

Earlier quoted context omitted.

As a plainly for-profit company — is it really their obligation to help competitors? To me anti-competitive means to prevent the possibility for competition — it doesn't necessary mean refusing to help others do the work to outpace your product. Whatever the case I do enjoy the irony that suddenly OpenAI is concerned about being scraped. XD

> Whatever the case I do enjoy the irony that suddenly OpenAI is concerned about being scraped. XD Maybe it wasn't enforced this aggressively, but they've always had a TOS clause saying you can't use the output of their models to train other models. How they rationalize taking everyone else's data for training while forbidding using their own data for training is anyones guess.

Scraping for me, but not for thee.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#20

Post by aeternum »

Disappointing especially since the stress the importance of seeing the chain of thought to ensure AI safety. Seems it is safety for me but not for thee.

If history is our guide, we should be much more concerned about those who control new technology rather than the new technology itself.

Keep your eye not on the weapon, but upon those who wield it.

Post reply on HN