OpenAI threatens to revoke o1 access for asking it about its chain of thought • Hacker News Board

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#71

Post by decremental » Fri, Sep 13, 2024, 9:20 PM UTC

IncreasePosts wrote: post #33

contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

Or, without the safety prompts, it outputs stuff that would be a PR nightmare. Like, if someone asked it to explain differing violent crime rates in America based on race and one of the pathways the CoT takes is that black people are more murderous than white people. Even if the specific reasoning is abandoned later, it would still be ugly.

The real danger of an advanced artificial intelligence is that it will make conclusions that regular people understand but are inconvenient for the regime. The AI must be aligned so that it will maintain the lies that people are supposed to go along with.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#72

Post by owenpalmer » Fri, Sep 13, 2024, 9:21 PM UTC

mrinterweb wrote: post #40

The name "OpenAI" is a contraction since they don't seem "open" in any way. The only way I see "open" applying is "open for business."

They have several open models, including Whisper.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#73

Post by accountnum » Fri, Sep 13, 2024, 9:22 PM UTC

mjburgess wrote: post #24

bugglebeetle wrote: post #12

Earlier quoted context omitted.

> the 'big leap forward' is just another dataset Yeah, that’s called machine learning.

You may want to file a complaint with OpenAI then, in their latest interface they call sampling from these prior conversations they've recorded, "thinking".

They're not sampling from prior conversations. The model constructs abstracted representations of the domain-specific reasoning traces. Then it applies these reasoning traces in various combinations to solve unseen problems.

If you want to call that sampling, then you might as well call everything sampling.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#74

Post by tptacek » Fri, Sep 13, 2024, 9:22 PM UTC

n42 wrote: post #69

tptacek wrote: post #57

Earlier quoted context omitted.

What do you mean, "anti-competitive"? There is no rule of competition that says you need to reveal trade secrets to your competitors.

isn't it such that saying something is anti-competitive doesn't necessarily mean 'in violation of antitrust laws'? it usually implies it, but I think you can be anti-competitive without breaking any rules (or laws). I do think it's sort of unproductive/inflammatory in the OP, it isn't really nefarious not to want people to have easy access to your secret sauce.

In what sense is not giving your competitors ammunition "anti-competitive"? That seems pretty competitive to me. More to the point: it's almost universally how competition in our economy actually works.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#75

Post by Vegenoid » Fri, Sep 13, 2024, 9:23 PM UTC

contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

> Honestly with all the hubbub about superintelligence you'd almost think o1 is secretly plotting the demise of humanity but is not yet smart enough to completely hide it

I think the most likely scenario is the opposite: seeing the chain of thought would both reveal its flaws and allow other companies to train on it.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#76

Post by int_19h » Fri, Sep 13, 2024, 9:23 PM UTC

The best part is that you still get charged per token for those CoT tokens that you're not allowed to ask it about.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#77

Post by darby_nine » Fri, Sep 13, 2024, 9:23 PM UTC

Skyy93 wrote: post #64

darby_nine wrote: post #21

"chain of thought" is just search, right? Wouldn't it make sense to tailor the search with heuristics relevant to the problem at hand?

No it is not just search. Chain of thought is the generation of new context from the inputs combined with a divide and conquer strategy. The model does not really searches it just breaks the problem in smaller chunks.

I don't get the distinction. Are you not just searching through chunks?

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#78

Post by mjburgess » Fri, Sep 13, 2024, 9:23 PM UTC

accountnum wrote: post #73

mjburgess wrote: post #24

Earlier quoted context omitted.

You may want to file a complaint with OpenAI then, in their latest interface they call sampling from these prior conversations they've recorded, "thinking".

They're not sampling from prior conversations. The model constructs abstracted representations of the domain-specific reasoning traces. Then it applies these reasoning traces in various combinations to solve unseen problems. If you want to call that sampling, then you might as well call everything sampling.

They're generative models. By definition, they are sampling from a joint distribution of text tokens fit by approximation to an empirical distribution.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#79

Post by varenc » Fri, Sep 13, 2024, 9:24 PM UTC

htrp wrote: post #67

The o1 model already pretty much explains exactly how it runs the chain of thought though? Unless there is some special system instruction that you've specifically fine tuned for?

I too am confused by this. When using the chatgpt.com interface it seems to expose its chain-of-thought quite obviously? Or the "chain-of-thought" available from chatgpt.com isn't the real chain-of-thought? Here's an example screenshot: https://dl.dropboxusercontent.com/s/ecpbkt0yforhf20/chain-of...

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#80

Post by yard2010 » Fri, Sep 13, 2024, 9:24 PM UTC

archgoon wrote: post #18

I mean, say what you want about Meta only releasing the weights and calling it open source, what they're doing is better than this.

Facebook created products to induce mental illness for the lolz (and bank accounts I guess?) of the lizards behind it[0]

IMHO people like these are the most dangerous to human society, because unlike regular criminals, they find their ways around the consequences to their actions.

[0] https://slate.com/technology/2017/11/facebook-was-designed-t...