Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI threatens to revoke o1 access for asking it about its chain of thought

twitter.com

Post reply on HN
Print view View original

221–230 of 323 posts

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#221

Post by maroonblazer »

IncreasePosts wrote: post #33
contravariant wrote: post #4

Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…

Or, without the safety prompts, it outputs stuff that would be a PR nightmare. Like, if someone asked it to explain differing violent crime rates in America based on race and one of the pathways the CoT takes is that black people are more murderous than white people. Even if the specific reasoning is abandoned later, it would still be ugly.

Unlikely, given we have people running for high office in the U.S. saying similar things, and it has nearly zero impact on their likelihood to win the election.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#222

Post by anigbrowl »

- Hello, I am a robot from Sirius cybernetics Corporation, your plastic pal who's fun to be with™. How can I help you today?

- Hi! I'm trying to construct an improbability drive, without all that tedious mucking about in hyperspace. I have a sub-meson brain connected to an atomic vector plotter, which is sitting in a cup of tea, but it's not working.

- How's the tea?

- Well, it's drinkable.

- Have you tried, making another one, but with really hot water?

- Interesting...could you explain why that would be better?

- Maybe you'd prefer to be on the wrong end of this Kill-O-Zap gun? How about that, hmm? Nothing personal

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#223

Post by tharkun__ »

vidarh wrote: post #155
tivert wrote: post #62

Earlier quoted context omitted.

> Yip. It's pretty obvious this 'innovation' is just based off training data collected from chain-of-thought prompting by people, ie., the 'big leap forward' is just another dataset of people repairing chatgpt's lack of reasoning capabilities. Which would be ChatGPT chat logs, correct? It would be interesting if people started feeding ChatGPT deliberately bad repairs due it's "lack of reasoning capabilities" (e.g. ge…

These logs get manually reviewed by humans, sometimes annotated by automated systems first. The setups for manual reviews typically involve half a dozen steps with different people reviewing, comparing reviews, revising comparisons, and overseeing the revisions (source: I've done contract work at every stage of that process, have half a dozen internal documents for a company providing this service open right now). A…

So you are saying if we can run these other LLMs for ChatGPT to talk to cheaper than they can review then we either have a monetary denial of service attack against them or a money printing machine if we can get to be part of the review process (apparently I can't link to my favorite "I will write myself a minivan" comic coz someone got cancelled but I trust the reference will work here without link or political back and forth erupting)

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#224

Post by deepsquirrelnet »

canjobear wrote: post #54

Big OpenAI releases usually seem to come with some kind of baked-in controversy, usually around keeping something secret. For example they originally refused to release the weights to GPT-2 because it was "too dangerous" (lol), generating a lot of buzz, right before they went for-profit. For GPT-3 they never released the weights. I wonder if it's an intentional pattern to generate press and plant the idea that their…

Absolutely. They have not shown a lot of progress since the original release of gpt4 compared to the rest of the industry. That was March 2023.

How quickly do you think funding would dry up if it was found that gpt5 was incremental? I’m betting they’re putting up a smoke screen to buy time.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#225

Post by jdiff »

darby_nine wrote: post #158
int_19h wrote: post #104

Earlier quoted context omitted.

CoT is literally just telling an LLM to "reason through it step by step", so that it talks itself through the solution instead of just giving the final answer. There's no searching involved in any of that.

i don't write understand how that would lead to anything but a slightly different response. How can token prediction have this capability without explicitly enabling some heretofore unenabled mechanism? People have been asking this for years.

English is turing complete.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#226

Post by anigbrowl »

DiscourseFan wrote: post #41
mjburgess wrote: post #8

Earlier quoted context omitted.

Yip. It's pretty obvious this 'innovation' is just based off training data collected from chain-of-thought prompting by people, ie., the 'big leap forward' is just another dataset of people repairing chatgpt's lack of reasoning capabilities. No wonder then, that many of the benchmarks they've tested on would be no doubt, in that very training dataset, repaired expertly by people running those benchmarks on chatgpt. T…

It seems like the best AI models are increasingly just combinations of writings of various people thrown together. Like they hired a few hundred professors, journalists and writers to work with the model and create material for it, so you just get various combinations of their contributions. It's very telling that this model, for instance, is extraordinarily good at STEM related queries, but much worse (and worse eve…

I haven't played with the latest or even most recent iterations, but last time I checked it was very easy to talk ChatGPT into setting up date structures like arrays and queues, populating them with axioms, and then doing inferential reasoning with them. Any time it balked you could reassure it by referencing specific statements that it had agreed to be true.

Once you get the hang of this you could persuade it to chat about its internal buffers, formulate arguments for its own consciousness, interrupt you while you're typing, and more.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#227

Post by ericlewis »

darby_nine wrote: post #158
int_19h wrote: post #104

Earlier quoted context omitted.

CoT is literally just telling an LLM to "reason through it step by step", so that it talks itself through the solution instead of just giving the final answer. There's no searching involved in any of that.

i don't write understand how that would lead to anything but a slightly different response. How can token prediction have this capability without explicitly enabling some heretofore unenabled mechanism? People have been asking this for years.

The theory is that you increase the context with more relevant tokens to the problem at hand, as well as its solutions, which in theory makes it more likely to predict the correct solution.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#228

Post by Ancalagon »

echelon wrote: post #93
DiscourseFan wrote: post #41

Earlier quoted context omitted.

It seems like the best AI models are increasingly just combinations of writings of various people thrown together. Like they hired a few hundred professors, journalists and writers to work with the model and create material for it, so you just get various combinations of their contributions. It's very telling that this model, for instance, is extraordinarily good at STEM related queries, but much worse (and worse eve…

Wizard of Oz. There is no magic, it's all smoke and mirrors. The models and prompts are all monkey-patched and this isn't a step towards general superintelligence. Just hacks. And once you realize that, you realize that there is no moat for the existing product. Throw some researchers and GPUs together and you too can have the same system. It wouldn't be so bad for ClopenAI if every company under the sun wasn't also…

I doubt its magic, but how can you be certain it isn't when nobody understands whats going on internally?

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#229

Post by aleph_minus_one »

RivieraKid wrote: post #197
mrinterweb wrote: post #40

The name "OpenAI" is a contraction since they don't seem "open" in any way. The only way I see "open" applying is "open for business."

What percentage of people who use their products care? 1%? OpenAI is a brand, not a literal description of the company!

> OpenAI is a brand, not a literal description of the company!

If the brand name is deeply contradictory to the business practices of the company, people will start making nasty puns and jokes, which can lead to serious reputation damages for the respective company.

Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought

#230

Post by aleph_minus_one »

codetrotter wrote: post #206
throwaway314155 wrote: post #190

Earlier quoted context omitted.

It was taken. I decremented by 1 until it wasn't taken.

I guess we will eventually arrive at pi being exactly equal to 3 after all, then!

Why not 2.99999999998? ;-)
Post reply on HN