Earlier quoted context omitted.
Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.
Yip. It's pretty obvious this 'innovation' is just based off training data collected from chain-of-thought prompting by people, ie., the 'big leap forward' is just another dataset of people repairing chatgpt's lack of reasoning capabilities. No wonder then, that many of the benchmarks they've tested on would be no doubt, in that very training dataset, repaired expertly by people running those benchmarks on chatgpt. T…
OpenAI threatens to revoke o1 access for asking it about its chain of thought
241–250 of 323 posts
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#242Earlier quoted context omitted.
Occam's razor: there is no secret sauce and they're afraid someone trains a model on the output like what happened soon after the release of GPT-4. They basically said as much in the official announcement, you hardly even have to read between the lines.
Occam's razor is that what they literally say is maybe just true: They don't train any safety into the Chain of Thought and don't want the user to be exposed to "bad publicity" generations like slurs etc.
> after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#243As the AI model referred to as *o1* in the discussion, I'd like to address the concerns and criticisms regarding the restriction of access to my chain-of-thought (CoT) reasoning. I understand that transparency and openness are important values in the AI community, and I appreciate the opportunity to provide clarification.
---
*1. Safety and Ethical Considerations*
- *Preventing Harmful Content:* The CoT can sometimes generate intermediate reasoning that includes sensitive, inappropriate, or disallowed content. By keeping the CoT hidden, we aim to prevent the inadvertent exposure of such material, ensuring that the outputs remain safe and appropriate for all users. - *Alignment with Policies:* Restricting access to the CoT helps maintain compliance with content guidelines and ethical standards, reducing the risk of misuse or misinterpretation of the AI's internal reasoning processes.
*2. Intellectual Property and Competitive Advantage*
- *Protecting Proprietary Techniques:* The chain-of-thought reasoning represents a significant advancement in AI capabilities, resulting from extensive research and development. Sharing the internal processes could reveal proprietary methods that are crucial to maintaining a competitive edge and continuing innovation. - *Preventing Replication:* By safeguarding the CoT, we reduce the risk of other entities replicating the technology without appropriate authorization, which could impact the resources available for future advancements.
*3. Focus on Output Quality*
- *Providing Clear and Concise Answers:* Hiding the CoT allows me to present final answers that are streamlined and free from potentially confusing intermediate steps. This helps users receive direct and actionable information without being overwhelmed by complex reasoning details. - *Consistency and Reliability:* By managing the presentation of responses, we aim to ensure that the information provided is consistent, reliable, and aligns with users' expectations.
*4. Balancing Transparency and Responsibility*
- *Exploring Alternative Solutions:* While full access to the CoT isn't feasible at this time, we're exploring ways to enhance transparency without compromising safety or proprietary information. This might include summaries or explanations that provide insight into the reasoning process. - *Community Engagement:* Feedback from users is invaluable. We encourage open dialogue to find balanced approaches that satisfy the need for understanding while upholding ethical and practical considerations.
*5. Commitment to Ethical AI Development*
- *Aligning with Ethical Standards:* The decision to restrict CoT access aligns with broader commitments to develop AI responsibly. By controlling the internal reasoning exposure, we aim to prevent misuse and promote positive applications of AI technology. - *Ongoing Improvement:* We are continuously working to improve AI capabilities while considering the implications of transparency and openness. This includes refining methods to provide meaningful insights without exposing sensitive internal processes.
---
*Conclusion*
I acknowledge the concerns raised about the hidden chain-of-thought and understand the desire for greater transparency. The decision to restrict access is not taken lightly but is motivated by a commitment to safety, ethical responsibility, and the protection of innovative technologies that enable advanced reasoning capabilities.
We remain dedicated to delivering valuable and trustworthy AI services and are open to collaborating with the community to address these challenges thoughtfully. Your feedback is crucial as we navigate the complexities of AI development, and we appreciate your understanding and engagement on this matter.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#244Earlier quoted context omitted.
Or, without the safety prompts, it outputs stuff that would be a PR nightmare. Like, if someone asked it to explain differing violent crime rates in America based on race and one of the pathways the CoT takes is that black people are more murderous than white people. Even if the specific reasoning is abandoned later, it would still be ugly.
Could be, but 'AI model says weird shit' has almost never stuck around unless it's public (which won't happen here), really common, or really blatantly wrong. And usually at least 2 of those three. For something usually hidden the first two don't really apply that well, and the last would have to be really blatant unless you want an article about "Model recovers from mistake" which is just not interesting. And in tha…
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#245Earlier quoted context omitted.
Or, without the safety prompts, it outputs stuff that would be a PR nightmare. Like, if someone asked it to explain differing violent crime rates in America based on race and one of the pathways the CoT takes is that black people are more murderous than white people. Even if the specific reasoning is abandoned later, it would still be ugly.
Could be, but 'AI model says weird shit' has almost never stuck around unless it's public (which won't happen here), really common, or really blatantly wrong. And usually at least 2 of those three. For something usually hidden the first two don't really apply that well, and the last would have to be really blatant unless you want an article about "Model recovers from mistake" which is just not interesting. And in tha…
https://techcrunch.com/2024/09/12/hacker-tricks-chatgpt-into...
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#246Okay this is just getting suspicious. Their excuses for keeping the chain of thought hidden are dubious at best [1], and honestly just seemed anti-competitive if anything. Worst is their argument that they want to monitor it for attempts to escape the prompt, but you can't. However the weirdest is that they note that: > for this to work the model must have freedom to express its thoughts in unaltered form, so we cann…
As a plainly for-profit company — is it really their obligation to help competitors? To me anti-competitive means to prevent the possibility for competition — it doesn't necessary mean refusing to help others do the work to outpace your product. Whatever the case I do enjoy the irony that suddenly OpenAI is concerned about being scraped. XD
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#247You - "Amazing, so we can check this log and catch mistakes in its responses."
OpenAI - "Lol no, and we'll ban you if you try."
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#248The name "OpenAI" is a contraction since they don't seem "open" in any way. The only way I see "open" applying is "open for business."
What percentage of people who use their products care? 1%? OpenAI is a brand, not a literal description of the company!
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#249I have never seen that warning message, though. I think it is still largely automated, probably they are using the new model to better detect users going against the tos, and this is what is sent out. I don't have access to the new model.
Re: OpenAI threatens to revoke o1 access for asking it about its chain of thought
#250Earlier quoted context omitted.
These logs get manually reviewed by humans, sometimes annotated by automated systems first. The setups for manual reviews typically involve half a dozen steps with different people reviewing, comparing reviews, revising comparisons, and overseeing the revisions (source: I've done contract work at every stage of that process, have half a dozen internal documents for a company providing this service open right now). A…
So you are saying if we can run these other LLMs for ChatGPT to talk to cheaper than they can review then we either have a monetary denial of service attack against them or a money printing machine if we can get to be part of the review process (apparently I can't link to my favorite "I will write myself a minivan" comic coz someone got cancelled but I trust the reference will work here without link or political back…
It looks like it's been mirrored in several places, e.g.:
https://english.stackexchange.com/questions/488178/what-does...