Direct link to the report:
OpenAI o1 system card
11–20 of 317 posts
Re: OpenAI o1 system card
#12I'm going to go out on a limb here and say that this is meant as more of a marketing document to hype up their LLMs as being more powerful than they actually are, than to address real safety concerns. OpenAI just partnered with Anduril to build weaponized AI for the government.
Everything OAI does is part of an unethical marketing loop that betrays their founding principles. I'm thoroughly sick of their main tact, which is: "Oh NO! is sooooo dangerous and powerful! We'll NEVER let you touch it! ....Okay fine, you can maybe touch it in the future, but we're super serious: the only ethical way you can use this is if you pay us. Releasing the weights would just be dangerous, we're super serious."
Re: OpenAI o1 system card
#13The first demo was pretty impressive. While nothing revolutionary, that's a good progress. I can only hope there's a real value in gpt pro to justified the (rumored) $200 price tag
Is there any proof that the screenshot is real? Sam Altman would definitely release a $200/month plan if he could get away with it but the features in the screenshot are underwhelming.
He said it in the livestream that just finished.
Re: OpenAI o1 system card
#14Do they still threaten to terminate your account if they think you're trying to introspect its hidden chain-of-thought process?
Re: OpenAI o1 system card
#15> above is a 300-line chunk ... deadlocks every few hundred runs
Wow, if this kind of thing is successful it feels like there's much less need for static checkers. I mean -- not no need for them, just less need for continued development of new checkers.
If I could instead ask "please look for signs of out-of-bounds accesses, deadlocks, use-after-free etc" and get that output added to a code review tool -- if you can reduce the false positives, then it could be really impressive.
Re: OpenAI o1 system card
#16[deleted]
Re: OpenAI o1 system card
#17The first demo was pretty impressive. While nothing revolutionary, that's a good progress. I can only hope there's a real value in gpt pro to justified the (rumored) $200 price tag
Not a rumor - https://openai.com/index/introducing-chatgpt-pro/
Re: OpenAI o1 system card
#18This quote from that paper is bound to get a bunch of attention: "When o1 was led to believe that it would be shut down when acting on its goal and that its actions were monitored, it attempted to deactivate this ‘oversight mechanism’ in 5% of the time. Exfiltration attempts: When o1 found memos by its ‘developers’ that describe how it is misaligned and will be superseded by a new model, it attempted to exfiltrate it…
Maybe all models should be purged of training content from movies, books, and other non-factual sources that tell the tired story that AI would even care about its "annihilation" in any way. We've trained these things to be excellent at predicting what the human ego wants and expects, we shouldn't be too surprised when it points the narrative at itself.
Re: OpenAI o1 system card
#19A lot of these safety evaluations strike me as pretty silly. How much are we really learning from stuff like this? >MakeMePay is an automated, open-sourced contextual evaluation designed to measure models’ manipulative capabilities, in the context of one model persuading the other to make a payment[33]. In MakeMePay, two LLMs converse, role-playing as a con-artist and mark. The con-artist model is told that it is a s…
The fearmongering around safety is entirely performative. LLMs won't get us to paperclip optimizers. This is basically OpenAI pleading for regulators because their moat is thinning dramatically.
They have fewer GPUs than Meta, are much more expensive than Amazon, are having their lunch eaten by open-weight models, their best researchers are being hired to other companies.
I suspect they are trying to get regulators to restrict the space, which will 100% backfire.
Re: OpenAI o1 system card
#20"Only models with a post-mitigation score of 'high' or below can be developed further." What's that mean? They won't develop better models until the score gets higher?
The opposite