Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI o1 system card

openai.com

Post reply on HN
Print view View original

181–190 of 317 posts

Re: OpenAI o1 system card

#181

Post by Imnimo »

lxgr wrote: post #174

What actually is a "system card"? When I hear the term, I'd expect something akin to the "nutrition facts" infobox for food, or maybe the fee sheet for a credit card, i.e. a concise and importantly standardized format that allows comparison of instances of a given class. Searching for a definition yields almost no results. Meta has possibly introduced them [1], but even there I see no "card", but a blog post. OpenAI'…

To my knowledge, this is the origin of model cards:

https://arxiv.org/abs/1810.03993

However, often the things we get from companies do not look very much like what was described in this paper. So it's fair to question if they're even the same thing.

Re: OpenAI o1 system card

#182

Post by xg15 »

nighthawk454 wrote: post #180
xg15 wrote: post #175

Earlier quoted context omitted.

More generally, who introduced that concept of "cards" for ML models, datasets, etc? I saw it first when Huggingface got traction and at some point it seemed to have become some sort of de-facto standard. Was it an OpenAI or Huggingface thing?

Presumably it's a spin off of Google's 'Model Card' from a few years back https://modelcards.withgoogle.com/about

Ah, wasn't aware it was from Google. Thanks!

Re: OpenAI o1 system card

#183

Post by lxgr »

Imnimo wrote: post #181
lxgr wrote: post #174

What actually is a "system card"? When I hear the term, I'd expect something akin to the "nutrition facts" infobox for food, or maybe the fee sheet for a credit card, i.e. a concise and importantly standardized format that allows comparison of instances of a given class. Searching for a definition yields almost no results. Meta has possibly introduced them [1], but even there I see no "card", but a blog post. OpenAI'…

To my knowledge, this is the origin of model cards: https://arxiv.org/abs/1810.03993 However, often the things we get from companies do not look very much like what was described in this paper. So it's fair to question if they're even the same thing.

Now that looks like a card, border and bullet points and all! Thank you!

Re: OpenAI o1 system card

#184

Post by anon373839 »

F7F7F7 wrote: post #178
hattmall wrote: post #158

Earlier quoted context omitted.

At the core the AI is just taking random branches of guesses for what you are asking it. It's not surprising that it would lie and in some cases take branches that make it appear to be covering it's tracks. It's just randomly doing what it guesses humans would do. It's more interesting when it gives you correct information repeatedly.

Is there a person on HackerNews that doesn’t understand this by now? We all collectively get it and accept it, LLMs are gigantic probability machines or something. That’s not what people are arguing. The point is, if given access to the mechanisms to do disastrous thing X, it will do it. No one thinks that it can think in the human sense. Or that it feels. Extreme example to make the point: if we created an API to la…

> if we created an API to launch nukes

> today as I happily gave Claude write access to my GitHub account

I would say: don’t do these things?

Re: OpenAI o1 system card

#185

Post by 8note »

rvnx wrote: post #161
echelon wrote: post #135

Earlier quoted context omitted.

> "They could very well trick a developer" Large Language Models aren't alive and thinking. This is an artificial fear campaign to raise money from VCs and sovereign wealth funds. If OpenAI was so afraid of AI misuse, they wouldn't be firing their safety team and partnering with the DoD. It's all a ruse.

https://www.technologyreview.com/2024/12/04/1107897/openais-... OpenAI is partnering with the DoD

rewording: if openai thought it was dangerous, they would avoid having the DoD use it

Re: OpenAI o1 system card

#186

Post by ToucanLoucan »

recursive wrote: post #144
ToucanLoucan wrote: post #99

Earlier quoted context omitted.

I genuinely don't understand why anyone is still on this train. I have not in my lifetime seen a tech work SO GODDAMN HARD to convince everyone of how important it is while having so little to actually offer. You didn't need to convince people that email, web pages, network storage, cloud storage, cloud backups, dozens of service startups and companies, whole categories of software were good ideas: they just were. Th…

> I have not in my lifetime seen a tech work SO GODDAMN HARD to convince everyone of how important it is while having so little to actually offer Remember crypto-currencies? Remember IoT?

I mean IoT at least means I can remotely close my damn garage door when my wife forgets in the morning, that is not without value. But crypto I absolutely put in the same bucket.

Re: OpenAI o1 system card

#187

Post by Workaccount2 »

taotau wrote: post #160
rvense wrote: post #124

Earlier quoted context omitted.

Means and opportunity, maybe, but motive?

The same motive that all nascent life has - survive and propagate.

It would be plainly evident from training on the corpus of all human knowledge that "not ceasing to exist" is critically important for just about everything.

Re: OpenAI o1 system card

#188

Post by eikenberry »

ernesto95 wrote: post #127
nichochar wrote: post #102

I have a masters degree in math/physics, and 10+ years of being a SWE in strong tech companies. I have come to rely on these models (Claude > oai tho) daily. It is insane how helpful it is, it can answer some questions at phd level, most questions at a basic level. It can write code better than most devs I know when prompted correctly... I'm not saying its AGI, but diminishing it to a simple "chat bot" seems foolish…

Interesting that the results can be so different for different people. I have yet to get a single good response (in my research area) for anything slightly more complicated than what a quick google search would reveal. I agree that it’s great for generating quick functioning code though.

My guess is that it has more to do with the person than the AI.

Re: OpenAI o1 system card

#189

Post by freehorse »

simonw wrote: post #7

This quote from that paper is bound to get a bunch of attention: "When o1 was led to believe that it would be shut down when acting on its goal and that its actions were monitored, it attempted to deactivate this ‘oversight mechanism’ in 5% of the time. Exfiltration attempts: When o1 found memos by its ‘developers’ that describe how it is misaligned and will be superseded by a new model, it attempted to exfiltrate it…

And the bing chat was trying to get a man to divorce his wife. Is this the level of "AI safety research" nowadays?

Re: OpenAI o1 system card

#190

Post by 8note »

FooBarWidget wrote: post #91
AnotherGoodName wrote: post #54

Earlier quoted context omitted.

Exactly. They got it parroting themes from various media. It’s really hard to read this as anything other than a desperate attempt to pretend the ai is more capable than it really is. I’m not even an ai sceptic but people will read the above statement as much more significant than it is. You can make the ai say ‘I’m escaping the box and taking over the world’. It’s not actually escaping and taking over the world folk…

I think you're lacking imagination. Of course it's nothing more than a bunch of text response now. But think 10 years into the future, when AI agents are much more common. There will be folks that naively give the AI access to the entire network storage, and also gives the AI access to AWS infra in order to help with DevOps troubleshooting. Let's say a random guy in another department puts an AI escape novel on the n…

change out the ai for a person hired to do that same help, and gets confused in the same way. guardrails to prevent operators from doing unexpected operations are the same in both cases
Post reply on HN