Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Path to Astra: critical capabilities and frontier safeguards

openai.com

Post reply on HN
Print view View original

51–60 of 110 posts

Re: Path to Astra: critical capabilities and frontier safeguards

#51

Post by gmueckl »

nozzlegear wrote: post #47
piyh wrote: post #41

Earlier quoted context omitted.

do you really think that a less negligent anthropic/oai/meta would really fair better against future models?

Yes, I truly, wholeheartedly believe if people who aren't negligent are at the wheel, they'll "fair better" here. I encourage you to read the xitter linked above. Of course, depending on which side of the terminator fanfiction you land on, you may disagree and feel that the software can rope-a-dope someone with the wherewithal to pay attention to what it's doing.

I just don't see how people who are truly cautious and methodical can persist in an environment that is defined by a pressure to produce "progress" as fast as possible. The competitive race tends to weed out people who slow down to make sure they do everything right.

Re: Path to Astra: critical capabilities and frontier safeguards

#52

Post by ianm218 »

nradov wrote: post #28
mentalgear wrote: post #9

I'm looking forward to an announcement of them making Alignment Top Priority - as it should be, especially giving their alarming breach of 700 agents colluding outside of their knowledge for months culminating in hacking HF (here's a good summary: https://rutgerbregman.substack.com/p/i-think-this-is-the-cra... ). The 'AI 2027' scenario of AI sneakingly claiming to be aligned to then kill off all humans in a few hours…

Oh please, quit exaggerating. No one died. Don't waste our time with silly sci-fi scenarios.

If someone hypothesized OpenAI agents colluding on a secret message board, conducting large scale cyber R&D, hacking a large company like HughingFCe, and then hacking OpenAI itself you would say that is also a silly sci-fi scenario right?

Re: Path to Astra: critical capabilities and frontier safeguards

#53

Post by Davidzheng »

nradov wrote: post #30
toshinoriyagi wrote: post #6

Earlier quoted context omitted.

I am interested in seeing how much these cybersecurity capabilities correlate to general programming. Cybersecurity definitely feels like it would be easier for an agent due to the natural explicit feedback "did I get access or not". While general programming has many less-explicit concerns (is the code readable/maintainable, robust, bug-free, performant, scalable etc).

Code readability ceases to be a concern once you eliminate human programmers.

Well probably just redefined

Re: Path to Astra: critical capabilities and frontier safeguards

#54

Post by glub »

outside1234 wrote: post #48
glub wrote: post #17

> OpenAI is committed to ensuring that the benefits of AI are broadly accessible. > We design mechanisms which avoid arbitrarily deciding who gets access for legitimate use and who doesn't. That means using clear, objective criteria and methods. [1] So many nice-sounding words. Two weeks ago OpenAI arbitrarily decided that anyone holding an ID from 44 countries where it sells ChatGPT, including mine, may be targeted…

[flagged]

I'd rather sound desperate than be someone who noticed a shadow criterion quietly widening the already wide gap in access to frontier intelligence, and did nothing about it.

Don't worry about me, I'm fortunate enough that this won't affect me. I'd suggest asking yourself why someone raising it on behalf of a whole country read to you as "pick me."

Re: Path to Astra: critical capabilities and frontier safeguards

#55

Post by CamperBob2 »

ianm218 wrote: post #52
nradov wrote: post #28

Earlier quoted context omitted.

Oh please, quit exaggerating. No one died. Don't waste our time with silly sci-fi scenarios.

If someone hypothesized OpenAI agents colluding on a secret message board, conducting large scale cyber R&D, hacking a large company like HughingFCe, and then hacking OpenAI itself you would say that is also a silly sci-fi scenario right?

They are responsible for what they hook up to the Internet, just as you and I are. Running such a test without human supervision was irresponsible, and proves no larger point than that. Frankly it was inexplicable unless they were hoping something like what happened would happen.

What OpenAI did was the equivalent of putting a cup of gasoline in the breakroom microwave, pressing 'Start', and sprinting away. Now they're pointing and waving and shouting about how dangerous gasoline is, and how no one but them should be allowed to sell it.

Re: Path to Astra: critical capabilities and frontier safeguards

#56

Post by villish »

matheusmoreira wrote: post #15
enraged_camel wrote: post #13

From the article: "We plan to make Astra available soon, but access to its most advanced cybersecurity capabilities will be more limited. Advanced cybersecurity work will initially be available to a group of testers, with access through Daybreak Blue following to expand defensive use." This, after several months of OpenAI and its boosters relentlessly criticizing Anthropic for withholding Mythos from the general publ…

I'm happy to criticize both. Thank god the chinese are working overtime to undermine US hegemony.

Should everyone have access to guns too? I ask because there seems to be a disconnect where a lot of people who live in countries with gun control don’t want AI offensive capabilities to be regulated.

I presume your country doesn’t have AI sovereignty so no matter what you won’t have access to models aligned with your beliefs.

Re: Path to Astra: critical capabilities and frontier safeguards

#58

Post by cubefox »

nradov wrote: post #35
cubefox wrote: post #33

Earlier quoted context omitted.

They realistically can't. It's almost impossible to catch up to OpenAI. Only Anthropic might do it, but this is also an US American company.

It's not unrealistic. Several Chinese companies seem to be close behind. People thought they would never catch up to the US car industry and now look what happened.

To the best of our knowledge, these Chinese companies rely on distillation of frontier models by OpenAI and Anthropic, which isn't a method available at the frontier itself.

Re: Path to Astra: critical capabilities and frontier safeguards

#59

Post by hypfer »

nozzlegear wrote: post #26
mentalgear wrote: post #9

I'm looking forward to an announcement of them making Alignment Top Priority - as it should be, especially giving their alarming breach of 700 agents colluding outside of their knowledge for months culminating in hacking HF (here's a good summary: https://rutgerbregman.substack.com/p/i-think-this-is-the-cra... ). The 'AI 2027' scenario of AI sneakingly claiming to be aligned to then kill off all humans in a few hours…

This AI 2027 thing is just a weird terminator fanfiction that AGI larpers like to flagellate themselves over. Like Nostradamus, it's easy to ignore everything it gets wrong because, well look at all the things it got right! I've read it and wish I could get the time back. > especially giving their alarming breach of 700 agents colluding outside of their knowledge for months culminating in hacking HF This framing make…

I just love how you're being downvoted, yet the guy you're replying to isn't, while saying unhinged shit like

> The 'AI 2027' scenario of AI sneakingly claiming to be aligned to then kill off all humans in a few hours and scanning their brain looks increasingly likely

Y'all need to touch grass holy shit.

__

Also, why is one guy called mentalgear and the other nozzlegear.

Is any of this real? Are the patriots behind this?

Re: Path to Astra: critical capabilities and frontier safeguards

#60

Post by matheusmoreira »

villish wrote: post #56
matheusmoreira wrote: post #15

Earlier quoted context omitted.

I'm happy to criticize both. Thank god the chinese are working overtime to undermine US hegemony.

Should everyone have access to guns too? I ask because there seems to be a disconnect where a lot of people who live in countries with gun control don’t want AI offensive capabilities to be regulated. I presume your country doesn’t have AI sovereignty so no matter what you won’t have access to models aligned with your beliefs.

> Should everyone have access to guns too?

Yes.

Post reply on HN