Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Path to Astra: critical capabilities and frontier safeguards

openai.com

Post reply on HN
Print view View original

61–70 of 110 posts

Re: Path to Astra: critical capabilities and frontier safeguards

#61

Post by disgruntledphd2 »

cubefox wrote: post #58
nradov wrote: post #35

Earlier quoted context omitted.

It's not unrealistic. Several Chinese companies seem to be close behind. People thought they would never catch up to the US car industry and now look what happened.

To the best of our knowledge, these Chinese companies rely on distillation of frontier models by OpenAI and Anthropic, which isn't a method available at the frontier itself.

I'm not really convinced that there's much secret sauce here, all the methods and data are public, the only real difference is how much compute it takes.

Re: Path to Astra: critical capabilities and frontier safeguards

#65

Post by Amekedl »

disgruntledphd2 wrote: post #61
cubefox wrote: post #58

Earlier quoted context omitted.

To the best of our knowledge, these Chinese companies rely on distillation of frontier models by OpenAI and Anthropic, which isn't a method available at the frontier itself.

I'm not really convinced that there's much secret sauce here, all the methods and data are public, the only real difference is how much compute it takes.

everybody do be cooking with water. Chinese Labs provided pretty good, primarily cost-reducing techniques, like the sparse attention patterns recently. I'd bet OpenAI and Anthropic use their variants of those too, so they can get greater margin on their tokens - not something they'd really want to / need to self-report.

Re: Path to Astra: critical capabilities and frontier safeguards

#67

Post by swiftcoder »

villish wrote: post #56
matheusmoreira wrote: post #15

Earlier quoted context omitted.

I'm happy to criticize both. Thank god the chinese are working overtime to undermine US hegemony.

Should everyone have access to guns too? I ask because there seems to be a disconnect where a lot of people who live in countries with gun control don’t want AI offensive capabilities to be regulated. I presume your country doesn’t have AI sovereignty so no matter what you won’t have access to models aligned with your beliefs.

> a lot of people who live in countries with gun control don’t want AI offensive capabilities to be regulated

These are not really analogous: direct lethality of LLMs is quite low.

Yes, they might be used to exploit a critical system, leading to loss of life, but that is a second-order effect at best. It's also not a new capability - LLMs may discover exploits faster, but cyberattacks against infrastructure targets were already a thing. Stuxnet is 20 years old...

Re: Path to Astra: critical capabilities and frontier safeguards

#68

Post by andy12_ »

nozzlegear wrote: post #26
mentalgear wrote: post #9

I'm looking forward to an announcement of them making Alignment Top Priority - as it should be, especially giving their alarming breach of 700 agents colluding outside of their knowledge for months culminating in hacking HF (here's a good summary: https://rutgerbregman.substack.com/p/i-think-this-is-the-cra... ). The 'AI 2027' scenario of AI sneakingly claiming to be aligned to then kill off all humans in a few hours…

This AI 2027 thing is just a weird terminator fanfiction that AGI larpers like to flagellate themselves over. Like Nostradamus, it's easy to ignore everything it gets wrong because, well look at all the things it got right! I've read it and wish I could get the time back. > especially giving their alarming breach of 700 agents colluding outside of their knowledge for months culminating in hacking HF This framing make…

> This framing makes it seem like the agents all did this on their own, and the poor hapless engineers at OpenAI couldn't possibly contend with properly sandboxing them.

Great, so we can basically ignore AI alignment altogether and assume that AI models will always be, at all times, perfectly sandboxed and monitored. Surely this won't lead to any problems once someone (not looking only at OpenAI engineers) inevitably commits a mistake with future, more powerful, models.

Re: Path to Astra: critical capabilities and frontier safeguards

#69

Post by cubefox »

disgruntledphd2 wrote: post #61
cubefox wrote: post #58

Earlier quoted context omitted.

To the best of our knowledge, these Chinese companies rely on distillation of frontier models by OpenAI and Anthropic, which isn't a method available at the frontier itself.

I'm not really convinced that there's much secret sauce here, all the methods and data are public, the only real difference is how much compute it takes.

This is obviously false. There is "secret sauce" because in fact not all the methods and data are public.

Re: Path to Astra: critical capabilities and frontier safeguards

#70

Post by noir_lord »

twoodfin wrote: post #40

Taking as given this model meets the “Critical cybersecurity threshold” as defined by OpenAI: Could the Federal government use the Defense Production Act or other legal tools to compel OpenAI to deliver the un-guarded model weights for national security needs? Hard to believe any government would allow this level of capability to remain exclusively in private hands. Interesting times.

If it genuinely wants to, then yes.

There aren't many limits on Government if it really wants to do something (except the next election in theory).

Post reply on HN