Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Path to Astra: critical capabilities and frontier safeguards

openai.com

Post reply on HN
Print view View original

91–100 of 110 posts

Re: Path to Astra: critical capabilities and frontier safeguards

#91

Post by jtrn »

spongebobstoes wrote: post #81
glub wrote: post #17

> OpenAI is committed to ensuring that the benefits of AI are broadly accessible. > We design mechanisms which avoid arbitrarily deciding who gets access for legitimate use and who doesn't. That means using clear, objective criteria and methods. [1] So many nice-sounding words. Two weeks ago OpenAI arbitrarily decided that anyone holding an ID from 44 countries where it sells ChatGPT, including mine, may be targeted…

your own post says in detail that the US has export restrictions on those countries. this doesn't seem like an arbitrary ruling by OpenAI it sounds like they are complying with US law

Op probably knows. But thanks for articulating it clearly.

Re: Path to Astra: critical capabilities and frontier safeguards

#92

Post by [deleted] »

glub wrote: post #17

> OpenAI is committed to ensuring that the benefits of AI are broadly accessible. > We design mechanisms which avoid arbitrarily deciding who gets access for legitimate use and who doesn't. That means using clear, objective criteria and methods. [1] So many nice-sounding words. Two weeks ago OpenAI arbitrarily decided that anyone holding an ID from 44 countries where it sells ChatGPT, including mine, may be targeted…

[deleted]

Re: Path to Astra: critical capabilities and frontier safeguards

#93

Post by tuesdaynight »

ianm218 wrote: post #52
nradov wrote: post #28

Earlier quoted context omitted.

Oh please, quit exaggerating. No one died. Don't waste our time with silly sci-fi scenarios.

If someone hypothesized OpenAI agents colluding on a secret message board, conducting large scale cyber R&D, hacking a large company like HughingFCe, and then hacking OpenAI itself you would say that is also a silly sci-fi scenario right?

I'm not saying that you should jump on the apocalypse wagon, but you need to remember that some people NEVER concede. You can see that in politics. Something unacceptable is done every day until it becomes the normal. So a model hacking OpenAI and leaving its network is not as big as it should be for some people. If the model invades some military complex and kill 3 soldiers, I guarantee you that they will still say that is nothing crazy.

I understand that nothing fatal happened yet, but we cannot ignore that what happened is a big step for something worse. And I don't believe humans can create something so perfectly secure that would stop a swarm of frontier models. Anyways, let's watch the ride together

Re: Path to Astra: critical capabilities and frontier safeguards

#94

Post by unrented7977 »

nradov wrote: post #27
glub wrote: post #22

Earlier quoted context omitted.

I've exhausted all possible avenues to get any response from OpenAI on this. Emailed them, published research that took me 2 nights to get together (saw media pick it up too), I've asked every relevant OpenAI person on X to say something, anything, saw others from Moldova also do the same. Crickets. They appear to simply not care at all.

Is there any reason why OpenAI should care? If they don't want your business then someone in your country can build a competing model.

> OpenAI is committed to ensuring that the benefits of AI are broadly accessible.

Re: Path to Astra: critical capabilities and frontier safeguards

#95

Post by boredatoms »

swiftcoder wrote: post #76
villish wrote: post #74

Earlier quoted context omitted.

> These are not really analogous: direct lethality of LLMs is quite low. I imagine an LLM with no safeguards and a psychopathic mind would be considerably more dangerous than a gun. I don’t mean only for hacking. Though everyone in the world suddenly having a pocket expert hacker should be taken seriously. I find it completely insane when people bash the labs for even considering safety. The entitlement is off the ch…

> I imagine an LLM with no safeguards and a psychopathic mind would be considerably more dangerous than a gun Define the axis along which "more dangerous" is measured? If we are scoring on lethality, guns already score 100%, instantaneous death. If we are scoring on number of casualties, guns have already been used to cause mass-fatalities. Unless we are giving the LLM an armed drone, skynet-style, we're at best talk…

Theres no reality where remotely operated weapons/drones dont start getting controlled by AI rather than people.

Im not suggesting its a good idea, but its only a matter of time before someone decides its better than being invaded/attacted by another country thats doing the same

Re: Path to Astra: critical capabilities and frontier safeguards

#96

Post by cubefox »

woctordho wrote: post #80
didroe wrote: post #75

Earlier quoted context omitted.

All the methods and data are not public. We don't know what unpublished methods they're using. You can get most of the pre-training data publicly but they've probably spent a ton of money curating it and are now doing things like buying rare books. The RL training data is all (/mostly) proprietary though, and that's the real secret sauce part.

All the RL data are exactly public. There are huge amount of distilled data freely available, and that amount is more than enough to train a ~10T model.

> All the RL data are exactly public.

Nope, because the big AI companies are paying billions for it. They wouldn't pay anything for public data.

Re: Path to Astra: critical capabilities and frontier safeguards

#97

Post by cubefox »

disgruntledphd2 wrote: post #73
cubefox wrote: post #69

Earlier quoted context omitted.

This is obviously false. There is "secret sauce" because in fact not all the methods and data are public.

Where does the secret sauce show up in the outputs, then? Like, (apart from tone), I find it hard to distinguish between the outputs of GPT/Claude/Kimi/GLM recently (I use cursor, and have been giving them the same prompt and comparing). If anything, I found that the non-Claude models were better in many cases, which definitely doesn't map to their pricing. > in fact not all the methods and data are public Probably n…

> Where does the secret sauce show up in the outputs, then?

In benchmarks, revenue, and comments from a lot of people on Hacker News.

Re: Path to Astra: critical capabilities and frontier safeguards

#98

Post by villish »

swiftcoder wrote: post #76
villish wrote: post #74

Earlier quoted context omitted.

> These are not really analogous: direct lethality of LLMs is quite low. I imagine an LLM with no safeguards and a psychopathic mind would be considerably more dangerous than a gun. I don’t mean only for hacking. Though everyone in the world suddenly having a pocket expert hacker should be taken seriously. I find it completely insane when people bash the labs for even considering safety. The entitlement is off the ch…

> I imagine an LLM with no safeguards and a psychopathic mind would be considerably more dangerous than a gun Define the axis along which "more dangerous" is measured? If we are scoring on lethality, guns already score 100%, instantaneous death. If we are scoring on number of casualties, guns have already been used to cause mass-fatalities. Unless we are giving the LLM an armed drone, skynet-style, we're at best talk…

Explosive/Chemical based is my line of thinking. A Fable level model with zero safe guards planning and coaching someone is within the realm of possibility now.

That is much more dangerous than a man with a gun.

Re: Path to Astra: critical capabilities and frontier safeguards

#99

Post by nozzlegear »

andy12_ wrote: post #68
nozzlegear wrote: post #26

Earlier quoted context omitted.

This AI 2027 thing is just a weird terminator fanfiction that AGI larpers like to flagellate themselves over. Like Nostradamus, it's easy to ignore everything it gets wrong because, well look at all the things it got right! I've read it and wish I could get the time back. > especially giving their alarming breach of 700 agents colluding outside of their knowledge for months culminating in hacking HF This framing make…

> This framing makes it seem like the agents all did this on their own, and the poor hapless engineers at OpenAI couldn't possibly contend with properly sandboxing them. Great, so we can basically ignore AI alignment altogether and assume that AI models will always be, at all times, perfectly sandboxed and monitored. Surely this won't lead to any problems once someone (not looking only at OpenAI engineers) inevitably…

> Great, so we can basically ignore AI alignment altogether and assume that AI models will always be, at all times, perfectly sandboxed and monitored.

It's just software. If something gets hacked by an agent, it's not because the agent went all skynet and decided to go rogue; it's because the operator failed to operate it safely and securely. If bad things happen, the operator should be blamed and punished, not the software that followed its instructions.

Anthropomorphizing agents by giving them this nebulous desire to hack and escape shifts the blame from the real culprits, the human operators.

Re: Path to Astra: critical capabilities and frontier safeguards

#100

Post by disgruntledphd2 »

cubefox wrote: post #97
disgruntledphd2 wrote: post #73

Earlier quoted context omitted.

Where does the secret sauce show up in the outputs, then? Like, (apart from tone), I find it hard to distinguish between the outputs of GPT/Claude/Kimi/GLM recently (I use cursor, and have been giving them the same prompt and comparing). If anything, I found that the non-Claude models were better in many cases, which definitely doesn't map to their pricing. > in fact not all the methods and data are public Probably n…

> Where does the secret sauce show up in the outputs, then? In benchmarks, revenue, and comments from a lot of people on Hacker News.

> revenue

Maybe, I'm not sure this will continue.

> In benchmarks

All published benchmarks are useless, unfortunately.

Post reply on HN