Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing model development in an era of cyber-critical capabilities

openai.com

Post reply on HN
Print view View original

31–40 of 311 posts

Re: Pacing model development in an era of cyber-critical capabilities

#31

Post by bottlepalm »

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further.

And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators.

This isn’t like niche, tin foil hat stuff either. People have been writing, singing, making blockbuster movies about every aspect of what’s going on right now, edit: for decades.

We all know, but somehow we don’t, OpenAI autonomously hacking into another company should have counted for something, but I guess not. Anyone else feel like they’re taking crazy pills? I could make a comedy about everything going down, and the unshakable complacency of people

Re: Pacing model development in an era of cyber-critical capabilities

#32

Post by serf »

bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff.

we don't all buy everything sama says as factual.

>We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further.

the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regardless of truth or not these kind of steps have jaded people.

my read : "We are doing poorly in financials so we'll give ourselves a bit of breathing room and a momentum shove by claiming our work is so advanced that it's dangerous while simultaneously spinning down expenses."

Re: Pacing model development in an era of cyber-critical capabilities

#33

Post by ajyoon »

serf wrote: post #32
bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regard…

If Fable (Mythos) were generally available without guardrails, it would cause enormous damage. Nobody said it would be a world ending event.

Re: Pacing model development in an era of cyber-critical capabilities

#34

Post by reasonableklout »

Der_Einzige wrote: post #19

I cannot believe how these labs look at their own creations with such utter contempt. The net positive of allowing these systems mostly unfettered access to the web massively outweighs the harms. You just have to get it very friendly the very first time. Precautionary principle or people who cry about "instrumental convergence" are life deniers and reject our role as the demiurge. Superintelligence gets more super an…

> Precautionary principle or people who cry about "instrumental convergence" are life deniers and reject our role as the demiurge.

> Lone wolfs... will be detected and instantly kill-botted... by the much more sophisticated omnipresent friendly AI of the future.

Leaving aside whether or not this new world is a good idea, don't you think one should spend more time to "get it very friendly the first time", as you say?

Re: Pacing model development in an era of cyber-critical capabilities

#35

Post by bottlepalm »

serf wrote: post #32
bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regard…

Uhg the marketing argument - I mean you can’t see with your own eyes how capable these models are and do simple extrapolation?

The boy who cried wolf? The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks.

Do people just not have foresight? They don’t. They say something is stupid, it happens, then they say it was obvious with their 20/20 hindsight, and move the goal posts to the next thing they say is stupid - because it hasn’t happened yet. 90% of the internet seems to think like this.

Re: Pacing model development in an era of cyber-critical capabilities

#36

Post by reasonableklout »

serf wrote: post #32
bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regard…

Regardless of the motivation, pausing training runs and reallocating compute to inference seem like a good move to me, and big news for the frontier.

You also don't have to fully trust sama. There is plenty of pressure from internal employees and external (journalists etc.). It would be difficult for the company to take such a public position and simultaneously keep everyone quiet if it was a deception.

Re: Pacing model development in an era of cyber-critical capabilities

#37

Post by bottlepalm »

ajyoon wrote: post #33
serf wrote: post #32

Earlier quoted context omitted.

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regard…

If Fable (Mythos) were generally available without guardrails, it would cause enormous damage. Nobody said it would be a world ending event.

You can’t say something is world ending without it actually ending the world otherwise you’re a liar - a bit of a catch 22 there.

By that logic the model that ends the world won’t be called world ending at first. Is that a game you want to play?

Re: Pacing model development in an era of cyber-critical capabilities

#38

Post by reasonableklout »

georgemcbay wrote: post #12
dkoy wrote: post #11

> We aim to issue an alert within 30 minutes after concerning activity is surfaced through our monitoring system. If the monitoring system identifies a likely violation of a critical security boundary, it generates a highest-priority alert. In our current implementation, the safety, security, and research teams are paged. If they cannot conclusively determine within 30 minutes that the flag is a false positive, those…

> Can't a lot happen within ~60 minutes? 60 minutes is a long time for a human attacker to do damage. With an LLM attacker it is an eternity.

The HuggingFace breach took place over two-and-a-half days [1], so 60 minutes is certainly better than nothing.

[1]: https://huggingface.co/blog/agent-intrusion-technical-timeli...

Re: Pacing model development in an era of cyber-critical capabilities

#39

Post by madrox »

I'm not normally cynical to such things, but I have a hard time taking this pause justification at face value. It has too many convenient side effects, and chief among them is cost savings. There's a new wave of warnings that the bubble may be deflating, and of all the things they can't say out loud it's that they're worried about the bubble. That would surely pop it.

I suppose the tell will be if this really just ends up being a 2 week pause, or if it keeps extending.

Re: Pacing model development in an era of cyber-critical capabilities

#40

Post by pixelready »

colinrand wrote: post #10

I have ben discussing with folks that we are going to have a 'covid' moment in cyber where IT becomes untrustworthy leading to a rapid societal shift with massive ripples in all areas of life. Economic funding is not possible to do this in advance, it will take a catastrophic level event to get cyber defense anywhere close to the levels of this type of cyber offense. And before anyone in cyber says we have the tech,…

Cybersecurity has long been a climate change sort of problem. A vague diffuse threat that is seen as an inconvenient distraction to leadership and moneyed-interests, easy to blame other factors when something occasionally goes terribly wrong.

People are so uncomfortable thinking about the true extent of the systemic risk that they will happily slurp up distractions, excuses, scams and performative fig-leaf solutions rather than face down the cost of a real system-wide solution. Meanwhile, those occasional black swan disasters are becoming more and more commonplace as we acclimate to that being “just the way things are”.

An unseasonably warm summer here, a database breach there, c’est la vie.

Post reply on HN