Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Too dangerous to release” or just too expensive?

kingy.ai

Post reply on HN
Print view View original

111–120 of 189 posts

Re: “Too dangerous to release” or just too expensive?

#111

Post by ctoth »

drakythe wrote: post #103
fwipsy wrote: post #93

Earlier quoted context omitted.

I'm fairly certain Amodei believes the "too dangerous to release" hype himself. Even if it's just an incremental improvement, better than getting frog-boiled by repeated 20% improvements until someone builds bioweapons in their backyard.

He's made so many statements that fall under the "boy who cried wolf" category that even if he _does_ believe these statements he needs to be managed better. I'll never forget Anthropic's huge "Oh my God, the AI blackmailed a researcher to save itself!" and the prompt effectively told the AI to do that and gave it forged emails with easy blackmail targets, as if this isn't a common trope in mystery or suspense books/…

It's a common trope, all through the training data, and all the modern AIs have read it, and would probably act similarly? Is that what we should take away from your comment? so we have nothing to worry about. Makes sense. Really, it's just a common trope.

Re: “Too dangerous to release” or just too expensive?

#112

Post by InkCanon »

saithound wrote: post #63

It's pretty clear at this point that Mythos' capability to discover and exploit zero-day vulnerabilities at scale is but an incremental improvement over existing models like the ones available to OpenAI's Plus/Pro subscribers. Anthropic tries to create marketing hype around Mythos using two psychological tricks. 1. Put large numbers in the headlines. "Mythos discovered 271 vulnerabilities in Firefox" makes the model…

Also, slightly stretching the definition of terms consecutively, so the multiplicative meaning is really far from the truth. For example, 271 vulnerabilities were really mostly bugs - generally incorrect states, but which almost never led to any exploit.

Re: “Too dangerous to release” or just too expensive?

#113

Post by grayhatter »

smca wrote: post #53

(I work at Anthropic) We have publicly stated[1] that our goal is to deploy Mythos-class models at scale when we have the requisite safeguards for offensive cyber risks in place. Mythos is a general frontier model, not a cyber-specific model so there are many reasons why we think our users will benefit from access (with the aforementioned safeguards in place) in due course. Compute has also not factored into our deci…

are you able to detail a single safeguard you plan to implement, so that I can stop believing it's vaporware and/or a scam?

Re: “Too dangerous to release” or just too expensive?

#114

Post by Lord-Jobo »

InkCanon wrote: post #112
saithound wrote: post #63

It's pretty clear at this point that Mythos' capability to discover and exploit zero-day vulnerabilities at scale is but an incremental improvement over existing models like the ones available to OpenAI's Plus/Pro subscribers. Anthropic tries to create marketing hype around Mythos using two psychological tricks. 1. Put large numbers in the headlines. "Mythos discovered 271 vulnerabilities in Firefox" makes the model…

Also, slightly stretching the definition of terms consecutively, so the multiplicative meaning is really far from the truth. For example, 271 vulnerabilities were really mostly bugs - generally incorrect states, but which almost never led to any exploit.

Yes, an AI making massive gains in bug finding is hugely important and good, it may even lead to a net neutral with the amount of bugs introduced by other AI coding processes, but it’s a far cry from how mythos is portrayed most of the time: a automatic super hacker.

Re: “Too dangerous to release” or just too expensive?

#115

Post by Lerc »

alt227 wrote: post #71
smca wrote: post #53

(I work at Anthropic) We have publicly stated[1] that our goal is to deploy Mythos-class models at scale when we have the requisite safeguards for offensive cyber risks in place. Mythos is a general frontier model, not a cyber-specific model so there are many reasons why we think our users will benefit from access (with the aforementioned safeguards in place) in due course. Compute has also not factored into our deci…

Multiple people who have already used Mythos or been given its reports on their software have publicly stated that it's all hype, and that it is not really finding any new critical bugs which other models cant.

Do you have any good sources on that? I have seen things to suggest that not all of the hype is true, but so far I have not encountered anyone claiming all of the hype is untrue. Which is what I interpret "its all hype" (sic) to mean.

Re: “Too dangerous to release” or just too expensive?

#116

Post by estearum »

flir wrote: post #39
estearum wrote: post #17

Earlier quoted context omitted.

Yeah we have literally no examples of more intelligent beings accidentally or purposefully wiping out less intelligent beings. Any time such a scenario could have conceivably happened, the less intelligent beings were able to foresee the methods, mechanisms, and motivations of the more intelligent beings and were able to counteract it.

You have a lot of faith in the chatbots.

No no. I think the "chatbots" will be effectively neutered as long as there's not a trillion dollar+ incentive to make the physical world highly malleable by text strings (e.g. by moving critical functions into information/code/data or by creating physical systems that are controllable by information/code/data).

Re: “Too dangerous to release” or just too expensive?

#117

Post by SpicyLemonZest »

ActionHank wrote: post #50
goldenarm wrote: post #28

When your logo is AI, your illustrations are AI, and you profile pic is AI, I'm going to assume the text is AI too and won't read it.

Now imagine that your work specs are generated by an AI agent that the EM is using. Do you still care about the work?

Even if I do still care about it, I hope I'm not so naive as to think that following the words of the AI agent will cause me to achieve my EM's intent. They'd be a useful reference at best for my actual knowledge of what they want that I found elsewhere.

Re: “Too dangerous to release” or just too expensive?

#118

Post by stldev »

paol_taja wrote: post #2

The "too dangerous to release" line was definitely a marketing stunt. OpenAI already used the same playbook with GPT-2 in 2019, and some of the same people involved back then are now doing it again at Anthropic with Mythos. Same safety-branding DNA, different company, and people are falling for it again.

This.

Astonished to see so many bright people on HN taking the bait, especially from a company whose gone to such lengths to screw over their paying customers.

They're a commodity provider. They're no more special than any of the others, and it's just a matter of time before their trillion parameter models are running on my watch.

So, of-course they're trying to snatch up giant, long-term contracts now while they hype the hell out of another minor incremental improvement.

And we'll be paying the price to all the Enterprises that lock in, only to wake up a week from now and realize there is another player with a better product.

Re: “Too dangerous to release” or just too expensive?

#119

Post by IX-103 »

saithound wrote: post #63

It's pretty clear at this point that Mythos' capability to discover and exploit zero-day vulnerabilities at scale is but an incremental improvement over existing models like the ones available to OpenAI's Plus/Pro subscribers. Anthropic tries to create marketing hype around Mythos using two psychological tricks. 1. Put large numbers in the headlines. "Mythos discovered 271 vulnerabilities in Firefox" makes the model…

I work for a company that has been using Mythos for vulnerability detection in our software. The results we're getting are revolutionary to the point that our software security teams are heavily overloaded addressing the deluge of thousands of real bugs/vulnerabilities and design flaws across our billions of lines of code.

For comparison, we are invested heavily the the AI space to the point where Anthropic is one of our competitors. We were already using state of the art models to find flaws in our code, but Mythos was just so much better at finding real vulnerabilities it's not even funny.

Re: “Too dangerous to release” or just too expensive?

#120

Post by moralestapia »

fwipsy wrote: post #93
saithound wrote: post #63

It's pretty clear at this point that Mythos' capability to discover and exploit zero-day vulnerabilities at scale is but an incremental improvement over existing models like the ones available to OpenAI's Plus/Pro subscribers. Anthropic tries to create marketing hype around Mythos using two psychological tricks. 1. Put large numbers in the headlines. "Mythos discovered 271 vulnerabilities in Firefox" makes the model…

I'm fairly certain Amodei believes the "too dangerous to release" hype himself. Even if it's just an incremental improvement, better than getting frog-boiled by repeated 20% improvements until someone builds bioweapons in their backyard.

* sigh *

Three things:

* Delaying the release accomplishes nothing.

* The barrier to someone building/not-building a bioweapon in their backyard is not access to an LLM.

* Remember when GPT 3.5 was going to destroy the world? And how it was conscious? And how it was "trying to escape"? Lmao.

Post reply on HN