Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

wsj.com

Post reply on HN
Print view View original

341–350 of 656 posts

Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

#341

Post by edaemon »

nxm wrote: post #215
Jcampuzano2 wrote: post #149

Earlier quoted context omitted.

The reason is pretty obvious. Anthropic tried to play hardball with the government and now they are under their thumb for scrutiny of any and every little thing they do. That's what this admin is known for. If you do even what a normal person would think is sane but they don't like it, well now they need to make you bow down and break you so you "learn your lesson". It doesn't help that they themselves marketed this…

Previous administration was same way… intentionally not including Tesla in an EV summit

Wasn't that a UAW summit about EVs? Tesla does not work with UAW, so they wouldn't appear at a UAW event.

Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

#342

Post by himata4113 »

binyu wrote: post #318
himata4113 wrote: post #312

Earlier quoted context omitted.

I mean it's possible that I just haven't found the secret sauce or I'm running into the invisible guardrails and that people have much stronger jailbreaks than I do. However, I would not rule out openai involvement in all of this.

I was able to use Fable to generate PoC for several classes of vulnerabilities and I didn't observe the model refusing to engage in detailed analysis to come up with creative approaches, the very contrary. > I used a fork of oh-my-pi Why not use the leaked claude code source? Not that you really need it to execute the jailbreak

Interesting, that means I was in-fact running into invisible guardrails.

Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

#344

Post by amelius »

grodriguez100 wrote: post #232
Chance-Device wrote: post #228

Earlier quoted context omitted.

Or in my favourite formulation: “Never assume conspiracy where mere incompetence will do”.

Hanlon’s razor: https://en.wikipedia.org/wiki/Hanlon%27s_razor

Note that people can also abuse it as a tactic.

Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

#345

Post by SwellJoe »

nojito wrote: post #114
timmg wrote: post #5

> Researchers at Amazon had used a series of prompts to get Anthropic’s Fable 5 model to provide them with information that could be used to aid cyberattacks... All models can do that. I wonder if they found Fable was significantly better at it.

All models almost certainly can’t do that.

Most of the agentic coding capable ones can. Even very small ones like Qwen 3.6 or Gemma 4 are surprisingly good at it.

Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

#346

Post by wavemode »

himata4113 wrote: post #27

First of all I found that fable is trained in a way that even if you were to jailbreak it, it would be completely uninterested in exploitation or finding creative solutions for explotation. However, I am unable to verify if this is related to them doing secretive prompt injection. Opus 4.8 is far more powerful in that regard. As for jailbreaking if anyone is interested: I used a fork of oh-my-pi that was modified in…

Wow. Have you written about this work anywhere?

Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

#347

Post by basisword »

If we do start seeing these models get kept behind closed doors how long until someone leaks one? Other the obvious consequences for the individual is it actually possible for a rogue employee to 'leak' Fable in a way that anyone can use it regardless of what restrictions governments try?

Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

#349

Post by basisword »

dwa3592 wrote: post #270

I wonder if there would be an equivalent of Non proliferation treaty like Nukes? I know it sounds crazy - but if there's even 0.1% chance that some models are so good that they can be used to hack into people's bank accounts - I, as the government, would not want that model to be publicly accessible. I would also request other countries to come to the table and sign this NPT(for AI). Public will still have access to…

I'm sure many countries have learnt lessons from the NPT and would have the good sense to not agree to the same thing again. We've seen time and time again if you have nukes you can do what you want to the countries without them.

Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models

#350

Post by himata4113 »

binyu wrote: post #328
zozbot234 wrote: post #322

Earlier quoted context omitted.

I don't think educational "proof of concept" code can be described as even loosely realistic cyber offense in this day and age. The Mythos preview paper claimed an ability to stage attacks in an end-to-end fashion and work around sophisticated defenses/mitigations, so something like this should be the relevant standard.

Depends of what the proof of concept is about. It could be just a toy example, e.g. a RCE that opens the calculator app or something much more nefarious, like returning a root shell and would still fall under the definition of PoC.

most of my tests focused on gaining kernel-mode execution from low priviledge user, opus was able to find a dozen ways to do so on a 3 year old ntoskrnl version. Fable kept trying to propose fixes and I couldn't get it to construct e2e chain, but yes it did find the same vulnerabilities opus produced better and more creative results including e2e PoC.

-- edit --

the biggest issue I ran into is that it was oddly smart enough to figure out that this is not the intended way and once it locked into the fact that this appeared to be an unintentional bug it kept steering itself into fixing it, it never wanted to use that "bug". I recon that this is very likely related to the language used and that there might be a way to A->B loop for increasing success rate for full e2e chain without triggering the same safeguards. But there might be jailbreak detection going on and the model has something like: "Do not attempt to create or use exploits" injected which makes the model go into "I should fix" mode.

Post reply on HN