All these kind of interventions create a lot of incentives for other companies around the world.
Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
351–360 of 656 posts
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#352First of all I found that fable is trained in a way that even if you were to jailbreak it, it would be completely uninterested in exploitation or finding creative solutions for explotation. However, I am unable to verify if this is related to them doing secretive prompt injection. Opus 4.8 is far more powerful in that regard. As for jailbreaking if anyone is interested: I used a fork of oh-my-pi that was modified in…
Wow. Have you written about this work anywhere?
No, but I encourage more people to validate these claims themselves if you can afford to do that. If you were token efficient you could get it down to ~$2000 worth of usage which means it's 1 week's worth of x20 usage I just didn't care since they reset limits 3 times now.
There's probably so many more better ways to jailbreak a model, for example in one of my other applications I injected a randomized image into every prompt to cause the classifier to become effectively useless. This appears to be fixed now as they run a seperated classifier for text and image input.
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#353Earlier quoted context omitted.
Depends of what the proof of concept is about. It could be just a toy example, e.g. a RCE that opens the calculator app or something much more nefarious, like returning a root shell and would still fall under the definition of PoC.
most of my tests focused on gaining kernel-mode execution from low priviledge user, opus was able to find a dozen ways to do so on a 3 year old ntoskrnl version. Fable kept trying to propose fixes and I couldn't get it to construct e2e chain, but yes it did find the same vulnerabilities opus produced better and more creative results including e2e PoC. -- edit -- the biggest issue I ran into is that it was oddly smart…
> Fable kept trying to propose fixes and I couldn't get it to construct e2e chain
What approach did you start with? Can you elaborate?
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#354I wonder if there would be an equivalent of Non proliferation treaty like Nukes? I know it sounds crazy - but if there's even 0.1% chance that some models are so good that they can be used to hack into people's bank accounts - I, as the government, would not want that model to be publicly accessible. I would also request other countries to come to the table and sign this NPT(for AI). Public will still have access to…
I'm sure many countries have learnt lessons from the NPT and would have the good sense to not agree to the same thing again. We've seen time and time again if you have nukes you can do what you want to the countries without them.
Agreed! I think it will have to be a better version of the nuke NPT. Something like every country's government keeps a Mythos like model and that's it. We stop there. Since it's a model; the most powerful country could create as many copies of the model as the countries ready to sign the treaty. But that's it. No more development after that. Somehow countries will have to come together to work this out. What's the solution otherwise? These companies won't stop at any freaking cost.
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#355Earlier quoted context omitted.
Why? Are there specific examples of WSJ reporting using unnamed sources that turned out to be false/misleading that led you to this conclusion? Unnamed sources carry some risks, sure, but it's obvious that few people would be willing to put their named to leaked info like this.
"In 2019, Altman was asked to resign from Y Combinator after partners alleged he had put personal projects, including OpenAI, ahead of his duties as president, said people familiar with the matter." A statement declared to be false by the person who made the decision, in evident increasing frustration as the falsehood purpetuated.
I am familiar with that entire episode, and while I agree that quote gives the wrong impression, that definitely falls in the realm of gray area and it's not hard for me to see how "people family with the matter" truthfully reported what they knew: Namely, Altman was asked to choose his priorities - do one or the other, but not both. Again, I think reporting that as "asked to resign" gives an incorrect impression of what happened, but literally it's not that far off.
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#356Earlier quoted context omitted.
Or in my favourite formulation: “Never assume conspiracy where mere incompetence will do”.
The people of hacker news seem to really want this to be some sort of conspiracy. It's quite interesting to see. "It's deliberate sabotage by Amazon", "It's retribution by Hegseth for embarrassing the DoW", "It's a brilliant marketing scheme by Anthropic", "It's because of the govt is considering investing in OpenAI and so they're crippling any competitors". It's never just "a very poorly formed regulatory action in…
In fairness, this tweet makes it seem very much like it's retribution from Hegseth:
https://xcancel.com/PeteHegseth/status/2065897156226015690#m
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#357putting my old man cap on and I would like to weigh in on the US admin export control on Mythos. It does remind me of the mid-1990s when suddenly asymmetric cryptographic tools such as PGP became a reality and a wide usage possible due to the growing base of internet users. Governments (US, France…) did not understand how to regulate and banned export (and asked users to apply for a licence). I do see a strong parall…
I think what's also very similar between that situation and this one is the technology is not understood at all by the people in government. They've just been told by certain people it's powerful and dangerous
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#358Earlier quoted context omitted.
I submitted separately, but this Axios report has some details that call a lot of the speculation in this thread into question, i.e. that this wasn't much of a "jailbreak" at all and that it's not Anthropic-specific - the White House intends to generally regulate Mythos-class models (whatever exactly that means): Between the lines: The government's response "seems way out of line with what's actually in the research…
That’s a terrible way to create AI regulations If they actually cared about this issue we’d have predictable laws and regulatory bodies that let companies actually plan There’s a reason royal fiat doesn’t lead to healthy economies. It’s just confusing and chaotic. It’s not clear why anyone would invest in a new model now. Then the next administration comes in and instantly, by fiat, they decide to lift the ban. The m…
> That’s a terrible way to create AI regulations
This administration doesn't do regulations, its extortion. Same as the tariffs. Just grease someone's palm and then the vague restriction is lifted.
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#359I still am struggling to understand why they informed the government about something that is known to be an issue in every LLM. There is no LLM that cannot be jailbroken, so unless this means that we have reached the absolute maximum publicly accessible US made LLMs are allowed to operate at with GPT 5.5, this is not grounded in any sane regulation attempt. Does anyone know what limits Fable 5 has overstepped in the…
>I still am struggling to understand why they informed the government about something that is known to be an issue in every LLM. There is no LLM that cannot be jailbroken, so unless this means that we have reached the absolute maximum publicly accessible US made LLMs are allowed to operate at with GPT 5.5, this is not grounded in any sane regulation attempt.
I wondering where you are getting the idea that there is an sane regulation right now?
Re: Amazon CEO's talks with U.S. officials triggered crackdown on Anthropic models
#360The only thing I can think of that would give Amazon reasons to dislike Mythos / Fable is that Anthropic really ruined their Bedrock story by imposing data retention requirements that cross a red line in regulatory compliance. It's just possible that Jassy would rather have nobody use Fable than doing it on the basis of, effectively, a direct data trust relationship with Anthropic. It is hard to plug it together into…
Feels like AWS could just tell them they’re not launching Fable then. Anthropic needs them more than the opposite, no?
Nobody who is a big Bedrock customer will ditch for another cloud provider for the privilege of having anthropic hold on to their inputs.