Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

theregister.com

Post reply on HN
Print view View original

271–280 of 382 posts

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#271

Post by bflesch »

thinkindie wrote: post #252

As an European, I really don't get where this strategy wants to take the USA to. It's pretty clear everyone is getting scared about changes like this that happen overnight, without clear reason and completely unpredictable. Business requires a stable environment, and Trump is making everything in his power to disrupt business stability. Ultimately, I see the rest of the world (especially Europe) relying less and less…

> Ultimately, I see the rest of the world (especially Europe) relying less and less on US tech. The long term damage is done.

They know it and they try to slow it down as much as possible.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#272

Post by Bender »

thinkindie wrote: post #252

As an European, I really don't get where this strategy wants to take the USA to. It's pretty clear everyone is getting scared about changes like this that happen overnight, without clear reason and completely unpredictable. Business requires a stable environment, and Trump is making everything in his power to disrupt business stability. Ultimately, I see the rest of the world (especially Europe) relying less and less…

relying less and less on US tech

Not just US vs non-US, but any hard dependency on a 3rd party is a risk to any service level agreement. In my opinion any service reaching out to a 3rd party should at most be a value added service not a core part of a business and certainly not part of any contracts. If I had to choose a phrase for businesses that build dependencies on 3rd parties it would be "fragility as a disservice" or FaaD and investors need not risk investing into a fragile model.

The same must apply to individuals. One's career must not depend on a 3rd party service or their career stability and growth are at the whims of the wind of change.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#273

Post by ceejayoz »

catigula wrote: post #243

>“The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense,” said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided. This literally means the models are too dangerous to release, and yet he and they reached the opposite conclusion. A lot of people have been saying this repeatedly for a long time.

> This literally means the models are too dangerous to release…

Unless you believe Anthropic has an irreplacable wizard or genie or fairy chained up somewhere that other providers can't replicate, someone is going to release such a thing, and that someone might be a lot more cavalier about the safety of it.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#274

Post by reheher33 »

I think this is just yet another act in theater around Anthropic IPO.

I doubt Anthropic has enough computing resources, to satisfy demand for Fable. More so with long 1M context many users take full advantage off. On other side they needed to make Fable public, in "trial version" so people could independently experiment and verify it.

I think this ban is the best outcome for Anthropic. It means they want bleed out cash and compute, gave them cheap publicity, and allowed users to try it! Actual paying customers will still get full access!

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#275

Post by [deleted] »

pixel_popping wrote: post #238

Of course it isn't about that, what we see online in the "news" is completely irrelevant with reality in most cases, it's exhausting to see people parroting what giant corps & gov are saying as if it's not extremely well crafted and plain false or deceptive most of the time. It's not even about politic left or right, both sides are acting completely dumb about it, look at Google trends, people are literally being "sw…

[deleted]

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#276

Post by MarkusQ »

ltononro wrote: post #257

This is one of the things I am most afraid of. Governments can break the progress of AI and this could be a bubble burster?

If it is a bubble, shouldn't we _want_ it to burst, and the sooner the better?

If the price for tulips had falling back to something reasonable in week two, or if the US markets had had a decent correction in '97, everyone but the wild speculators would have been better off.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#277

Post by peter422 »

Also for all the people saying Amazon's part in this couldn't be fabricated, remember that Amazon is a "friend of the administration". During Andy Jassy's tenure, they paid $75MM (wildly outbidding everybody else) for a Melania documentary that grossed ~16MM, a move publicly defended by Jeff Bezos. Any neutral observer could see this was a wild overpay, and after the fact, a terrible business move. But that is not what Amazon said or continues to say. This was just a bribe with more steps to it.

When the government comes out and says this is due to something Amazon pointed out, even if that is a complete lie, they know that Amazon won't say anything publicly about it. Amazon wants to maintain their "friend of the administration" status that they paid a lot of money to get.

It is frustrating for all of us to have to think about our government like this, but if you just look at the reality of what is happening it is very difficult to trust not only anything the government is saying, but also anything companies aligned with the government are saying.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#278

Post by nradov »

ianm218 wrote: post #71
pjc50 wrote: post #57

Earlier quoted context omitted.

> Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work Exactly. AI safety is nonsensical. You cannot define the set of "bad strings". The billion monkeys with typewriters are eventually going to be able to produce them. Any "safety" system for constraining LLM output is going to have a nonzero leak rate. But on the other hand, this is also irrelevant, unless you're irresponsible enough to c…

Isn’t your point that AI safety is impossible to prevent 100% of bad things? It is quite hard (but not impossible) to get an the frontier AI to tell you how to build a nuke or launder money now, where jailbreaks used to be trivial “ignore all previous instructions”. It seems like a worthwhile effort.

It's stupid to think that preventing LLMs from giving instructions on building nuclear weapons is at all worthwhile. Total waste of effort, done for PR purposes only. The knowledge has been published in open literature for decades. The real obstacle is access to uranium and refining equipment. No LLM can meaningfully help you get around that.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#279

Post by convolvatron »

drivebyhooting wrote: post #269
pixel_popping wrote: post #238

Of course it isn't about that, what we see online in the "news" is completely irrelevant with reality in most cases, it's exhausting to see people parroting what giant corps & gov are saying as if it's not extremely well crafted and plain false or deceptive most of the time. It's not even about politic left or right, both sides are acting completely dumb about it, look at Google trends, people are literally being "sw…

I could accept those mental gymnastics 4 months ago. But I’m afraid the quagmire in Iran has disillusioned me of any competency the administration might have. Trump and co are not playing 4D chess. It looks more and more like 1D checkers.

I think a lot of this discussion is just off base. if you assume that the administration is actually trying to govern the country, then yes it seems really keystone cops. but if your point is to use the federal government to accomplish your personal goals (i.e. taking over Venezuela), then things kind snap back into focus a little. but we argue about what the plan is, and how people are going to win elections and all sorts of charmingly naive things. by the time trump leaves he'll have built an international cabal of thieves working at all levels of many governments. he doesn't give a shit about the presidency in and of itself at all. maybe he'll have a stooge for president, maybe not, but he'll have what he wants.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#280

Post by steveBK123 »

MPSimmons wrote: post #126
HarHarVeryFunny wrote: post #69

Earlier quoted context omitted.

Exactly - it effectively is a "jail break" since it accomplishes something the model's security filter was trying to prevent, and the ridiculous simplicity of it shows just how broken that type of security is. I wonder if Dario is now regretting hyping up how dangerous the model is? How does he walk this back? Do the feds let him just put a band-aid on it?

I think it's a side effect of the Transformer architecture. The worldview where all input is equally trusted, and there's no concept of "the other", makes it hard to build effective guardrails where some input is trusted and other input is not trusted.

It seems like real robust guardrails would require some sort of "world model" or some other word to describe - AI that understands intent.

Transformers are (to grossly summarize & I don't mean this as an insult) like auto-complete on steroids. So we have cat&mouse guardrails the way swear word filters and Chinese censorship work. People come up with increasingly complex miss-spelling, euphemisms & indirections to get around the filters like saying May 35th.

I suppose one solution would be to completely vet the training data such that nothing deemed "dangerous" exists in the data, which would be a huge effort.

Even this might not work because for example you could ensure no bomb-related data is in the training data, but there's lots of chemistry data adjacent that if probed the right way would allow the LLM to synthesize the answer. Various forms of "how do I store X,Y,Z safely such that nothing bad happens" prompts probably get you on the way.

Post reply on HN