Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

theregister.com

Post reply on HN
Print view View original

81–90 of 382 posts

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#81

Post by NiloCK »

zipy124 wrote: post #29
dathinab wrote: post #9

Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…

What's surprising to me is that anyone who has a CS education thinking that jailbreaks are not trivial. It is as simple as normal algorithmic reduction [1], e.g can I transform a dangerous task into a not-dangerous task that the LLM will agree to solve, and then re-transform back. [1]: https://en.wikipedia.org/wiki/Reduction_(complexity)

I think that as simple as is doing a lot of work when the problem domain is all natural language (or more - all strings?) rather than some well specified DSA problem.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#82

Post by nicman23 »

SpaceL10n wrote: post #77
functionmouse wrote: post #45

Earlier quoted context omitted.

they're setting the scene for an attempt to scare the geriatric decision makers into banning free and open source ML, as it's the industry's only real competition

or are you setting the scene for well-meaning technocrats to back unrestricted AI development in hopes it will bring about utopia while dismissing the damage it could cause in the hands of adversarial groups?

fight fight fight fight

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#83

Post by pjc50 »

brookst wrote: post #41
ceejayoz wrote: post #31

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/XZ_Utils_backdoor > A subsequent investigation found that the campaign to insert the backdoor into the XZ Utils project was a culmination of over two years of effort, starting in 2021, by a user going by the name "Jia Tan". They used sock puppetry in a pressure campaign against the original maintainer of XZ Utils, eventually being given maintainer permissions on the project.

Can we retire the “seatbelts are useless because they can’t prevent every loss of life” approach to risk mitigation please? If the acceptance criteria is “would prevent every single past instance and every imaginable future instance”, then yes, no mitigation is every sufficient to address any problem in the world, so we might as well give up. But I don’t think that’s the right lens to use.

That depends on whether it's a issue of accidents or a "you have to get lucky every time, we only have to get lucky once" issue.

Re: Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers

#86

Post by NiloCK »

cogman10 wrote: post #56
dist-epoch wrote: post #28

Earlier quoted context omitted.

It is fixable. Model requires proof that you are a legitimate developer of that piece of software. Every Anthropic/OpenAI account will have a list of projects the model is allowed to work on for security issues.

Ok, and how is that determined? How does anthropic know my "kernel" project isn't a personal toy and not the Linux kernel? How does anthropic determine I'm a legitimate kernel hacker? What proof do I give them and how does it tie back to my email? What would the steps be to create a new project? Do I need to send anthropic a list of my team members each time and keep them updated as the company changes? Shall I be gi…

This is a credentials and access list oAuth style problem, and not really intractable.

For package X, I should be able to present my npm (homebrew, apt, nuget, etc) credentials with publishing rights for the package.

If package X is of sufficient public interest (user count, nature/sensitivity of user data, downstream distribution, etc), then the public interest + cryptographic credentials should permit access to best-available security auditing.

Yes, we still are trusting trust, that the owner of the package itself is not malicious, but that's not a sharp degradation from status quo.

Post reply on HN