281–290 of 382 posts
a) In order to make us safe, the LLM should help us find (and fix) the vulnerabilities in our own code.
b) In order for us to be safe, the LLM should not find vulnerabilities in other people's code.
I don't think this is resolvable in a way where both (a) and (b) win.
Earlier quoted context omitted.
Exactly - it effectively is a "jail break" since it accomplishes something the model's security filter was trying to prevent, and the ridiculous simplicity of it shows just how broken that type of security is. I wonder if Dario is now regretting hyping up how dangerous the model is? How does he walk this back? Do the feds let him just put a band-aid on it?
I also have a 100% success rate jail breaking them by breaking the work down into small pieces and stripping all security related language. Smaller tasks, test engineering and normal programming language. Fable found a few bugs in my harness for me before they pulled it. I was testing it vs ChatGPT, Gemini, and Opus. It was doing well at bug hunting.
Earlier quoted context omitted.
The idea that checking for a US ID could possibly stop actual foreign bad actors from using it is also farcical. Millions of stolen identity documents can be bought on the dark web for relatively cheap. North Koreans have been hiring real American citizens for years to infiltrate tons of US tech companies as employees. And ya, it's pretty easy to hide your intent once you have access.
I think your really anchored on anyone successfully breaking restrictions means any restriction is impossible. So your starting from the position that if it is possible for any actor in the world to get past a restriction, then the whole restriction is a farce. KYC for example does stop most money laundering and financial crime. The most resourced actors like governments/ cartels often find ways around and it is a ga…
Clearly, there's no such thing as a perfect exclusion rule at any of these scales, but the false-negative to false-positive ratio seems like it will be way higher if Anthropic starts trying to verify IDs.
Lol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixabl…
It’s not even a jail break, it’s literally what anyone wants from a coding assistant. Is the coding assistant supposed to see vulnerabilities and intentionally leave them be? Maybe add them randomly just to double plus good its inability to see any security issues? This isn’t about security holes or risks, it’s about retribution and picking the winners and losers, and probably a large amount of self dealing as the fa…
Earlier quoted context omitted.
What makes it a risky bet?
Because it would really increase the interest for Chinese/EU models and would even create real incentives to build models outside of the US.
I suspect that this is true for any nation with sufficient AI capabilities.
Earlier quoted context omitted.
I also have a 100% success rate jail breaking them by breaking the work down into small pieces and stripping all security related language. Smaller tasks, test engineering and normal programming language. Fable found a few bugs in my harness for me before they pulled it. I was testing it vs ChatGPT, Gemini, and Opus. It was doing well at bug hunting.
This is the same way you get people to do bad stuff as well. Make the task small enough so that the moral curvature of the topology is flat and even though they know it is a not-good part of a larger bad part they just shrug. Look at all the wonderful people we know who are working at Amazon and Meta? Corporatism has already jailbroken society.
Of course it isn't about that, what we see online in the "news" is completely irrelevant with reality in most cases, it's exhausting to see people parroting what giant corps & gov are saying as if it's not extremely well crafted and plain false or deceptive most of the time. It's not even about politic left or right, both sides are acting completely dumb about it, look at Google trends, people are literally being "sw…
It's difficult to see how this motivates AI companies to relocate to the US, since US companies are the ones subject to bans.
Earlier quoted context omitted.
What makes it a risky bet?
The dual risks of either a) accidentally pushing a foreign competitor into the lead and losing dominant status, or b) pushing the underlying companies hard enough that they decide to relocate. a) is specifically the risk that the export controls push companies in other countries to prefer non-US models due to the lowered risk of getting cut off from a model. The increase in revenue for non-US AI providers combined wi…
But then the foreign competitor would stop the proliferation of their model and we would just go back and forth - American companies could "release" their model and after time gain the advantage back using the same tactics that the foreign competitor used.
> b) pushing the underlying companies hard enough that they decide to relocate.
This sounds like a reasonable risk to identify, but I would just say that it's not super clear-cut where you would relocate to.
Of course it isn't about that, what we see online in the "news" is completely irrelevant with reality in most cases, it's exhausting to see people parroting what giant corps & gov are saying as if it's not extremely well crafted and plain false or deceptive most of the time. It's not even about politic left or right, both sides are acting completely dumb about it, look at Google trends, people are literally being "sw…
> it makes sense so the US remain a superpower by forcing tech businesses and research to move/re-incorporate to the US so practically anything "new" will always be US Made. It's difficult to see how this motivates AI companies to relocate to the US, since US companies are the ones subject to bans.