Responding to the next frontier of critical cyber capabilities
1–10 of 207 posts
Re: Responding to the next frontier of critical cyber capabilities
#2Ah yes let the FUD continue. This is a real problem but so far not nearly as severe as any of the marketing has made it out to be to the overall detriment of everyone including these companies announcing these scary capabilities. These announcements always included half hearted attempts at security layers which has now been demonstrated to benefit attackers more than defenders.
I wish I had a real solution to this beyond a dark age of the Internet where people have to finally come to terms with the general poor quality all modern software tends to normalize at.
Re: Responding to the next frontier of critical cyber capabilities
#3Ah yes let the FUD continue. This is a real problem but so far not nearly as severe as any of the marketing has made it out to be to the overall detriment of everyone including these companies announcing these scary capabilities. These announcements always included half hearted attempts at security layers which has now been demonstrated to benefit attackers more than defenders. I wish I had a real solution to this be…
[deleted]
Re: Responding to the next frontier of critical cyber capabilities
#4> We are implementing stricter security controls for higher-capability models and associated activities, including isolated testing environments
Stricter than what? You never even disclosed what happened in the first incident? This is nothing more than a setup to make it happen again and say "See? It broke out again, from an even stricter sandbox!"
Re: Responding to the next frontier of critical cyber capabilities
#5 We are sharing this because we believe it’s important to be transparent with the public and the safety and security communities about this potential shift in capabilities.
Yet another PR piece. Sigh.
Re: Responding to the next frontier of critical cyber capabilities
#6> We are implementing stricter security controls for higher-capability models and associated activities, including isolated testing environments Stricter than what? You never even disclosed what happened in the first incident? This is nothing more than a setup to make it happen again and say "See? It broke out again, from an even stricter sandbox!"
They actually did a detailed presentation at BlackHat about the HuggingFace incident, and events that led to it.
Re: Responding to the next frontier of critical cyber capabilities
#7> We are implementing stricter security controls for higher-capability models and associated activities, including isolated testing environments Stricter than what? You never even disclosed what happened in the first incident? This is nothing more than a setup to make it happen again and say "See? It broke out again, from an even stricter sandbox!"
> including isolated testing environments
Given the attack vector having possible super-human capability, I'm not sure such an environment exists. "Isolated" according to who?
Maybe seL4 could be a viable option here...
Re: Responding to the next frontier of critical cyber capabilities
#8Damage done.
The next frontier is getting all our shit out of reach of these companies/models/platforms and putting them back on prem.
Re: Responding to the next frontier of critical cyber capabilities
#9In my personal experience Sol with cyber verification is extremely capable of finding vulnerabilities, and it works even with binaries if you have some kind of IDA/Ghidra CLI access. Of course, unless the binary is protected with Denuvo/VMProtect/etc.
It sounds absurd, but in the last few weeks I've had a few cases where Sol found an RCE in self-hosted web applications in literal minutes just from reading the code (I prefer when it tries to reason statically instead of spamming runtime probes at first).
In another case it found an arbitrary file write in multiplayer in an old game by reverse engineering the binary - any other player in a match could just send you files to anywhere on your system.
I do these things for pure entertainment and curiosity, not for money from bug bounties, so if Sol can find those with a trivial prompt in tens of minutes for me, then what can focused companies/actors find in days or weeks?
Although I think most vulnerabilities are going to be closed in popular software by mid 2027, except in niche old or abandoned projects.
Re: Responding to the next frontier of critical cyber capabilities
#10Ah yes let the FUD continue. This is a real problem but so far not nearly as severe as any of the marketing has made it out to be to the overall detriment of everyone including these companies announcing these scary capabilities. These announcements always included half hearted attempts at security layers which has now been demonstrated to benefit attackers more than defenders. I wish I had a real solution to this be…
The recent Hugging Face incident did not seem like FUD to me