Heck of a Patch Tuesday.
Project Glasswing: Securing critical software for the AI era
171–180 of 921 posts
Re: Project Glasswing: Securing critical software for the AI era
#172It's messed up that Anthropic simultaneously claims to be a public benefit copro and is also picking who gets to benefit from their newly enhanced cybersecurity capabilities. It means that the economic benefit is going to the existing industry heavyweights. (And no, the Linux Foundation being in the list doesn't imply broad benefit to OSS. Linux Foundation has an agenda and will pick who benefits according to what is…
Not only companies, they're going to be taking applications from individual researchers. No doubt that it will only be granted to only established researchers, effectively locking out graduates and those early in their career. This is bad.
They are not unique in this. Apple and Tesla have similar programs. More nuance is warranted here. They are trying to balance the need to enable external research with the need to protect users from arbitrary 3rd parties having special capabilities that could be used maliciously
Re: Project Glasswing: Securing critical software for the AI era
#173Earlier quoted context omitted.
Just reading this, the inevitable scaremongering about biological weapons comes up. Since most of us here are devs, we understand that software engineering capabilities can be used for good or bad - mostly good, in practice. I think this should not be different for biology. I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capab…
I feel somebody better qualified should write a comprehensive review of how these models can be used in biology. In the meantime, here are my two cents: - the models help to retrieve information faster, but one must be careful with hallucinations. - they don't circumvent the need for a well-equipped lab. - in the same way, they are generally capable but until we get the robots and a more reliable interface between mo…
[deleted]
Re: Project Glasswing: Securing critical software for the AI era
#174Related ongoing threads: System Card: Claude Mythos Preview [pdf] - https://news.ycombinator.com/item?id=47679258 Assessing Claude Mythos Preview's cybersecurity capabilities - https://news.ycombinator.com/item?id=47679155 I can't tell which of the 3 current threads should be merged - they all seem significant. Anyone?
I think merging them into either this thread, or the System Card makes the most sense to me.
Re: Project Glasswing: Securing critical software for the AI era
#175Earlier quoted context omitted.
> picking who gets to benefit from their newly enhanced cybersecurity capabilities You could say this about coordinated disclosure of any widespread 0-day or new bug class, though
That's a really good point! But: - Coordinated disclosure is ethically sketchy. I know why we do it, and I'm not saying we shouldn't. But it's not great. - This isn't a single disclosure. This is a new technology that dramatically increases capability. So, even if we thought that coordinated disclosure was unambiguously good, then I think we'd still need to have a new conversation about Mythos
Totally agree, it’s an uncomfortable compromise.
Re: Project Glasswing: Securing critical software for the AI era
#176It's messed up that Anthropic simultaneously claims to be a public benefit copro and is also picking who gets to benefit from their newly enhanced cybersecurity capabilities. It means that the economic benefit is going to the existing industry heavyweights. (And no, the Linux Foundation being in the list doesn't imply broad benefit to OSS. Linux Foundation has an agenda and will pick who benefits according to what is…
In the long term, you're right, but in the short term, it's going to be a bloodbath.
That's assuming the model is actually as good as they say it is. Given the amount of AI researchers over the past 3 years claiming supernatural capability from the LLM they have built, my bayesian skepticism is through the roof.
Re: Project Glasswing: Securing critical software for the AI era
#177>We plan to launch new safeguards with an upcoming Claude Opus model, allowing us to improve and refine them with a model that does not pose the same level of risk as Mythos Preview2. This seems like the real news. Are they saying they're going to release an intentionally degraded model as the next Opus? Big opportunity for the other labs, if that's true.
> Big opportunity for the other labs, if that's true.
It sounds like this is considered military grade technology as cryptography in the 90s. The big difference is it's very expensive to create, and run those models. It's not about the algorithm. If the story rhymes it could be a big opportunity to other regions in the world.
Re: Project Glasswing: Securing critical software for the AI era
#178It's messed up that Anthropic simultaneously claims to be a public benefit copro and is also picking who gets to benefit from their newly enhanced cybersecurity capabilities. It means that the economic benefit is going to the existing industry heavyweights. (And no, the Linux Foundation being in the list doesn't imply broad benefit to OSS. Linux Foundation has an agenda and will pick who benefits according to what is…
Releasing the model to bad actors at the same time as the major OS, browser, and security companies would be one idea. But some might consider that "messed up" too, whatever you mean by that. But in terms of acting in the public benefit, it seems consistent to work with companies that can make significant impact on users' security. The stated goal of Project Glasswing is to "secure the world's most critical software," not to be affirmative action for every wannabe out there.
Re: Project Glasswing: Securing critical software for the AI era
#179Earlier quoted context omitted.
> If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry If Apple and Google actually cared about security of their users, they would remove a ton of obvious malware from their app stores. Instead, they tighten their walled garden pretending that it's for your security.
Some links for the downvoters: https://news.ycombinator.com/item?id=46911901 https://news.ycombinator.com/item?id=47457963
You're being downvoted because you posted a non sequitur, not because people don't believe you. Vulnerabilities in the OS are not the same thing as apps using the provided APIs, even if they are predatory apps which suck.
Re: Project Glasswing: Securing critical software for the AI era
#180It's messed up that Anthropic simultaneously claims to be a public benefit copro and is also picking who gets to benefit from their newly enhanced cybersecurity capabilities. It means that the economic benefit is going to the existing industry heavyweights. (And no, the Linux Foundation being in the list doesn't imply broad benefit to OSS. Linux Foundation has an agenda and will pick who benefits according to what is…
What? The economic benefit of system critical software not totally breaking in a few weeks goes to roughly everyone. In so far Apple/Google/MS/Linux Foundation economically benefit from being able to patch pressing critical software issues upfront (I am not even exactly sure what that is supposed to mean, it's not like anyone is going to use more or less Windows or Android if this happened any other way), that's a good thing for everyone and the economic benefits of that manifest for everyone.