Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

421–430 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#421

Society is about to pay a steep price for the software industry's cavalier attitude toward memory safety and control flow integrity.

Thank god, finally someone said it. I don't know the first thing about cybersecurity, but in my experience all these sandbox-break RCEs involve a step of highjacking the control flow. There were attempts to prevent various flavors of this, but imo, as long as dynamic branches exist in some form, like dlsym(), function pointers, or vtables, we will not be rid of this class of exploit entirely. The latter one is the mo…

[deleted]

Re: Project Glasswing: Securing critical software for the AI era

#422

Earlier quoted context omitted.

> PRISM was probably the state sponsored program affecting civilian life the most? No state-sponsored hacking affected Americans materially. I just don't think we were networked enough in the 2010s. The risk is higher now since we're in a more warmongering world. ( Kompromat on a power-plant technician is a risk in peace. It means blackouts in war.) The fact that Iran hasn't been able to do diddly squat in America sh…

>No state-sponsored hacking affected Americans materially. Uh, what? NotPetya was kind of a big deal.

Not in the US. I had to look it up and I work in infrastructure software

Re: Project Glasswing: Securing critical software for the AI era

#423

Earlier quoted context omitted.

I would have not believed your argument 3 months ago but I strongly suspect Anthropic actively engages in model quality throttling due to their compute constraints. Their recent deal for multi GWs worth of data center might help them correct their approach.

For what it's worth Anthropic explicity denies that. "To state it plainly: We never reduce model quality due to demand, time of day, or server load" Also can see https://marginlab.ai/trackers/claude-code/ It's very interesting to me how widespread this conception is. Maybe it's as simple as LLM productivity degrading over time within a project, as slop compounds. Or more recently since they added a 1m context window,…

It has nothing to do with the context window. Reasoning brought measured approaches grounded with actual tool calls. All of that short-circuits into a quick fix approach that is unlike Opus-4.5 or 4.6. Sonnet-4.5 used to do that. My context window is always < 200K.

Re: Project Glasswing: Securing critical software for the AI era

#424

> Mythos Preview identified a number of Linux kernel vulnerabilities that allow an adversary to write out-of-bounds (e.g., through a buffer overflow, use-after-free, or double-free vulnerability.) Many of these were remotely-triggerable. However, even after several thousand scans over the repository, because of the Linux kernel’s defense in depth measures Mythos Preview was unable to successfully exploit any of these…

It could very well be an actual reachable buffer overflow, but with KASLR, canaries, CET and other security measures, it's hard to exploit it in a way that doesn't immediately crash the system.

Re: Project Glasswing: Securing critical software for the AI era

#426
I chuckle every time says something in line of "the model is so good that we won't release it to general public, ekhm, because safety".

Because the exact same thing has been said on every single upcoming model since GPT 3.5.

At this point, this must be an inside joke to do this just because.

Re: Project Glasswing: Securing critical software for the AI era

#427

Earlier quoted context omitted.

~1/3rd of US citizens voted for him. Don’t lump us all in.

Some of you are just guilty of negligence yes.

I vote in local, state, and federal elections. I have volunteered with multiple campaigns and causes, and given substantial time/labor to the EFF. I have been harassed by Trump supporters while filming protests and other civic action. Please do not presume to know me.

I get you’re angry but you’re swinging at the wrong person.

Re: Project Glasswing: Securing critical software for the AI era

#429

> Mythos Preview identified a number of Linux kernel vulnerabilities that allow an adversary to write out-of-bounds (e.g., through a buffer overflow, use-after-free, or double-free vulnerability.) Many of these were remotely-triggerable. However, even after several thousand scans over the repository, because of the Linux kernel’s defense in depth measures Mythos Preview was unable to successfully exploit any of these…

I agree the wording is a bit alarmist, but a closer example to what they are saying is:

  bool silly_mistake = false;
  
  //... lots of lines of code

  free(x);

  //... lots of lines of code

  if (silly_mistake) { // silly_mistake shown to be false at this point in the program in all testing, so far
     free(x);
  }
A bug like above would still be something that would be patched, even if a way to exploit it has not yet been found, so I think it's fair to call out (perhaps with less sensationalism).

FWIW there's a whole boutique industry around finding these. People have built whole careers around farming bug bounties for bugs like this. I think they will be among the first set of software engineers really in trouble from AI.

Re: Project Glasswing: Securing critical software for the AI era

#430
post #159

It's messed up that Anthropic simultaneously claims to be a public benefit copro and is also picking who gets to benefit from their newly enhanced cybersecurity capabilities. It means that the economic benefit is going to the existing industry heavyweights. (And no, the Linux Foundation being in the list doesn't imply broad benefit to OSS. Linux Foundation has an agenda and will pick who benefits according to what is…

Not only companies, they're going to be taking applications from individual researchers. No doubt that it will only be granted to only established researchers, effectively locking out graduates and those early in their career. This is bad.

Why should unproven college students be given access to a cyber superweapon?
Post reply on HN