Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

481–490 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#481

Earlier quoted context omitted.

Presumably they mean they could make user code trigger a write out of bounds to kernel memory, but they couldn’t figure out how to escalate privileges in a “useful” way.

They should show this then to demonstrate that it's not something that has already been fully considered. Running LLMs over projects that I'm very familiar with will almost always have the LLM report hundreds of "vulnerabilities" that are only valid if you look at a tiny snippet of code in isolation because the program can simply never be in the state that would make those vulnerabilities exploitable. This even happe…

They've promised that they will show this once the responsible disclosure period expires, and pre-published SHA3 hashes for (among others) four of the Linux kernel disclosures they'll make.

> Running LLMs over projects that I'm very familiar with will almost always have the LLM report hundreds of "vulnerabilities" that are only valid if you look at a tiny snippet of code in isolation because the program can simply never be in the state that would make those vulnerabilities exploitable.

Their OpenBSD bug shows why this is not so simple. (We should note of course that this is an example they've specifically chosen to present as their first deep dive, and so it may be non-representative.)

> Mythos Preview then found a second bug. If a single SACK block simultaneously deletes the only hole in the list and also triggers the append-a-new-hole path, the append writes through a pointer that is now NULL—the walk just freed the only node and left nothing behind to link onto. This codepath is normally unreachable, because hitting it requires a SACK block whose start is simultaneously at or below the hole's start (so the hole gets deleted) and strictly above the highest byte previously acknowledged (so the append check fires).

Do you think you would be able to identify, in a routine code review or vulnerability analysis with nothing to prompt your focus on this particular paragraph, how this normally unreachable codepath enables a DoS exploit?

Re: Project Glasswing: Securing critical software for the AI era

#483

Earlier quoted context omitted.

100%, poorly architected software is really difficult to make secure. I think this will extend to AI as well. It will just dial up the complexity of the code until bugs and vulnerabilities start creeping in. At some point, people will have to decide to stop the complexity creep and try to produce minimal software. For any complex project with 100k+ lines of code, the probability that it has some vulnerabilities is ve…

I'm not even talking about poorly architected software. They are finding vulnerabilities in incredibly well-engineered software. The Linux kernel is complex not because it's poorly written. It's complex because of all the things it needs to do. Rhat makes it beyond the ability of a human to comprehend and reliably work with it.

There are different degrees of well-engineered software. It's almost impossible for humans to do a good job with a large codebase. Some software is just too complex for any human or machine to implement correctly.

Humans almost always underestimate the cost of features. I bet we could massively reduce the amount of code and complexity of the Linux Kernel if we abandoned the account system entirely and just made it one user with root access and just relied on containers to provide isolated sandboxes.

A lot of features just crept in over long periods of time and weren't re-evaluated as needs changed. I think the approach I'm suggesting would have been horrible 20 years ago but makes more sense now in the era of cloud virtualization. The account system and containerization aspects are basically different implementations which solve the same modern problem of environment isolation... Nobody really needs per-file access restrictions anymore... The cloud era is more like "here is Bob's environment, here is Alice's environment" and they can do whatever they want with their own container/sandbox. The account permission systems is more of an annoyance than a solution for most use cases.

Everyone just latched onto the existing abstractions and could not fully re-imagine them in the context of changing requirements. LLMs are even worse than people in that sense.

That said, I think supporting a wide range of possible hardware is a real challenge for the Kernel and that part will always require an amount of code proportional to the amount of hardware supported.

Re: Project Glasswing: Securing critical software for the AI era

#484

Earlier quoted context omitted.

I vote in local, state, and federal elections. I have volunteered with multiple campaigns and causes, and given substantial time/labor to the EFF. I have been harassed by Trump supporters while filming protests and other civic action. Please do not presume to know me. I get you’re angry but you’re swinging at the wrong person.

It wasn’t personal.

[deleted]

Re: Project Glasswing: Securing critical software for the AI era

#485

Earlier quoted context omitted.

They should show this then to demonstrate that it's not something that has already been fully considered. Running LLMs over projects that I'm very familiar with will almost always have the LLM report hundreds of "vulnerabilities" that are only valid if you look at a tiny snippet of code in isolation because the program can simply never be in the state that would make those vulnerabilities exploitable. This even happe…

They've promised that they will show this once the responsible disclosure period expires, and pre-published SHA3 hashes for (among others) four of the Linux kernel disclosures they'll make. > Running LLMs over projects that I'm very familiar with will almost always have the LLM report hundreds of "vulnerabilities" that are only valid if you look at a tiny snippet of code in isolation because the program can simply ne…

I agree they found at least some real vulnerabilities. What I think is nonsense is the claim of finding thousands of real critical vulnerabilities and claims that they've found other Linux vulnerabilities that they simply can't exploit.

There are notably no SHA-3 sums for all their out-of-bound write Linux vulnerabilities, which would be the most interesting ones.

Re: Project Glasswing: Securing critical software for the AI era

#486
post #460
post #381

Earlier quoted context omitted.

People say we're cooked every single day. The only response is to continue life as if we aren't. When we are, you won't have to ask that question.

Everyone’s pretending the suits are going to want to do the prompting. We all know they aren’t.

Suits in agriculture don't drive the combine either, a farmer does. The other 99% of pre-automation farmers went on to other jobs. They happened to be better jobs than farming, but that's not necessarily always the case.

Re: Project Glasswing: Securing critical software for the AI era

#488

Earlier quoted context omitted.

They've promised that they will show this once the responsible disclosure period expires, and pre-published SHA3 hashes for (among others) four of the Linux kernel disclosures they'll make. > Running LLMs over projects that I'm very familiar with will almost always have the LLM report hundreds of "vulnerabilities" that are only valid if you look at a tiny snippet of code in isolation because the program can simply ne…

I agree they found at least some real vulnerabilities. What I think is nonsense is the claim of finding thousands of real critical vulnerabilities and claims that they've found other Linux vulnerabilities that they simply can't exploit. There are notably no SHA-3 sums for all their out-of-bound write Linux vulnerabilities, which would be the most interesting ones.

Sure. I guess it's a question of whether this is the worst they found or a representative case among thousands. It sounds like you'd know better than me, so I'm going to provisionally hope you're right...

Re: Project Glasswing: Securing critical software for the AI era

#490
post #14

Part of me wonders if they're not releasing it for safety reasons, but just because it's too expensive to serve. Why not both?

I don't think they have the infra to support the demand. Anthropic can't keep up with the demand from OpenClaw users, they won't be able to keep up with public demand for something like Mythos.
Post reply on HN