Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: Securing critical software for the AI era

anthropic.com

Post reply on HN
Print view View original

691–700 of 921 posts

Re: Project Glasswing: Securing critical software for the AI era

#691

Post by saagarjha »

Gigachad wrote: post #243
9cb14c1ec0 wrote: post #52

Now, its very possible that this is Anthropic marketing puffery, but even if it is half true it still represents an incredible advancement in hunting vulnerabilities. It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobi…

Apple has already largely crushed hacking with memory tagging on the iPhone 17 and lockdown mode. Architectural changes, safer languages, and sandboxing have done more for security than just fixing bugs when you find them.

Memory tagging has not “crushed hacking” it’s just changed the kinds of exploits that work

Re: Project Glasswing: Securing critical software for the AI era

#692

Post by saagarjha »

lmf4lol wrote: post #618
3pt14159 wrote: post #383

Earlier quoted context omitted.

If what you are saying is true, then you would see exploit marketplaces list iOS exploits at hundreds of millions of dollars. Right now a cursory glance sets the price for zero click persistent exploit at $2m behind Android at $2.5m. Still high, and yes, higher than five years ago when it was around $1m for both, but still not "largely crushed". It is still easy to get into a phone if you are a state actor.

Hi, would you mind explaining how this works? Something is finding an exploit in Android/iOS and then he sells it for 2.5m/2m on some dark market?

It’s somewhat more complicated than this but vaguely yes

Re: Project Glasswing: Securing critical software for the AI era

#693

Post by saagarjha »

Gigachad wrote: post #339
snazz wrote: post #335

Earlier quoted context omitted.

As I understood it, Memory Integrity Enforcement adds an additional check on heap dereferences (and it doesn’t apply to every process for performance reasons). Why does it crush hacking rather than just adding another incremental roadblock like many other mitigations before?

I'm not certain there is a performance hit since there is dedicated silicon on the chip for it. I believe the checks can also be done async which reduces the performance issues. It also doesn't matter that it isn't running by default in apps since the processes you really care about are the OS ones. If someone finds an exploit in tiktok, it doesn't matter all that much unless they find a way to elevate to an exploit…

An exploit in TikTok is bad if your goal is to gain access to a TikTok account. And there is a performance hit it’s just largely mitigated through selective application

Re: Project Glasswing: Securing critical software for the AI era

#694

Post by saagarjha »

qingcharles wrote: post #208
georgemcbay wrote: post #106

Earlier quoted context omitted.

> It will be interesting to see where this goes. If its actually this good, and Apple and Google apply it to their mobile OS codebases, it could wipe out the commercial spyware industry, forcing them to rely more on hacking humans rather than hacking mobile OSes. It will likely cause some interesting tensions with government as well. eg. Apple's official stance per their 2016 customer letter is no backdoors: https://…

I assume that right now some of the biggest spenders on tokens at Anthropic are state intelligence communities who are burning up GPU cycles on Android, Chromium, WebKit code bases etc trying to find exploits.

In theory Anthropic does not permit this use.

Re: Project Glasswing: Securing critical software for the AI era

#696

Post by maccard »

KronisLV wrote: post #675
maccard wrote: post #632

Earlier quoted context omitted.

Disagree - we’re being told on one hand that we are 6 months away from AI writing all Code, and 3 months into that the tools are unusable for complex engineering [1]. Every time I mention this I’m told “but have you tried the latest model and this particular tool” - yes I have, but if I need to be on the hottest new model for it to be functional that means the last time you claimed it was solved, it wasn’t solved. [0…

> Every time I mention this I feel like there’s a bunch of factors for why it will never be the same for many folks, from the models and harnesses, to the domains and existing tests/tooling. I feel bad for the people for whom it doesn’t work, but Claude Opus has written most of my code in 2026 so far. I had to build some tools around linting entire projects and most of my tokens are probably referencing existing stuf…

> I feel like there’s a bunch of factors for why it will never be the same for many folks, from the models and harnesses, to the domains and existing tests/tooling.

If the argument is “you have to use the right model, harness, test and tooling for it to work” then it’s not replacing software engineers any time soon.

The other thing is - where are all the web apps, mobile apps, games, desktop apps, from these 100x productivity multipliers. we’re 1-2 years into these tools being widely mainstream and available and I’m not seeing applications that took years to ship before appear at 100x the rate, or games being shipped by tiny teams, or new ideas of mobile apps coming out at 100x the rate. What we do see is vibe coded slop, stability issues with massive companies (windows, AWS for example), and mass layoffs back to pre-covid levels blamed on AI but everyone knows it’s a regression to the mean after a massive over hiring when money was cheap.

It’s like the emperor has no clothes on this topic to me.

Re: Project Glasswing: Securing critical software for the AI era

#697

Post by lmf4lol »

saagarjha wrote: post #692
lmf4lol wrote: post #618

Earlier quoted context omitted.

Hi, would you mind explaining how this works? Something is finding an exploit in Android/iOS and then he sells it for 2.5m/2m on some dark market?

It’s somewhat more complicated than this but vaguely yes

interesting. and how do they find a buyer? is there a marketplace for this?

sorry for the dumb questions. I know nothing about this field :-)

Re: Project Glasswing: Securing critical software for the AI era

#698

Post by steinwinde »

tomaskafka wrote: post #680
steinwinde wrote: post #613

From a non-US perspective this must be disquieting to read: Not so much that Anthropic considers only US companies as partners. But what does Anthropic do to prevent malicious use of its software by its own government? > Anthropic has also been in ongoing discussions with US government officials about Claude Mythos Preview and its offensive and defensive cyber capabilities. As we noted above, securing critical infras…

There is very little Anthropic can do - that job is up to US citizens creating and enforcing checks and balances. You can’t ask a company legally bound by your country laws (made by your own representatives) to protect you or anyone else from said laws. That is your job. And it is other countries job to protect themselves from other countries weapons. As EU citizen I’d much rather if EU had a frontier model on par, b…

According to Dario Amodei, companies bear a lot of responsibility and must act on this. Just read https://www.darioamodei.com/essay/the-adolescence-of-technol... . But it seems that he has given up on this, even if he has a president that demands "complete and total control of Greenland" etc. What "allies" is this Anthropic statement referring to anyway?

Re: Project Glasswing: Securing critical software for the AI era

#699

Post by nicce »

stratos123 wrote: post #661
sam0x17 wrote: post #563

It's all just really genius marketing. In 6 months Mythos will be nothing special, but right now everyone is being manipulated into fearing its release, as a marketing ploy. This is the same reason AI founders perennially worry in public that they have created AGI...

I find it very unlikely that Mythos will "be nothing special". Current Opus is already "special" enough to find dozens of real bugs in Firefox and the Linux kernel, and Mythos is, it seems, a full OOM above it.

Most flagship models have found real bugs. Not sure if Opus deserves mention alone. Even open models have found many.

Re: Project Glasswing: Securing critical software for the AI era

#700

Post by saagarjha »

lmf4lol wrote: post #697
saagarjha wrote: post #692

Earlier quoted context omitted.

It’s somewhat more complicated than this but vaguely yes

interesting. and how do they find a buyer? is there a marketplace for this? sorry for the dumb questions. I know nothing about this field :-)

Yes, that’s the complicated part. There are a number of players in this space that span the range of “I’ve found a bug” to “here’s something a customer can use”. Each gets progressively more money for the value add. You can capture more for yourself if you do more of the steps. Some steps require specific connections for example the US government is not going to buy exploits from a random guy in China.
Post reply on HN