Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

US summons bank bosses over cyber risks from Anthropic's latest AI model

theguardian.com

Post reply on HN
Print view View original

31–40 of 101 posts

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#31

Post by __natty__ »

I wonder whether this kind of release of model could become the spark that ignites a new digital "cold war" between us, europe, india and china, in which they will try to outwit their rivals and compromise their critical infrastructure using artificial intelligence.

Also I’d like to believe that this really is such a huge step forward compared to Opus, but lately I’ve found it hard to believe when I look at the statements made by the CEOs of AI companies and their associates, who are fuelling the hype surrounding this topic even further. Of course, it is good that large companies and industries that are crucial to the country are the first to have access to this, but until the launch takes place, I will approach this with a degree of scepticism.

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#32

Post by bwfan123 »

icedchai wrote: post #28
PedroBatista wrote: post #2

The more I live the more I believe people at the top operated in some sort of cult mentality. The level of gullibleness, temporary lack of critical thinking is only matched by their sociopathy and Machiavellianism. I'm sure it's a great big model, but the level of hype and dishonesty is something out of Sam Altman's book. Of course it's because of the upcoming IPO, but that's the end game, for now it's critical to ge…

A good percentage of cybersecurity has always been theater. If their model helps to separate the wheat from the chaff, maybe it'll be an improvement.

> A good percentage of cybersecurity has always been theater

It is great to be in a "best-effort" business where there are no consequences for bad things happening. Cybersecurity is one of those businesses. Web search, feeds and ads are another.

Imagine you are selling locks to secure homes. A thief breaks the lock. The lock-maker is not held liable. In fact, they now start selling stronger locks, and lock sales actually improve with more thefts.

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#33

Post by SpicyLemonZest »

colechristensen wrote: post #27
reducesuffering wrote: post #23

Earlier quoted context omitted.

Or, you're wrong. And the smartest AI Research Scientists and the top banking officials are both correctly worried about the ramifications. That's what you'd expect if there really was an issue here. Are you aware of the deep seated bugs in critical software that were already uncovered with Mythos? Are you able to steelman the issue here at all?

Two things can be true. Historically bad security that people just got by with matched with powerful tools that aren't any better than the best people, but now can be deployed by mediocre people.

Which is exactly what Anthropic understands the situation to be. They state at the beginning of the Glasswing blogpost that Mythos is not better than the best vulnerability researchers. But it doesn't have to be to become a tremendously big deal.

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#34

Post by alephnerd »

skybrian wrote: post #7
causal wrote: post #6

Maybe it's marketing, but I think it's regrettable that Anthropic paired project Glasswing with Mythos. It really makes it seem like Mythos is the threat, rather than the fact that tons of vulnerabilities have always been ignored throughout the software world. If Glasswing has been started years ago with the goal of applying fixes to AI-found gaps, then this would just be another model to add to that effort. But doin…

A year ago the LLM's weren't good enough to find these security issues. They could have done other stuff. But then again, the big tech companies were already doing other stuff, with bug bounties, fuzzing, rewriting key libraries, and so on. This initiative probably could have started a few months sooner with Opus and similar models, though.

> A year ago the LLM's weren't good enough to find these security issues

I know of two F100s that already started using foundation models for SCA in tandem with other products back in 2024. It's noisy, but a false positive is less harmful than an undetected true positive depending on the environment.

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#36

Post by SpicyLemonZest »

icedchai wrote: post #28
PedroBatista wrote: post #2

The more I live the more I believe people at the top operated in some sort of cult mentality. The level of gullibleness, temporary lack of critical thinking is only matched by their sociopathy and Machiavellianism. I'm sure it's a great big model, but the level of hype and dishonesty is something out of Sam Altman's book. Of course it's because of the upcoming IPO, but that's the end game, for now it's critical to ge…

A good percentage of cybersecurity has always been theater. If their model helps to separate the wheat from the chaff, maybe it'll be an improvement.

I'm definitely optimistic that the long-term trajectory is positive. All important software can undergo extensive penetration testing with cutting-edge vulnerability research techniques before launch? Sounds great. The problem is what goes wrong on the pathway to there.

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#37

Post by adrian_b »

skybrian wrote: post #7
causal wrote: post #6

Maybe it's marketing, but I think it's regrettable that Anthropic paired project Glasswing with Mythos. It really makes it seem like Mythos is the threat, rather than the fact that tons of vulnerabilities have always been ignored throughout the software world. If Glasswing has been started years ago with the goal of applying fixes to AI-found gaps, then this would just be another model to add to that effort. But doin…

A year ago the LLM's weren't good enough to find these security issues. They could have done other stuff. But then again, the big tech companies were already doing other stuff, with bug bounties, fuzzing, rewriting key libraries, and so on. This initiative probably could have started a few months sooner with Opus and similar models, though.

Using multiple older open weights models can find all the security issues that have been found by Mythos.

However, no single model of those could find everything that was found by Mythos.

https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag...

Nevertheless, the distance between free models and Mythos is not so great as claimed by the Anthropic marketing, which of course is not surprising.

In general, this is expected to be also true for other applications, because no single model is equally good for everything, even the SOTA models, trying multiple models may be necessary for obtaining the best results, but with open weights models trying many of them may add negligible cost, especially if they are hosted locally.

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#38

Post by petcat »

sroussey wrote: post #3

Promoting the model as potentially dangerous might backfire with the government banning it from being released by executive order.

> the government banning it from being released by executive order.

There's no legal mechanism for the president or the government at all to do that.

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#39

Post by mieubrisse »

__natty__ wrote: post #31

I wonder whether this kind of release of model could become the spark that ignites a new digital "cold war" between us, europe, india and china, in which they will try to outwit their rivals and compromise their critical infrastructure using artificial intelligence. Also I’d like to believe that this really is such a huge step forward compared to Opus, but lately I’ve found it hard to believe when I look at the state…

This invisible cyberwar is already happening; it's just that the brains powering it is getting smarter.

Re: US summons bank bosses over cyber risks from Anthropic's latest AI model

#40

Post by tokioyoyo »

nothinkjustai wrote: post #4

Looks like the marketing worked at least somewhat lol. Such an obvious playbook by now I’m surprised some people here fell for it.

If it’s all marketing gimmick, then all companies that have collaborated to patch their bugs are collectively lying. If that’s the case, and they can get both OSS maintainers and the ones are on payrolls of Microsoft et al. to lie… hats off to them honestly, they deserve all the marketing exposure.
Post reply on HN