Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Assessing Claude Mythos Preview's cybersecurity capabilities

red.anthropic.com

Post reply on HN
Print view View original

51–59 of 59 posts

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#51

Post by halJordan »

staticassertion wrote: post #6

I'd love to see them point at a target that's not a decades old C/C++ codebase. Of the targets, only browsers are what should be considered hardened, and their biggest lever is sandboxing, which requires a lot of chained exploits to bypass - we're seeing that LLMs are fast to discover bugs, which means they can chain more easily. But bug density in these code bases is known to be extremely high - especially the under…

I love the goal post shifting. All modern code is ai slop right? Isn't the whole point we hate ai bc it generates vulnerable slop?

Nope, not allowed to attack bsd code, it's gotta be electron-shit #9001 or we can't trust it

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#52

Post by staticassertion »

halJordan wrote: post #51
staticassertion wrote: post #6

I'd love to see them point at a target that's not a decades old C/C++ codebase. Of the targets, only browsers are what should be considered hardened, and their biggest lever is sandboxing, which requires a lot of chained exploits to bypass - we're seeing that LLMs are fast to discover bugs, which means they can chain more easily. But bug density in these code bases is known to be extremely high - especially the under…

I love the goal post shifting. All modern code is ai slop right? Isn't the whole point we hate ai bc it generates vulnerable slop? Nope, not allowed to attack bsd code, it's gotta be electron-shit #9001 or we can't trust it

I genuinely have no clue what you're talking about. What did I call ai slop?? Who said I hate ai????? No clue. Electron???? What are you talking about lol

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#53

Post by vonneumannstan »

dist-epoch wrote: post #26
vonneumannstan wrote: post #8

Earlier quoted context omitted.

>If we don't innovate, someone else will. Terrible take. You don't get to push the extinction button just because you think China will beat you to the punch. >This is the very nature of being a human being. We summit mountains, regardless of the danger or challenge. No, just no... We barely survived the Cold War, at times because of pure luck. AI is at least as dangerous as that, if not more. We have far exceeded our…

You assume there is the option of not pushing the extinction button. Nobody asked chimps if they wanted humans around. This processes are outside control.

Until recently Claude wasn't building itself. A group of people with agency were.

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#54

Post by boshalfoshal »

dist-epoch wrote: post #25
awestroke wrote: post #4

This is becoming a bit scary. I almost hope we'll reach some kind of plateau for llm intelligence soon.

The immediate plateau is the energy output of the Sun captured by the Dyson Swarm around it. Until there it's smooth sailing.

unironically true

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#55

Post by hollerith »

dist-epoch wrote: post #26
vonneumannstan wrote: post #8

Earlier quoted context omitted.

>If we don't innovate, someone else will. Terrible take. You don't get to push the extinction button just because you think China will beat you to the punch. >This is the very nature of being a human being. We summit mountains, regardless of the danger or challenge. No, just no... We barely survived the Cold War, at times because of pure luck. AI is at least as dangerous as that, if not more. We have far exceeded our…

You assume there is the option of not pushing the extinction button. Nobody asked chimps if they wanted humans around. This processes are outside control.

Humanity stopped germ-line human genetic engineering (possible since the early 1970s) and humanity can (and should) stop OpenAI, Anthropic, etc.

Datacenters that use literal gigawatts of electricity are not exactly easy to conceal from the authorities.

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#56

Post by chris_st »

SpicyLemonZest wrote: post #34
chris_st wrote: post #27

Earlier quoted context omitted.

Well, maybe not... see Simon Willison's ongoing reporting [0] on all the bug reports for `curl` people are finding with LLMs. Interesting to see them go from "DON'T GIVE US AI SLOP!" to "Wow, lots of actual bugs found, including [ed: at least one] bug found by two people!" [0]: https://simonwillison.net/search/?q=curl

curl is both very high-profile and very security-central though. A lot of people would happily pay $100 to tuck "found a curl vulnerability" under their belt. I'm not sure that's even true for, say, Notepad++, much less all the random FOSS projects with 1 maintainer and 50 stars whose names I've never thought about twice.

But it's pretty cool that LLM bug hunting is pretty cheap... the 1-person projects can do it themselves, don't have to contract out to some huge security company.

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#57

Post by xpe »

michaelbuckbee wrote: post #29
touristtam wrote: post #28

Earlier quoted context omitted.

_SHOULD_ yes sure, but realistically is that going to happen?

As doom and gloom as things are generally, I do think things have gotten better. Due to legislation and commercial pressure things like wifi routers shipping with the same default password and open settings have gotten better. Webhosts and ISPs have implemented many improvements to protecting their residential customers. I take your point, but think that it's also maybe too far.

> As doom and gloom as things are generally, I do think things have gotten better.

The question isn't "are companies making some security improvements?". That's one-sided. The question is "are companies making security improvements FAST ENOUGH to deal with the increased risks?"

Re: Assessing Claude Mythos Preview's cybersecurity capabilities

#58

Post by xpe »

shepherdjerred wrote: post #32
torginus wrote: post #30

My two cents is LLMs are way stronger in areas where the reward function is well known, such as exploiting - you break the security, you succeed. It's much harder to establish whats a usable and well architected, novel piece of software, thus in that area, progress isn't nearly as fast, while here you can just gradient descent your way to world domination, provided you have enough GPUs.

Construction is always more expensive than destruction

To be pedantic: construction with an interconnected complex set of durable goals is hard. The general rule is that optimization over a constrained space is expensive.

But standing up a house of cards is pretty cheap. Examples include: shell corporations, formulaic business plans, AI slop, surface-level conversation, color by numbers, tract housing, cravenly only appealing the base desires of people, & c. (This might be the first time I've connected the dots in this way -- and it explains my distaste for all those things.)

But "cheap" isn't necessarily insecure. Installing bollards around building entrances is relatively cheap insurance against vehicular attacks. So this is more complicated than it seems. "Fast" doesn't mean unsafe. Even "hastily created" software _could_ be (relatively) secure if it was highly constrained to provably hardened patterns. A big problem comes when attacking a cheap target builds capability for the attacker. In a way, this analogous to how viruses attack. Start with an easy target, hijack the cell machinery, multiply, repeat.

Maybe this formulation is accurate?: If you creates something beyond your ability to understand it, then get ready to get pwned. "Staying in one's lane" in this sense might be 'safe' at least narrowly speaking (unless an entire industry is operating in a state of delusion, which is arguably the case now.)

Post reply on HN