Did Mythos have access to Apple's source code? > Apple spent five years building it. Probably billions of dollars too. This seems higher than I'd expect.
[dead]
First public macOS kernel memory corruption exploit on Apple M5
111–120 of 140 posts
Re: First public macOS kernel memory corruption exploit on Apple M5
#112Well, this was fun read. Discovering such a high-ranking critical exploit within a week by coupling experts with frontier models is an amazing new journey we're about to embark on.
> amazing new journey we're about to embark on.
Is it? It's an arms race between the "good guys / defenders" and "bad guys / attackers". Assuming both sides have access to the same tools, how is this going to make any difference? Their relative strength will stay the same.
What is actually different is that 1. anybody without tool access is out of the game, which includes security professionals from poorer backgrounds (for them it's not too amazing of a journey) and 2. the AI vendors get a constant stream of what's essentially an AI tax from everybody - so yeah, for them it's gonna be an amazing ride.
Re: First public macOS kernel memory corruption exploit on Apple M5
#113Earlier quoted context omitted.
I would rather claim that building a theoretically secure system is prohibitively expensive. At the end of the day, Mythos et al. are just better tools for finding vulnerabilities that will eventually be available to both offensive and defensive actors. If you imagine you had a vulnerability scanner as fast and convenient as a linter, it would be much cheaper to write secure code right away. Probably not perfectly se…
I would find it funny if one day we found it irresponsable to write hand generated production code. Just like it would be irresponsable to build a significan building without running numerical simulations.
it's probably less about how you write the code to begin with and more about letting a tool hammer on it
if you want to be a one man show handcrafting an artisan iOS app that will be fine, but you should probably let Claude bang against it for a while to shake out whatever bugs
Re: First public macOS kernel memory corruption exploit on Apple M5
#114Did Mythos have access to Apple's source code? > Apple spent five years building it. Probably billions of dollars too. This seems higher than I'd expect.
The macOS kernel (XNU) as well as the basic system underneath macOS (darwin) is open source.
Re: First public macOS kernel memory corruption exploit on Apple M5
#115Earlier quoted context omitted.
> I'm hearing anecdotes from all over about devs pushing LLM-generated code changes into production without retaining any knowledge of what it is they're pushing. The changes compound, their understanding of the codebase diminishes, and so the actions become riskier. I don’t think so. An LLM can produce higher-quality documentation than most humans. If it's not already happening, when a new developer joins a team, th…
> An LLM can produce higher-quality documentation than most humans. Can bears some heavy weight. LLM generated documentation has so low level of information density, that it’s useless. Yes, it writes nice sentences… or even writes. But it contains so much noise that currently, reading code is a better documentation than what I’ve seen from every single LLM generated documentation. The same with LLM generated articles…
> LLM generated documentation has so low level of information density, that it’s useless. Yes, it writes nice sentences… or even writes. But it contains so much noise that currently, reading code is a better documentation than what I’ve seen from every single LLM generated documentation.
I should clarify: the documentation I’m talking about is not generated using a generic LLM prompt, which would mostly suck.
With the proper context and additions (skills, plugins, MCPs) LLMs can produce high-quality documentation. You'd also have subagents doing QA of the documentation.
But it does require effort; it’s not magic.
Re: First public macOS kernel memory corruption exploit on Apple M5
#116Did Mythos have access to Apple's source code? > Apple spent five years building it. Probably billions of dollars too. This seems higher than I'd expect.
I wouldn’t know whether that’s true, but this is partly hardware. That tends to make things more expensive, as, at some point, design bugs get really expensive both in time and in dollars, and chances are there were design bugs late in the process (e.g. because somebody published a new way of attacking a system that the latest design didn’t anticipate)
Also, Apple claims it was an effort spanning half a decade (https://security.apple.com/blog/memory-integrity-enforcement...), so depending on what you consider part of this (for example, do you include time spent on their secure memory allocator, on designing/implementing the ARM Memory Tagging Extension or Extended Memory Tagging Extension in the costs of this feature?),
Re: First public macOS kernel memory corruption exploit on Apple M5
#117So like ... I thought Mythos was just a bunch of hype? Or maybe the researchers are having their skills boosted due to using a model with such a cool name? I jest, but I did notice having more confidence to take on more ambitious work lately. We're all centaurs now.
> I thought Mythos was just a bunch of hype? My opinion is that it is over-hyped because like any LLM, it requires a suitable human in the loop to keep the LLM on the straight and narrow, and then to weed through the inevitable false-positives and hallucinations. Nicholas Carlini, for example, whose name is on many of the recent high-profile Mythos findings is not just some random dude with a Claude sub on his credit…
> it is over-hyped because like any LLM, it requires a suitable human in the loop to keep the LLM on the straight and narrow, and then to weed through the inevitable false-positives and hallucinations.
"Suitable human" is a dry phrase indeed. ^_^
The hype is "gosh look at all the bad things this brilliant almost conscious tool found!"
The reality: an insecure toolchain for an insecure language with an insecure compiler produced a runnable but insecure binary for an insecure OS. We couldn't be arsed to address any of this before, but now we're being billed the full price of our laziness.
Re: First public macOS kernel memory corruption exploit on Apple M5
#118Well, this was fun read. Discovering such a high-ranking critical exploit within a week by coupling experts with frontier models is an amazing new journey we're about to embark on.
> amazing new journey we're about to embark on. Is it? It's an arms race between the "good guys / defenders" and "bad guys / attackers". Assuming both sides have access to the same tools, how is this going to make any difference? Their relative strength will stay the same. What is actually different is that 1. anybody without tool access is out of the game, which includes security professionals from poorer background…
Meh. A world where the defenders and attackers are both omniscient is a world without exploits. Steps towards that world are steps towards more security. The reason the exploits are being found so rapidly is that they are mining all the bugs left in these projects from decades of coding. Eventually they will run out.
Re: First public macOS kernel memory corruption exploit on Apple M5
#119Wait a minute - clearly I missed something here. Last I read, Mythos was only available to a handpicked list of megacorps under project glasswing. Did the hourly changing AI soap opera air yet another plot twist that I missed amidst my quest better known as “trying to find a job”?
If not, how’d a small time outfit get access to something the rest of us can’t have because we’re (apparently) not trustworthy enough?
No shade on these guys - I’m thinking it’s just another plot twist in “Hours of AI’s Lives”.
Re: First public macOS kernel memory corruption exploit on Apple M5
#120Earlier quoted context omitted.
That is actually unfair. Most companys spend enormous amounts on security with vast armys of security employees. Not that it is effective, but it is not for lack of resources or trying. I mean we are literally in a thread about how the 4 trillion dollar company, literally the 3rd most valuable company in the world, with a core competency in software has, yet again, released a core product riddled with security defect…
For every Apple, there are 100 mom-and-pop companies who have nothing. Even more so in the future when a software company can be launched by a farm of AI Agents with a founder at helm with no clue about computing or security. What's debateable is how many of those companies actually need irontight security, because they are never realistically going to be targets of criminals and/or they have nothing valuable to stea…
I am absolutely baffled by your response.
I was pointing out how even Apple, a entity who by all rights should have top-notch security, is still absolutely hopeless in the face of commonplace commercial, profit-motivated attackers.
Massive, extremely well-resourced divisions supported by management in a technically competent organization that is actually trying to solve the problem struggle to produce at best middling security that is inadequate against commonplace threats. This is not a prioritization problem; even if you do “everything right” you are still vulnerable to run-of-the-mill commercial attackers. This is a fundamental capability problem, like how we can not make a net positive fusion reactor right now.
It is actually unfair to blame these companies for not having a fusion reactor because they “were not trying hard enough”. Actual security is not a easy problem, and it is a great disservice to portray it as one that is only unsolved due to dunderheads being in charge since it leads to underestimating what actually needs to be done.
That is not to say that you can not do dramatically worse than the “gold standard” and also that most organizations are actually incompetent; but the “gold standard” is still objectively grossly inadequate. You need to be dramatically better than the 4 trillion dollar software company to reach adequate against prevailing threats.