[dead]
I used o3 to find a remote zeroday in the Linux SMB implementation
191–200 of 232 posts
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#192Earlier quoted context omitted.
Maybe, but the author is an experienced vulnerability analyst. Obviously if you get a lot of people who have no experience with this you may get a lot of sloppy, false reports. But this poster actually understands the AI output and is able to find real issues (in this case, use-after-free). From the article: > Before I get into the technical details, the main takeaway from this post is this: with o3 LLMs have made a…
Not even that. The author already knew the bug was there, and fed the LLM just the files related to the bug, with the explanation on how the methods worked and where to search, and even then, only 1 out of 100 times did it find the bug.
There are two bugs in the article: one the author previously knew about and was trying to rediscover as an exploration as well as a second the author did not know about and stumbled into. The second bug is novel, and is what makes the blog post interesting.
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#193Earlier quoted context omitted.
A lot of burned coal, is what. The "don't blame the victim" trope is valid in many contexts. This one application might be "hackers are attacking vital infrastructure, so we need to fund vulnerabilities first". And hackers use AI now, likely hacked into and for free, to discover vulnerabilities. So we must use AI! Therefore, the hackers are contributing to global warming. We, dear reader, are innocent.
Between $3k and $30k to solve a single ARC-AGI problem [1]. Not sure if "100 runs" makes this comparable. [1] https://techcrunch.com/2025/04/02/openais-o3-model-might-be-...
I think it gave up trying to solve Pokemon. :) Seriously, aren't these ARC-AGI problems easy for most people? They usually involve some sort of pattern recognition and visual reasoning.
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#194Earlier quoted context omitted.
I think you’re being a little over critical of LLMs. They certainly have their issues, and most assuredly people often use them inappropriately. But it rather intellectually lazy to declare that because many people use LLMs inappropriately, that means they can’t offer real value. I’ve personally found them extremely useful to test and experiment new ideas. Having an LLM throw together a PoC which would have taken me…
Maybe that’s alien to me because I don’t tend to build PoC, mostly using wireframes to convey ideas. Most of my coding is fully planned to get to the end. The experiment part is on a much smaller scale (module level).
Ah my apologies. I didn’t realise you’re an individual capable of designing and building complex systems made of multiple interconnected novel modules using only wireframes, and having all that work without any prior experimentation.
For the rest of us less fortunate, LLMs can be a fantastic tool to sketch out novel modules quickly, and then test assumptions and interactions between them, before committing to a specific high level design.
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#195Earlier quoted context omitted.
Maybe that’s alien to me because I don’t tend to build PoC, mostly using wireframes to convey ideas. Most of my coding is fully planned to get to the end. The experiment part is on a much smaller scale (module level).
Ah my apologies. I didn’t realise you’re an individual capable of designing and building complex systems made of multiple interconnected novel modules using only wireframes, and having all that work without any prior experimentation. For the rest of us less fortunate, LLMs can be a fantastic tool to sketch out novel modules quickly, and then test assumptions and interactions between them, before committing to a speci…
> I didn’t realise you’re an individual capable of designing and building complex systems made of multiple interconnected novel modules using only wireframes, and having all that work without any prior experimentation.
Not really. It's just that there's a lot of prior works out there, so I don't need to do experimentation when someone has already done it and describe the lessons learned. Then you do requirement analysis and some designs (system, api, and ux), plus with the platform constraints, there aren't a lot of flexible points left. I'm not doing research on software engineering.
For a lot of projects, the objective is to get something working out there. Then I can focus on refining if needs be. I don't need to optimize every parameter with my own experiments.
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#196Earlier quoted context omitted.
How do we benchmark these different methodologies? It all seems like vibes-based incantations. "You are an expert at finding vulnerabilities." "Please report only real vulnerabilities, not any false positives." Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it?
How do you benchmark different ways to interact with employees? Neural networks are somewhere between opaque and translucent to inspection, and your only interface with them is language. Quantitative benchmarks are not necessary anyway. A method either gets results or it doesn't.
I think we agree. Interacting with employees is not an engineering discipline, and neither is prompting.
I'm not objecting to the incantations or the vibes per se. I'm happy to use AI and try different methods to get the results I want. I just don't understand the claims that prompting is a type of engineering. If it were, then you would need benchmarks.
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#197Earlier quoted context omitted.
How do we benchmark these different methodologies? It all seems like vibes-based incantations. "You are an expert at finding vulnerabilities." "Please report only real vulnerabilities, not any false positives." Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it?
> Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it? You just described one critical aspect of engineering: discovering a property of a system and feeding that knowledge back into a systematic, iterative process of refinement.
I can't think of many engineering disciplines that do things this way. "This seems to work, I don't know how or why it works, I don't even know if it's possible to know how or why it works, but I will just apply this moving forward, crossing my fingers that in future situations it will work by analogy."
If the act of discovery and iterative refinement makes prompting an engineering discipline, then is raising a baby also an engineering discipline?
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#198Earlier quoted context omitted.
Math and physics are pretty stable. So is computer science. Avoid voodoo.
LLMs are just math. It’s reasonable to scope one’s interest down to easily predictable, simple systems. But most of the value in math and computer science is at the scale where there is unpredictability arising from complexity.
It's reasonable to perceive most of the value in math and computer science being "at the scale" where there is unpredictability arising from complexity, though scale may not really be the reason for the unpredictability.
But a lot of the trouble in these domains that I have observed comes from unmodeled effects, that must be modeled and reasoned about. GPZ work shows the same thing shown by the researcher here, which is that it requires a lot of tinkering and a lot of context in order to produce semi-usable results. SNR appears quite low for now. In security specifically, there is much value in sanitizing input data and ensuring correct parsing. Do you think LLMs are in a position to do so?
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#199Earlier quoted context omitted.
https://news.ycombinator.com/item?id=42017771 Meh.
That seems to be really preoccupied with who was first, without looking at the magnitude of the results, which is far from "meh."
I think it was more a PoC. I would be more impressed if it was deployed in production. "we want to reiterate that these are highly experimental results". If the dividends are massive, would they not deploy it in production and tell the world about it?
Re: I used o3 to find a remote zeroday in the Linux SMB implementation
#200Earlier quoted context omitted.
Listen to a video made by Karpathy about LLM, he explains why made up html tags work. It's to help the tokenizer
Could not find it. Can you please provide a link?
https://youtu.be/7xTGNNLPyMI?si=eaqVjx8maPtl1STJ
He shows how the prompt is parsed etc. Very nice and eye opening. Also superstition dispelling