Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used o3 to find a remote zeroday in the Linux SMB implementation

sean.heelan.io

Post reply on HN
Print view View original

121–130 of 232 posts

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#121

Post by skydhash »

avianlyric wrote: post #90
roywiggins wrote: post #67

Earlier quoted context omitted.

Engineering principles are probably the best we've got when it comes to trying to work with a poorly understood system? That doesn't mean they'll work necessarily, but...

> Engineering principles are probably the best we've got when it comes to trying to work with a poorly understood system? At its heart that all engineering principles exist to do. Allow us to extract useful value, and hopefully predictable outcomes from systems that are either poorly understood, or too expensive to economically characterise. Engineering is more-or-less the science of “good enough”. There’s a reason w…

From "Modern Software Engineering" by David Farley

> Software engineering is the application of an empirical, scientific approach to finding efficient, economic solutions to practical problems in software.

> The adoption of an engineering approach to software development is important for two main reasons. First, software development is always an exercise in discovery and learning, and second, if our aim is to be “efficient” and “economic,” then our ability to learn must be sustainable.

> This means that we must manage the complexity of the systems that we create in ways that maintain our ability to learn new things and adapt to them.

That is why I don't care about LLMs per se, but their usage is highly correlated to the wish of the user to not learn anything, just have some answer, even incorrect, as long as it passes the evaluation process (compilation, review, ci tests,..). If the usage is to learn, I don't have anything to say.

As for efficient and economical solutions that can be found with them,...

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#123

Post by NitpickLawyer »

epolanski wrote: post #74
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

I find your take amusing considering that's literally the only part of the post he admits to just vibing it: > In fact my entire system prompt is speculative so consider it equivalent to me saying a prayer, rather than anything resembling science or engineering

The difference between vibing and "engineering" is keeping good records, logs and prompt provenance in a methodical way? Also have a (manual) way of reviewing the results. :) (paraphrased from mythbusters)

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#124

Post by usr1106 »

gerdesj wrote: post #108

I'll have to get my facts straight but I'm pretty sure that ksmbd is ... not used much (by me). https://lwn.net/Articles/871866/ This is also nothing to do with Samba which is a well trodden path. So why not attack a codebase that is rather more heavily used and older? Why not go for vi?

Good link. After reading this it's not a surprise that this code has security vulnerabilities. But of course from knowing that there must be more to actually finding it, it's still a big leap.

4 years after the article, does any relevant distro have that implementation enabled?

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#125

Post by victor106 »

p0w3n3d wrote: post #57
kweingar wrote: post #41

Earlier quoted context omitted.

How do we benchmark these different methodologies? It all seems like vibes-based incantations. "You are an expert at finding vulnerabilities." "Please report only real vulnerabilities, not any false positives." Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it?

Listen to a video made by Karpathy about LLM, he explains why made up html tags work. It's to help the tokenizer

Could not find it. Can you please provide a link?

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#127

Post by kristopolous »

georgemcbay wrote: post #117
kristopolous wrote: post #96

Earlier quoted context omitted.

I refer to the first method as "catholic prompting" - shame, fear and guilt.

As someone from a traditional Boston Catholic family who graduated from Catholic grade and high school and who has since moved away from religion but still has a lot of family and friends who are Catholic, the fact that someone found the idea that Catholics are prone to shame, fear and guilt offensive almost makes me doubt they are Catholic. I've yet to meet one Catholic IRL who wouldn't have a laugh about that, rega…

I think the proper thing to do is if someone is offended is "alright sure, whatever, there you go".

Others being offended isn't something you control, responding to it is

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#128

Post by martinald »

Legend2440 wrote: post #54
martinald wrote: post #52

I think this is the biggest alignment problem with LLMs in the short term imo. It is getting scarily good at this. I recently found a pretty serious security vulnerability in an open source very niche server I sometimes use. This took virtually no effort using LLMs. I'm worried that there is a huge long tail of software out there which wasn't worth finding vulnerabilities in for nefarious means manually but if it was…

If attackers can automatically scan code for vulnerabilities, so can defenders. You could make it part of your commit approval process or scan every build or something.

A lot of this code isn't updated though. Think of how many abandoned wordpress plugins there are (for example). So the defenders could, but how do they get that code to fix it?

I agree after time you end up with a steady state but in the short medium term the attackers have a huge advantage.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#129

Post by ptdnxyz »

kweingar wrote: post #41
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

How do we benchmark these different methodologies? It all seems like vibes-based incantations. "You are an expert at finding vulnerabilities." "Please report only real vulnerabilities, not any false positives." Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it?

How do you benchmark different ways to interact with employees? Neural networks are somewhere between opaque and translucent to inspection, and your only interface with them is language.

Quantitative benchmarks are not necessary anyway. A method either gets results or it doesn't.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#130

Post by antirez »

Retr0id wrote: post #7

The article cites a signal to noise ratio of ~1:50. The author is clearly deeply familiar with this codebase and is thus well-positioned to triage the signal from the noise. Automating this part will be where the real wins are, so I'll be watching this closely.

I bet automatic this part will be simple. In general LLMs that have a given semantical ability "X" to do some task, have greater than X ability to check, among N replies about doing the same task, which reply is the best, especially if via binary tournament like RAInk did (it was posted here a few weeks ago). There is also the possibility to use agreement among different LLMs. I'm surprised Gemini 2.5 PRO was not used here, in my experience it is the most powerful LLM to do that kind of stuff.
Post reply on HN