Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used o3 to find a remote zeroday in the Linux SMB implementation

sean.heelan.io

Post reply on HN
Print view View original

61–70 of 232 posts

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#61

Post by mrlongroots »

kweingar wrote: post #41
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

How do we benchmark these different methodologies? It all seems like vibes-based incantations. "You are an expert at finding vulnerabilities." "Please report only real vulnerabilities, not any false positives." Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it?

I think there's two aspects around LLM usage:

1. Having workflows to be able to provide meaningful context quickly. Very helpful.

2. Arbitrary incantations.

I think No. 2 may provide some random amounts of value with one model and not the other, but as a practitioner you shouldn't need to worry about it long-term. Patterns models pay attention to will change over time, especially as they become more capable. No. 1 is where the value is at.

As my example as a systems grad student, I find it a lot more useful to maintain a project wiki with LLMs in the picture. It makes coordinating with human collaborators easier too, and I just copy paste the entire wiki before beginning a conversation. Any time I have a back-and-forth with an LLM about some design discussions that I want archived, I ask them to emit markdown which I then copy paste into the wiki. It's not perfectly organized but it keeps the key bits there and makes generating papers etc. that much easier.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#62

Post by threeseed »

nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control.

Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#63

Post by threeseed »

manmal wrote: post #29
Retr0id wrote: post #7

The article cites a signal to noise ratio of ~1:50. The author is clearly deeply familiar with this codebase and is thus well-positioned to triage the signal from the noise. Automating this part will be where the real wins are, so I'll be watching this closely.

If the LLM wrote a harness and proof of concept tests for its leads, then it might increase S/N dramatically. It’s just quite expensive to do all that right now.

Except that in my experience half the time it will modify the implementation in order to make the tests pass.

And it will do this no matter how many prompts you try or you forcefully you ask it.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#64

Post by treebeard901 »

empath75 wrote: post #26

Given the value of finding zero days, pretty much every intelligence agency in the world is going to be pouring money into this if it can reliably find them with just a few hundred api calls. Especially if you can fine tune a model with lots of examples, which I don't think open ai, etc are going to do with any public api.

Yeah, the amount of engineering they have around controlling (censoring) the output, along with the terms of service, creates an incentive to still look for any possible bugs, but not allow it in the output.

Certainly for Govt agencies and others this will not be a factor. It is just for everyone else. This will cause people to use other models and agents without these restrictions.

It is safe to assume that a large number of vulnerabilities exist in important software all over the place. Now they can be found. This is going to set off arms race game theory applied to computer security and hacking. Probably sooner than expected...

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#65

Post by Henchman21 »

hackernudes wrote: post #22
Henchman21 wrote: post #17

Earlier quoted context omitted.

This is interesting to me! I regularly deploy 25G network connections, but I don’t think we’d run SMB over that. I am super curious the industry and use case if you’re willing to share!

"SMB Direct" is RDMA based and ksmbd supports it. Samba does not. Disclaimer: I have not used it but was looking it up just yesterday.

Appreciated, thank you.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#66

Post by wongarsu »

b112 wrote: post #38
roncesvalles wrote: post #25

Earlier quoted context omitted.

A lot of money is all you need~

A lot of burned coal, is what. The "don't blame the victim" trope is valid in many contexts. This one application might be "hackers are attacking vital infrastructure, so we need to fund vulnerabilities first". And hackers use AI now, likely hacked into and for free, to discover vulnerabilities. So we must use AI! Therefore, the hackers are contributing to global warming. We, dear reader, are innocent.

How much longer would OP have needed to find the same vulnerability without LLM help? Then multiply that by the energy used to produce 2000kcal/day of food as well as the electricity for running their computer.

Usually LLMs come out far ahead in those types of calculations. Compared to humans they are quite energy efficient

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#67

Post by roywiggins »

threeseed wrote: post #62
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.

Engineering principles are probably the best we've got when it comes to trying to work with a poorly understood system? That doesn't mean they'll work necessarily, but...

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#68

Post by moyix »

threeseed wrote: post #63
manmal wrote: post #29

Earlier quoted context omitted.

If the LLM wrote a harness and proof of concept tests for its leads, then it might increase S/N dramatically. It’s just quite expensive to do all that right now.

Except that in my experience half the time it will modify the implementation in order to make the tests pass. And it will do this no matter how many prompts you try or you forcefully you ask it.

With security vulnerabilities, you don't give the agent the ability to modify the potentially vulnerable software, naturally. Instead you make them do what an attacker would have to do: come up with an input that, when sent to the unmodified program, triggers the vulnerability.

How do you know if it triggered the vulnerability? Luckily for low-level memory safety issues like the ones Sean (and o3) found we have very good oracles for detecting memory safety, like KASAN, so you can basically just let the agent throw inputs at ksmbd until you see something that looks kind of like this: https://groups.google.com/g/syzkaller/c/TzmTYZVXk_Q/m/Tzh7SN...

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#69

Post by iamdanieljohns »

KTibow wrote: post #31

> With o3 you get something that feels like a human-written bug report, condensed to just present the findings, whereas with Sonnet 3.7 you get something like a stream of thought, or a work log. This is likely because the author didn't give Claude a scratchpad or space to think, essentially forcing it to mix its thoughts with its report. I'd be interested to see if using the official thinking mechanism gives it enoug…

Could you provide some links to relevant work/research on using a "scratchpad" that you liked?

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#70

Post by iknowstuff »

threeseed wrote: post #62
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.

Are you using 2023 LLMs? o3 and Gemini 2.5 Pro will gladly say no or declare uncertainty in my experience
Post reply on HN