Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used o3 to find a remote zeroday in the Linux SMB implementation

sean.heelan.io

Post reply on HN
Print view View original

161–170 of 232 posts

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#161

Post by ngneer »

baq wrote: post #138
threeseed wrote: post #62

Earlier quoted context omitted.

It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.

You’re confusing engineering with maths. You engineer your prompting to maximize the chance the LLM does what you need - in your example, the true answer - to get you closer to solving your problem. It doesn’t matter what the LLM does internally as long as the problem is being solved correctly. (As an engineer it’s part of your job to know if the problem is being solved correctly.)

Maybe very very soft "engineering". Do you have metrics on which prompt is best? What units are you measuring this in? Can you follow a repeatable process to obtain a repeatable result?

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#162

Post by ngneer »

what-the-grump wrote: post #103
jcims wrote: post #80

Earlier quoted context omitted.

>people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. What's the alternative?

Pretending that the world is stable predictable and feeling in control while making fun of other people. Obviously.

Math and physics are pretty stable. So is computer science. Avoid voodoo.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#163

Post by ngneer »

Aurornis wrote: post #109
Retr0id wrote: post #7

The article cites a signal to noise ratio of ~1:50. The author is clearly deeply familiar with this codebase and is thus well-positioned to triage the signal from the noise. Automating this part will be where the real wins are, so I'll be watching this closely.

I’ve developed a few take-home interview problems over the years that were designed to be short, easy for an experienced developer, but challenging for anyone who didn’t know the language. All were extracted from real problems we solved on the job, reduced into something minimal. Every time a new frontier LLM is released (excluding LLMs that use input as training data) I run the interview questions through it. I’ve b…

I do the same, but entry level problems that require healthy analysis. New frontier LLMs do not manage to do so well at all.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#164

Post by antirez »

Either I'm very lucky or as I suspected Gemini 2.5 PRO can more easily identify the vulnerability. My success rate is so high that running the following prompt a few times is enough: https://gist.github.com/antirez/8b76cd9abf29f1902d46b2aed3cd...

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#165

Post by ngneer »

paulddraper wrote: post #133
eqvinox wrote: post #89

Anyone else feel like this is a best case application for LLMs? You could in theory automate the entire process, treat the LLM as a very advanced fuzzer. Run it against your target in one or more VMs. If the VM crashes or otherwise exhibits anomalous behavior, you've found something. (Most exploits like this will crash the machine initially, before you refine them.) On one hand: great application for LLMs. On the oth…

https://security.googleblog.com/2024/11/leveling-up-fuzzing-...

https://news.ycombinator.com/item?id=42017771

Meh.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#166

Post by avianlyric »

skydhash wrote: post #121
avianlyric wrote: post #90

Earlier quoted context omitted.

> Engineering principles are probably the best we've got when it comes to trying to work with a poorly understood system? At its heart that all engineering principles exist to do. Allow us to extract useful value, and hopefully predictable outcomes from systems that are either poorly understood, or too expensive to economically characterise. Engineering is more-or-less the science of “good enough”. There’s a reason w…

From "Modern Software Engineering" by David Farley > Software engineering is the application of an empirical, scientific approach to finding efficient, economic solutions to practical problems in software. > The adoption of an engineering approach to software development is important for two main reasons. First, software development is always an exercise in discovery and learning, and second, if our aim is to be “eff…

I think you’re being a little over critical of LLMs. They certainly have their issues, and most assuredly people often use them inappropriately. But it rather intellectually lazy to declare that because many people use LLMs inappropriately, that means they can’t offer real value.

I’ve personally found them extremely useful to test and experiment new ideas. Having an LLM throw together a PoC which would have taken me an hour to create, in less than 5mins, is a huge time saver. Makes it possible to iterate through many more ideas and test my understanding of systems far more efficiently than doing the same by hand.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#167

Post by mehulashah »

The scary part of this is that the bad guys are doing the same thing. They’re looking for zero day exploits, and their ability to find them just got better. More importantly, it’s now almost automated. While the arms race will always continue, I wonder if this change of speed hurts the good guys more than the bad guys. There are many of these, and they take time to fix.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#168

Post by Enginerrrd »

nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

Wrangling LLM's is remarkably like wrangling interns in my experience. Except that the LLM will surprise you by being both much smarter and much dumber.

The more you can frame the problem with your expertise, the better the results you will get.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#169

Post by seanheelan »

stonepresto wrote: post #147

I know there were at least a few kernel devs who "validated" this bug, but did anyone actually build a PoC and test it? It's such a critical piece of the process yet a proof of concept is completely omitted? If you don't have a PoC, you don't know what sort of hiccups would come along the way and therefore can't determine exploitability or impact. At least the author avoided calling it an RCE without validation. But…

Hi, author here. Yes, I built a PoC. Yes, it triggered a KASAN report/crash.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#170

Post by ramy_d »

seanheelan wrote: post #143
iandanforth wrote: post #15

The most interesting and significant bit of this article for me was that the author ran this search for vulnerabilities 100 times for each of the models. That's significantly more computation than I've historically been willing to expend on most of the problems that I try with large language models, but maybe I should let the models go brrrrr!

I realised I didn't mention it in the article, so in case you're curious it cost about $116 to run the 100k token version 100 times.

thank you, I was going to ask about this. It's not a crazy amount...
Post reply on HN