Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used o3 to find a remote zeroday in the Linux SMB implementation

sean.heelan.io

Post reply on HN
Print view View original

181–190 of 232 posts

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#181

Post by pixl97 »

0points wrote: post #158
nindalf wrote: post #44

Earlier quoted context omitted.

The author is up front about the limitations of their prompt. They say > In fact my entire system prompt is speculative in that I haven’t ran a sufficient number of evaluations to determine if it helps or hinders, so consider it equivalent to me saying a prayer, rather than anything resembling science or engineering. Once I have ran those evaluations I’ll let you know.

Author seems to downplay their own expertise and attribute it to the LLM, while at the same time admitting he's vibe prompting the LLM and dismissing wrong results while hyping the ones that happen to work out for him. This seems more like wishful thinking and fringe stuff than CS.

Science starts at the fringe with a "that's interesting"

The interesting thing here is the LLM can come to very complex correct answers some of the time. The problem space of understanding and finding bugs is so large that this isn't just by chance, it's not like flipping a coin.

The issue for any particular user is the amount of testing required to make this into science is really massive.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#182

Post by pixl97 »

shakna wrote: post #135
jcims wrote: post #80

Earlier quoted context omitted.

>people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. What's the alternative?

Using predictable systems. If your C compiler invents a new function call for a non-existent function while generating code, that's usually a bug. If an LLM does, that's... Normal. And a non-event.

If we have to use predictable systems, how could we use humans in the first place?

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#183

Post by lyu07282 »

stonepresto wrote: post #171
lyu07282 wrote: post #153

Earlier quoted context omitted.

Are you saying you want PoCs that trigger a crash from the use-after-free or you would only be satisfied by full on RCE PoCs?

PoCs should at least trigger a crash, overwrite a register, or have some other provable effect, the point being to determine: 1) If it is actually a UAF or if there is some other mechanism missing from the context that prevents UAF. 2) The category and severity of the vulnerability. Is it even a DoS, RCE, or is the only impact causing a thread to segfault? This is all part of the standard vulnerability research proce…

Even a rudimentary exploit can be a significant time investment, it is absolutely not common practice to develop, publish or to demand such exploits from researchers to demonstrate memory corruption vulnerabilities. Everyone thinks they are an expert in infosec its so funny.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#184

Post by skydhash »

avianlyric wrote: post #166
skydhash wrote: post #121

Earlier quoted context omitted.

From "Modern Software Engineering" by David Farley > Software engineering is the application of an empirical, scientific approach to finding efficient, economic solutions to practical problems in software. > The adoption of an engineering approach to software development is important for two main reasons. First, software development is always an exercise in discovery and learning, and second, if our aim is to be “eff…

I think you’re being a little over critical of LLMs. They certainly have their issues, and most assuredly people often use them inappropriately. But it rather intellectually lazy to declare that because many people use LLMs inappropriately, that means they can’t offer real value. I’ve personally found them extremely useful to test and experiment new ideas. Having an LLM throw together a PoC which would have taken me…

Maybe that’s alien to me because I don’t tend to build PoC, mostly using wireframes to convey ideas. Most of my coding is fully planned to get to the end. The experiment part is on a much smaller scale (module level).

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#185

Post by wyldfire »

seanheelan wrote: post #143
iandanforth wrote: post #15

The most interesting and significant bit of this article for me was that the author ran this search for vulnerabilities 100 times for each of the models. That's significantly more computation than I've historically been willing to expend on most of the problems that I try with large language models, but maybe I should let the models go brrrrr!

I realised I didn't mention it in the article, so in case you're curious it cost about $116 to run the 100k token version 100 times.

How many years/generations behind o3 are the freely available / local models?

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#186

Post by yencabulator »

JFingleton wrote: post #60
iandanforth wrote: post #15

The most interesting and significant bit of this article for me was that the author ran this search for vulnerabilities 100 times for each of the models. That's significantly more computation than I've historically been willing to expend on most of the problems that I try with large language models, but maybe I should let the models go brrrrr!

Zero days can go for $$$, or you can go down the bug bounty route and also get $$. The cost of the LLM would be a drop in the bucket. When the cost of inference gets near zero, I have no idea what the world of cyber security will look like, but it's going to be a very different space from today.

Except in this case the LLM was pointed at a known-to-exist vulnerability. $116 per handler per vulnerability type, unknown how many vulnerabilities exist.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#187

Post by paulddraper »

ngneer wrote: post #165
paulddraper wrote: post #133

Earlier quoted context omitted.

https://security.googleblog.com/2024/11/leveling-up-fuzzing-...

https://news.ycombinator.com/item?id=42017771 Meh.

That seems to be really preoccupied with who was first, without looking at the magnitude of the results, which is far from "meh."

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#188

Post by brookst »

ngneer wrote: post #162
what-the-grump wrote: post #103

Earlier quoted context omitted.

Pretending that the world is stable predictable and feeling in control while making fun of other people. Obviously.

Math and physics are pretty stable. So is computer science. Avoid voodoo.

LLMs are just math.

It’s reasonable to scope one’s interest down to easily predictable, simple systems.

But most of the value in math and computer science is at the scale where there is unpredictability arising from complexity.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#189

Post by nialv7 »

Retr0id wrote: post #7

The article cites a signal to noise ratio of ~1:50. The author is clearly deeply familiar with this codebase and is thus well-positioned to triage the signal from the noise. Automating this part will be where the real wins are, so I'll be watching this closely.

maybe we ask the AI to come up with an exploit, run it and see if it works? then you can RL on this.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#190

Post by eqvinox »

paulddraper wrote: post #133
eqvinox wrote: post #89

Anyone else feel like this is a best case application for LLMs? You could in theory automate the entire process, treat the LLM as a very advanced fuzzer. Run it against your target in one or more VMs. If the VM crashes or otherwise exhibits anomalous behavior, you've found something. (Most exploits like this will crash the machine initially, before you refine them.) On one hand: great application for LLMs. On the oth…

https://security.googleblog.com/2024/11/leveling-up-fuzzing-...

I mean, yes, they're doing it, but my question was really whether people share my belief that it's a particularly well-fitting application ;)

(Also yeah feels like the "FIRST!!1!eleven" thing metastasized from comment sections into C-level executives…)

Post reply on HN