Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used o3 to find a remote zeroday in the Linux SMB implementation

sean.heelan.io

Post reply on HN
Print view View original

141–150 of 232 posts

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#141

Post by seanheelan »

meander_water wrote: post #111

I'm not sure about the assertion that this is the first vulnerability found with an LLM. For e.g. OSS-Fuzz [0] has found a few using fuzzing, and Big Sleep using an agent approach [1]. [0] https://security.googleblog.com/2024/11/leveling-up-fuzzing-... [1] https://googleprojectzero.blogspot.com/2024/10/from-naptime-...

It's certainly not the first vulnerability found with an LLM =) Perhaps I should have been more clear though.

What the post says is "Understanding the vulnerability requires reasoning about concurrent connections to the server, and how they may share various objects in specific circumstances. o3 was able to comprehend this and spot a location where a particular object that is not referenced counted is freed while still being accessible by another thread. As far as I'm aware, this is the first public discussion of a vulnerability of that nature being found by a LLM."

The point I was trying to make is that, as far as I'm aware, this is the first public documentation of an LLM figuring out that sort of bug (non-trivial amount of code, bug results from concurrent access to shared resources). To me at least, this is an interesting marker of LLM progress.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#142

Post by 0xDEAFBEAD »

ptdnxyz wrote: post #131
kristopolous wrote: post #127

Earlier quoted context omitted.

I think the proper thing to do is if someone is offended is "alright sure, whatever, there you go". Others being offended isn't something you control, responding to it is

Giving in to people who are truly unreasonably offended (by proxy, for social validation, and so on) rewards and incentivizes the behavior, and in fact I believe you have an ethical obligation not to allow people to do this. Promoting antisocial behavior is antisocial. It's worth having some grace about it though.

To be fair, gundmc's original comment was rather prosocial as these things go:

>I find this use of "Catholic" pretty offensive and distasteful.

They didn't claim their preferences were universal. Nor did they attempt any personal attack on the person they were responding to. They simply described their emotional reaction and left it at that.

If everyone's initial default was the "gundmc approach" when they were offended, the internet would be a much nicer place :-)

So yeah, as far as I'm concerned, everyone in this comment chain is simply lovely :-)

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#143

Post by seanheelan »

iandanforth wrote: post #15

The most interesting and significant bit of this article for me was that the author ran this search for vulnerabilities 100 times for each of the models. That's significantly more computation than I've historically been willing to expend on most of the problems that I try with large language models, but maybe I should let the models go brrrrr!

I realised I didn't mention it in the article, so in case you're curious it cost about $116 to run the 100k token version 100 times.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#144

Post by tinco »

Henchman21 wrote: post #17
foobar10000 wrote: post #16

Earlier quoted context omitted.

Smb over 25gbit networks - user space samba is much worse there.

This is interesting to me! I regularly deploy 25G network connections, but I don’t think we’d run SMB over that. I am super curious the industry and use case if you’re willing to share!

I ran SMB over a 20gbit network (2x 10gbit). The use case was 3D rendering (photogrammetry specifically). There were multiple render nodes, and a central service coordinating the rendering process. The projects would be on SSD's on the central SMBD server, and after they were manually configured (using Agisoft Metashape) they'd be rendered. Projects would sometimes start as tens of gigabytes worth of photos, and the artifacts (including intermediates) would balloon into the hundreds of gigabytes, we'd have dozens of these projects per week.

I researched quite extensively prior to landing on SMB, but it really seems like there isn't a better way of doing this. The environment was mixed windows/linux, but if there was a better pure linux solution I would've pushed our office staff to switch to Ubuntu.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#145

Post by naasking »

kweingar wrote: post #41
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

How do we benchmark these different methodologies? It all seems like vibes-based incantations. "You are an expert at finding vulnerabilities." "Please report only real vulnerabilities, not any false positives." Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it?

> Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it?

You just described one critical aspect of engineering: discovering a property of a system and feeding that knowledge back into a systematic, iterative process of refinement.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#146

Post by sdoering »

XorNot wrote: post #99
sdoering wrote: post #48

Earlier quoted context omitted.

So basically running a microwave for about 800 seconds, or a bit more than 13 minutes per model? Oh my god - the world is gonna end. Too bad, we panicked because of exaggerated energy consumption numbers for using an LLM when doing individual work. Yes - when a lot of people do a lot of prompting, these 0ne tenth of a second to 8 seconds of running the microwave per prompt adds up. But I strongly suggest, that we cou…

The better answer is just "I don't care". Because I definitely don't care. Energy expenditure numbers are always used in isolation, lest any one have to deal with anything real about them, and always are content to ignore the abstraction which electricity is - namely, electricity is not coal. It's electricity. Unlike say, driving my petrol powered car, the power for my computers might come from solar panels, coal, nu…

Funny, I actually care. But I try to direct my care towards the real culprits.

So about 50% of CO2 emissions in Germany come from 20 sources. The campaigns like personal footprint (invented by BP) are there to shift the blame to consumers. Away from those with the biggest impact and the most options for action.

So yes, I f**ng don’t care if a security researcher leaves his microwave equivalent running for a few minutes. But I care, campaign in the bigger sense and also orient my own consumption wherever possible towards cleaner options.

Full well knowing that even as mostly being reasonable in my consumption, I definitely belong to those 5-10% of earth's population who drive the problem. Because more than half of the population in the so called first world live according to the Paris Climate Agreement. And it’s not the upper half of.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#147

Post by stonepresto »

I know there were at least a few kernel devs who "validated" this bug, but did anyone actually build a PoC and test it? It's such a critical piece of the process yet a proof of concept is completely omitted? If you don't have a PoC, you don't know what sort of hiccups would come along the way and therefore can't determine exploitability or impact. At least the author avoided calling it an RCE without validation.

But what if there's a missing piece of the puzzle that the author and devs missed or assumed o3 covered, but in fact was out of o3's context, that would invalidate this vulnerability?

I'm not saying there is, nor am I going to take the time to do the author's work for them, rather I am saying this report is not fully validated which feels like a dangerous precedent to set with what will likely be an influential blog post in the LLM VR space moving forward.

IMO the idea of PoC || GTFO should be applied more strictly than ever before to any vulnerability report generated by a model.

The underlying perspective that o3 is much better than previous or other current models still remains, and the methodology is still interesting. I understand the desire and need to get people to focus on something by wording it a specific way, it's the clickbait problem. But dammit, do better. Build a PoC and validate your claims, don't be lazy. If you're going to write a blog post that might influence how vulnerability researchers conduct their research, you should promote validation and not theoretical assumption. The alternative is the proliferation of ignorance through false-but-seemingly-true reporting, versus deepening the community's understanding of a system through vetted and provable reports.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#148

Post by dieortin »

gizmodo59 wrote: post #50
KTibow wrote: post #31

> With o3 you get something that feels like a human-written bug report, condensed to just present the findings, whereas with Sonnet 3.7 you get something like a stream of thought, or a work log. This is likely because the author didn't give Claude a scratchpad or space to think, essentially forcing it to mix its thoughts with its report. I'd be interested to see if using the official thinking mechanism gives it enoug…

Having tried both I’d say o3 is in a league of it’s own compared to 3.7 or even Gemini 2.5 pro. The benchmarks may show not a lot of gain but that matters a lot when the task is very complex. What’s surprising is that they announced it last November and only now it’s released a month back now? (I’m guessing lots of safety took time but no idea). Can’t wait for o4!

All your content threads from the past months consist on you saying how much better OpenAI products are than the competition, so that doesn’t inspire a ton of trust.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#149

Post by jp0001 »

We followed a very similar approach at work, created a test harness and tested all the models available in AWS bedrock and the OpenAI. We created our own code challenges not available on the Internet for training with vulnerable and non-vulnerable inline snippets and more contextual multi-file bugs. We also used 100 tests per challenge - I wanted to do 1000 test per challenge but realized that these models are not even close to 2 Sigma in accuracy! Overall we found very similar results. But, we were also able to increase accuracy using additional methods - which comes as additional costs. The issue I see overall is that we found is when dealing with large codebases you'll need to put blinders on the LLMs to shorten context windows so that hallucinated results are less likely to happen. The worst thing would be to follow red herrings - perhaps in 5 years we'll have models used for more engineering specific tasks that can be rated with Six Sigma accuracy if posed with the same questions and problems sets.
Post reply on HN