Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used o3 to find a remote zeroday in the Linux SMB implementation

sean.heelan.io

Post reply on HN
Print view View original

131–140 of 232 posts

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#131

Post by ptdnxyz »

kristopolous wrote: post #127
georgemcbay wrote: post #117

Earlier quoted context omitted.

As someone from a traditional Boston Catholic family who graduated from Catholic grade and high school and who has since moved away from religion but still has a lot of family and friends who are Catholic, the fact that someone found the idea that Catholics are prone to shame, fear and guilt offensive almost makes me doubt they are Catholic. I've yet to meet one Catholic IRL who wouldn't have a laugh about that, rega…

I think the proper thing to do is if someone is offended is "alright sure, whatever, there you go". Others being offended isn't something you control, responding to it is

Giving in to people who are truly unreasonably offended (by proxy, for social validation, and so on) rewards and incentivizes the behavior, and in fact I believe you have an ethical obligation not to allow people to do this. Promoting antisocial behavior is antisocial.

It's worth having some grace about it though.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#132

Post by stingraycharles »

kweingar wrote: post #41
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

How do we benchmark these different methodologies? It all seems like vibes-based incantations. "You are an expert at finding vulnerabilities." "Please report only real vulnerabilities, not any false positives." Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it?

It’s not that difficult to benchmark these things, eg have an expected result and a few variants of templates.

But yeah prompt engineering is a field for a reason, as it takes time and experience to get it right.

Problem with LLMs as well is that it’s inherently probabilistic, so sometimes it’ll just choose an answer with a super low probability. We’ll probably get better at this in the next few years.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#133

Post by paulddraper »

eqvinox wrote: post #89

Anyone else feel like this is a best case application for LLMs? You could in theory automate the entire process, treat the LLM as a very advanced fuzzer. Run it against your target in one or more VMs. If the VM crashes or otherwise exhibits anomalous behavior, you've found something. (Most exploits like this will crash the machine initially, before you refine them.) On one hand: great application for LLMs. On the oth…

https://security.googleblog.com/2024/11/leveling-up-fuzzing-...

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#134

Post by Terr_ »

threeseed wrote: post #62
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.

> Those prompts should be renamed as hints. [...] its sole overarching goal: to give you an answer no matter whether it's true or not.

I like to think of them as beginnings of an arbitrary document which I hope will be autocompleted in a direction I find useful... By an algorithm with the overarching "goal" of Make Document Bigger.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#135

Post by shakna »

jcims wrote: post #80
threeseed wrote: post #62

Earlier quoted context omitted.

It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.

>people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. What's the alternative?

Using predictable systems.

If your C compiler invents a new function call for a non-existent function while generating code, that's usually a bug.

If an LLM does, that's... Normal. And a non-event.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#136

Post by chii »

NitpickLawyer wrote: post #123
epolanski wrote: post #74

Earlier quoted context omitted.

I find your take amusing considering that's literally the only part of the post he admits to just vibing it: > In fact my entire system prompt is speculative so consider it equivalent to me saying a prayer, rather than anything resembling science or engineering

The difference between vibing and "engineering" is keeping good records, logs and prompt provenance in a methodical way? Also have a (manual) way of reviewing the results. :) (paraphrased from mythbusters)

as the mythbusters have famously said, the only difference between science and fucking around is writing it down.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#137

Post by moffkalast »

conradev wrote: post #104
epolanski wrote: post #74

Earlier quoted context omitted.

I find your take amusing considering that's literally the only part of the post he admits to just vibing it: > In fact my entire system prompt is speculative so consider it equivalent to me saying a prayer, rather than anything resembling science or engineering

A good engineer can vibe good engineering plans! Just like Eisenhower's famous "plans are useless, planning is indispensable" quote. The muscle you build is creating new plans, not memorizing them.

People also underestimate how much winging it is actually the ideal approach for a natural language interface, since that's the kind of thing it was trained on anyway.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#138

Post by baq »

threeseed wrote: post #62
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.

You’re confusing engineering with maths. You engineer your prompting to maximize the chance the LLM does what you need - in your example, the true answer - to get you closer to solving your problem. It doesn’t matter what the LLM does internally as long as the problem is being solved correctly.

(As an engineer it’s part of your job to know if the problem is being solved correctly.)

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#139

Post by [deleted] »

conradev wrote: post #104
epolanski wrote: post #74

Earlier quoted context omitted.

I find your take amusing considering that's literally the only part of the post he admits to just vibing it: > In fact my entire system prompt is speculative so consider it equivalent to me saying a prayer, rather than anything resembling science or engineering

A good engineer can vibe good engineering plans! Just like Eisenhower's famous "plans are useless, planning is indispensable" quote. The muscle you build is creating new plans, not memorizing them.

[deleted]

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#140

Post by tecleandor »

SamuelAdams wrote: post #35
mezyt wrote: post #19

Meanwhile, as a maintainer, I've been reviewing more than a dozen false positives slop CVEs in my library and not a single one found an actual issue. This article's is probably going to make my situation worse.

Maybe, but the author is an experienced vulnerability analyst. Obviously if you get a lot of people who have no experience with this you may get a lot of sloppy, false reports. But this poster actually understands the AI output and is able to find real issues (in this case, use-after-free). From the article: > Before I get into the technical details, the main takeaway from this post is this: with o3 LLMs have made a…

Not even that. The author already knew the bug was there, and fed the LLM just the files related to the bug, with the explanation on how the methods worked and where to search, and even then, only 1 out of 100 times did it find the bug.
Post reply on HN