Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used o3 to find a remote zeroday in the Linux SMB implementation

sean.heelan.io

Post reply on HN
Print view View original

201–210 of 232 posts

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#201

Post by naasking »

kweingar wrote: post #197
naasking wrote: post #145

Earlier quoted context omitted.

> Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it? You just described one critical aspect of engineering: discovering a property of a system and feeding that knowledge back into a systematic, iterative process of refinement.

I can't think of many engineering disciplines that do things this way. "This seems to work, I don't know how or why it works, I don't even know if it's possible to know how or why it works, but I will just apply this moving forward, crossing my fingers that in future situations it will work by analogy." If the act of discovery and iterative refinement makes prompting an engineering discipline, then is raising a baby…

Lots of engineering disciplines work this way. For instance, materials science is still crude, we don't have perfect theories for why some materials have the properties they do (like concrete or superconductors), we simply quantify what those properties are under a wide range of conditions and then make use of those materials under suitable conditions.

> then is raising a baby also an engineering discipline?

The key to science and engineering is repeatability. Raising a baby is an N=1 trial, no guarantees of repeatability.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#202

Post by brookst »

ngneer wrote: post #198
brookst wrote: post #188

Earlier quoted context omitted.

LLMs are just math. It’s reasonable to scope one’s interest down to easily predictable, simple systems. But most of the value in math and computer science is at the scale where there is unpredictability arising from complexity.

It's reasonable to perceive most of the value in math and computer science being "at the scale" where there is unpredictability arising from complexity, though scale may not really be the reason for the unpredictability. But a lot of the trouble in these domains that I have observed comes from unmodeled effects, that must be modeled and reasoned about. GPZ work shows the same thing shown by the researcher here, which…

I see LLMs as tools, so, sure I think they’re in a position to do so the same way pen testing tools or spreadsheets are.

In the hands of an expert, I believe they can help. In the hands of someone clueless, they will just confuse everyone, much like any other tool the clueless person uses.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#203

Post by justsomehnguy »

dotancohen wrote: post #155
dotancohen wrote: post #154

Earlier quoted context omitted.

I recall this even being in the Anthropic documentation.

Here, found it: > Use XML tags to structure your prompts > There are no canonical “best” XML tags that Claude has been trained with in particular, although we recommend that your tag names make sense with the information they surround. https://docs.anthropic.com/en/docs/build-with-claude/prompt-...

My guess would be there is enough training materiel what a mere tagging sometging is enough to have a bigger SNR.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#204

Post by baby »

baby wrote: post #93

I have a presentation here on doing it to target zk bugs https://youtu.be/MN2LJ5XBQS0?si=x3nX1iQy7iex0K66

Posted that in a haste, but meant to share as this might be interesting to people who are trying to do the same kind of things :) I have more updates now, reach out if you wanna talk!

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#205

Post by stonepresto »

lyu07282 wrote: post #183
stonepresto wrote: post #171

Earlier quoted context omitted.

PoCs should at least trigger a crash, overwrite a register, or have some other provable effect, the point being to determine: 1) If it is actually a UAF or if there is some other mechanism missing from the context that prevents UAF. 2) The category and severity of the vulnerability. Is it even a DoS, RCE, or is the only impact causing a thread to segfault? This is all part of the standard vulnerability research proce…

Even a rudimentary exploit can be a significant time investment, it is absolutely not common practice to develop, publish or to demand such exploits from researchers to demonstrate memory corruption vulnerabilities. Everyone thinks they are an expert in infosec its so funny.

Well, in another subthread the author said he did in fact make a crashing PoC. I guess it depends on the customer's standards, but I would say in the vast majority of cases (especially for nuanced memory corruptions in which the ability to make something exploitable depends on your ability to demonstrate control of the heap) a crashing PoC is the bare minimum. In most VDPs, BBPs, or red team engagements you are required to provide some sort of proof to claim, otherwise you'll be laughed out of the room.

I'm curious which sector of infosec you're referring to in which vulnerability researchers are not required to provide proofs of concept? Maybe internal product VR where there is already an established trust?

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#206

Post by topaz0 »

sadeshmukh wrote: post #118
topaz0 wrote: post #115

Earlier quoted context omitted.

Those types of calculation are extremely disingenuous.

What exactly is disingenuous about it?

It reduces the value of a human life to the incremental rate at which they produce some concrete product. It is absurd.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#207

Post by avianlyric »

skydhash wrote: post #195
avianlyric wrote: post #194

Earlier quoted context omitted.

Ah my apologies. I didn’t realise you’re an individual capable of designing and building complex systems made of multiple interconnected novel modules using only wireframes, and having all that work without any prior experimentation. For the rest of us less fortunate, LLMs can be a fantastic tool to sketch out novel modules quickly, and then test assumptions and interactions between them, before committing to a speci…

> I didn’t realise you’re an individual capable of designing and building complex systems made of multiple interconnected novel modules using only wireframes, and having all that work without any prior experimentation. Not really. It's just that there's a lot of prior works out there, so I don't need to do experimentation when someone has already done it and describe the lessons learned. Then you do requirement analy…

How do you handle work that involves building novel systems, where good prior art simply doesn’t exist?

I’m currently dealing with a project that involves developing systems where the existing prior art is either completely proprietary and inaccessible, or public, but extremely nacient and thus documented learnings are less developed than our own learnings and designs.

Many projects may have the primary objective of getting something working. But we don’t all have the luxury of being able to declare something working and walk away. I specifically have requirements around long term evolution of our project (I.e. over a 5-10 year time horizon at a minimum), plus long term operational burden and cost. While also delivering value in the short term.

LLM provide are an invaluable tool for exploring the many possible solutions to what we’re building, and helping to evaluate the longer term consequences of our design decisions, before we’ve committed significant resources to developing them completely.

Of course we could do all this without LLMs, but LLMs substantially increase the distance we can explore before timelines force us to commit.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#208

Post by limflick »

kweingar wrote: post #197
naasking wrote: post #145

Earlier quoted context omitted.

> Organizing things with made-up HTML tags because the models seem to like that for some reason. Where does engineering come into it? You just described one critical aspect of engineering: discovering a property of a system and feeding that knowledge back into a systematic, iterative process of refinement.

I can't think of many engineering disciplines that do things this way. "This seems to work, I don't know how or why it works, I don't even know if it's possible to know how or why it works, but I will just apply this moving forward, crossing my fingers that in future situations it will work by analogy." If the act of discovery and iterative refinement makes prompting an engineering discipline, then is raising a baby…

I think the point is that it's more about trial and error, and less about blindly winging it. When you don't know how a system seems to work, you latch on to whatever seems to initially work and proceed from there to find patterns. It's not an entire approach to engineering, just a small part of the process.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#209

Post by limflick »

threeseed wrote: post #62
nxobject wrote: post #33

A small thing, but I found the author's project-organization practices useful – creating individual .prompt files for system prompt, background information, and auxiliary instructions [1], and then running it through `llm`. It reveals how good LLM use, like any other engineering tool, requires good engineering thinking – methodical, and oriented around thoughtful specifications that balance design constraints – for b…

It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control. Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.

> It's amusing to me how people keep trying to apply engineering principles to an inherently unstable and unpredictable system in order to get a feeling of control.

Are you Insinuating that dealing with unstable and unpredictable systems isn't somewhere engineering principles are frequently applied to solve complex problems?

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#210

Post by limflick »

Aachen wrote: post #176
Aurornis wrote: post #109

Earlier quoted context omitted.

I’ve developed a few take-home interview problems over the years that were designed to be short, easy for an experienced developer, but challenging for anyone who didn’t know the language. All were extracted from real problems we solved on the job, reduced into something minimal. Every time a new frontier LLM is released (excluding LLMs that use input as training data) I run the interview questions through it. I’ve b…

> challenging for anyone who didn’t know the language. Interviewees don't get to pick the language? If you're hiring based on proficiency in a particular tech stack, I'm curious why. Are there that many candidates that you can be this selective? Is the language so dissimilar that the uninitiated would need a long time to get up to speed? Does the job involve working on the language itself and so a specifically deep u…

Aren't most interviews like this? Most dev openings I see posted mention the specific language who's expertise they're looking for and the number of years of experience needed working with said language as well.

It can be annoying, but manageable. I've never coded in Java for example, but knowing C#, C++ and Python I imagine it wouldn't be too hard to pick up.

Post reply on HN