Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I used o3 to find a remote zeroday in the Linux SMB implementation

sean.heelan.io

Post reply on HN
Print view View original

221–230 of 232 posts

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#221

Post by ponector »

Aachen wrote: post #176
Aurornis wrote: post #109

Earlier quoted context omitted.

I’ve developed a few take-home interview problems over the years that were designed to be short, easy for an experienced developer, but challenging for anyone who didn’t know the language. All were extracted from real problems we solved on the job, reduced into something minimal. Every time a new frontier LLM is released (excluding LLMs that use input as training data) I run the interview questions through it. I’ve b…

> challenging for anyone who didn’t know the language. Interviewees don't get to pick the language? If you're hiring based on proficiency in a particular tech stack, I'm curious why. Are there that many candidates that you can be this selective? Is the language so dissimilar that the uninitiated would need a long time to get up to speed? Does the job involve working on the language itself and so a specifically deep u…

That is the market nowadays. Employers seek not only deep knowledge in particular language, but also particular libraries. If you cannot answer interview questions about implementation of some features - you are out.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#222

Post by skydhash »

avianlyric wrote: post #220
skydhash wrote: post #213

Earlier quoted context omitted.

Maybe the main problem is not solved yet, but I highly doubt that the subproblems are not. Because that would be cutting edge domain, which is very much an outlier.

Ah so what exactly do you mean when you say > Most of my coding is fully planned to get to the end. The experiment part is on a much smaller scale (module level). I would seem that these statements taken together mean you don’t experiment at all?

That means that I take time to analyze the problem and come up with a convincing design (mostly research, and experience). After that I've just got a few parameters that I don't know much about. But that doesn't mean that I can't build the stuff. I just isolate them so that I can tweak them later. Why? Because they are often accidental complexities, not essential ones.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#223

Post by Aachen »

limflick wrote: post #210
Aachen wrote: post #176

Earlier quoted context omitted.

> challenging for anyone who didn’t know the language. Interviewees don't get to pick the language? If you're hiring based on proficiency in a particular tech stack, I'm curious why. Are there that many candidates that you can be this selective? Is the language so dissimilar that the uninitiated would need a long time to get up to speed? Does the job involve working on the language itself and so a specifically deep u…

Aren't most interviews like this? Most dev openings I see posted mention the specific language who's expertise they're looking for and the number of years of experience needed working with said language as well. It can be annoying, but manageable. I've never coded in Java for example, but knowing C#, C++ and Python I imagine it wouldn't be too hard to pick up.

Huh, okay. That's not how we run interviews but I guess it's at least a thing, even if not common around here that I've seen yet (I'm not super current on interview practices though)

Regarding the job ads, yes they'd describe the ideal candidate but I haven't the experience that the perfect candidate ever actually shows up. Like you say, knowing J, T and Z, the company is confident enough that you'll be able to quickly pick up dotting the Is and crossing the 7s

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#224

Post by tuetuopay »

Aachen wrote: post #176
Aurornis wrote: post #109

Earlier quoted context omitted.

I’ve developed a few take-home interview problems over the years that were designed to be short, easy for an experienced developer, but challenging for anyone who didn’t know the language. All were extracted from real problems we solved on the job, reduced into something minimal. Every time a new frontier LLM is released (excluding LLMs that use input as training data) I run the interview questions through it. I’ve b…

> challenging for anyone who didn’t know the language. Interviewees don't get to pick the language? If you're hiring based on proficiency in a particular tech stack, I'm curious why. Are there that many candidates that you can be this selective? Is the language so dissimilar that the uninitiated would need a long time to get up to speed? Does the job involve working on the language itself and so a specifically deep u…

> Interviewees don't get to pick the language?

For leetcode interviews, sure. Other than that, at least familiarity with the language is paramount, or with the same class of language.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#225

Post by shakna »

jcims wrote: post #218
shakna wrote: post #216

Earlier quoted context omitted.

Every engineering domain operates within "known bounds". That makes it dependable. An engineer doesn't just shrug and pick up slag because it contains the same materials as the original bauxite.

Of course not, but how did we get to that point with materials science and chemistry? We’re basically in the stone ages of understanding how to interact with synthetic intelligence.

Through research and experimentation, yes.

But attempts to integrate little understood things in daily life gave us radium toothpaste and lead poisoning. Let's not repeat stone age mistakes. Research first, integrate later.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#226

Post by avianlyric »

skydhash wrote: post #222
avianlyric wrote: post #220

Earlier quoted context omitted.

Ah so what exactly do you mean when you say > Most of my coding is fully planned to get to the end. The experiment part is on a much smaller scale (module level). I would seem that these statements taken together mean you don’t experiment at all?

That means that I take time to analyze the problem and come up with a convincing design (mostly research, and experience). After that I've just got a few parameters that I don't know much about. But that doesn't mean that I can't build the stuff. I just isolate them so that I can tweak them later. Why? Because they are often accidental complexities, not essential ones.

> That means that I take time to analyze the problem and come up with a convincing design (mostly research, and experience).

Ah I think we’re finally getting somewhere. My point is that you can use LLM as part of that research process. Not just as a poor substitute for proper research, but as a tool for experimental research. It’s supplemental to the normal research process, and is certainly not a tool for creating final outputs.

Using LLMs like that can make a meaningful difference to speed and quality of the analysis and final design. And something you should consider, rather than dismissing out of hand.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#227

Post by topaz0 »

sadeshmukh wrote: post #215
topaz0 wrote: post #206

Earlier quoted context omitted.

It reduces the value of a human life to the incremental rate at which they produce some concrete product. It is absurd.

Or, it elevates the tasks artificial intelligence produces to the actual difficulty of them - the human effort.

You're not thinking this through. Your human life (with its associated 2000 Cal/day) does so much more than find bugs in obscure codebases. Or at least, one would hope.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#228

Post by GaggiX »

yencabulator wrote: post #186
JFingleton wrote: post #60

Earlier quoted context omitted.

Zero days can go for $$$, or you can go down the bug bounty route and also get $$. The cost of the LLM would be a drop in the bucket. When the cost of inference gets near zero, I have no idea what the world of cyber security will look like, but it's going to be a very different space from today.

Except in this case the LLM was pointed at a known-to-exist vulnerability. $116 per handler per vulnerability type, unknown how many vulnerabilities exist.

The o3 discovered a new zero day exploit, it wasn't known previously, it's not the same one found by the author.

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#229

Post by tekacs »

tekacs wrote: post #53
martinald wrote: post #52

I think this is the biggest alignment problem with LLMs in the short term imo. It is getting scarily good at this. I recently found a pretty serious security vulnerability in an open source very niche server I sometimes use. This took virtually no effort using LLMs. I'm worried that there is a huge long tail of software out there which wasn't worth finding vulnerabilities in for nefarious means manually but if it was…

The (obvious) flipside of this coin is that it allows us to run this adversarially against our own codebases, catching bugs that could otherwise have been found by a researcher, but that we can instead patch proactively.\ I wouldn't (personally) call it an alignment issue, as such.

A few days later, case in point (I'm in no way affiliated): https://news.ycombinator.com/item?id=44117465

Re: I used o3 to find a remote zeroday in the Linux SMB implementation

#230

Post by whattheheckheck »

ngneer wrote: post #161
baq wrote: post #138

Earlier quoted context omitted.

You’re confusing engineering with maths. You engineer your prompting to maximize the chance the LLM does what you need - in your example, the true answer - to get you closer to solving your problem. It doesn’t matter what the LLM does internally as long as the problem is being solved correctly. (As an engineer it’s part of your job to know if the problem is being solved correctly.)

Maybe very very soft "engineering". Do you have metrics on which prompt is best? What units are you measuring this in? Can you follow a repeatable process to obtain a repeatable result?

Skillful maneuvering or direction
Post reply on HN