Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The coming industrialisation of exploit generation with LLMs

sean.heelan.io

Post reply on HN
Print view View original

151–160 of 174 posts

Re: The coming industrialisation of exploit generation with LLMs

#151

Post by Imustaskforhelp »

jerf wrote: post #130
akoboldfrying wrote: post #54

Earlier quoted context omitted.

What I mean is: There will be bugs* in that pure Go implementation, and static linking means you're baking them in forever. Why is this preferable to dynamic linking? * It's likely that C implementations will have bugs related to dynamic memory allocation that are absent from the Go implementation, because Go is GCed while C is not. But it would be very surprising if there were no bugs at all in the Go implementation…

About a year ago I had some code I had been working on for about a year subject to a pretty heavy-duty security review by a reputable review company. When they asked what language I implemented it in and I told them "Go", they joked that half their job was done right there. While Go isn't perfect and you can certainly write some logic bugs that sufficiently clever use of a more strongly-typed language might let you a…

I love golang a lot and I feel like in this context of QuickJS it would be interesting to see what a port of QuickJS with Golang might look like security wise & a comparison to rust in the amount of security as well.

Of course Golang and rust are apples to oranges comparison but still, if someone experienced in golang were to say port to QuickJS to golang and same for rust, aside from some performance cost which can arise from Golang's GC, what would be the security analysis of both?

Also Offtopic but I love how golang has a library for literally everything mostly but its language development ie runtime for interpreted langs/JIT's or transpilation efforts etc. do feel less than rust.

Like For python there's probably a library which can call rust code from Python, I wish if there was something like this for golang and I had found such a project (https://github.com/go-python/gopy) but it still just feels a little less targeted than rust within python which has libraries like polars and other more mature libraries

Re: The coming industrialisation of exploit generation with LLMs

#152

Post by KellyCriterion »

pnathan wrote: post #143
KellyCriterion wrote: post #135

Earlier quoted context omitted.

back then we were sharing Shareware or Freeare or PD-Ware by swapping disks and copying magazine disks :-D but, you are old enough - so mean pages like fosi.da.ru back then? ;-)

I don't remember all the places I got software... :)

...BBS systems e.g....

Re: The coming industrialisation of exploit generation with LLMs

#153

Post by koakuma-chan »

saagarjha wrote: post #103
0xDEAFBEAD wrote: post #73

Earlier quoted context omitted.

Can we fight fire with fire, and use LLMs to rewrite all the C in Rust?

Usually rewriting something in Rust requires nontrivial choices on the part of the translator that I’m not sure are currently within the reach of LLMs.

I heard this before, that apparently there are things you cannot implement in Rust. Like, apparently you cannot implement certain data structures in Rust. I think this is bullshit. Rust supports raw pointers, etc. You can implement whatever you want in Rust.

Re: The coming industrialisation of exploit generation with LLMs

#154

Post by staticassertion »

ahartmetz wrote: post #119
simonw wrote: post #5

> In the hardest task I challenged GPT-5.2 it to figure out how to write a specified string to a specified path on disk, while the following protections were enabled: address space layout randomisation, non-executable memory, full RELRO, fine-grained CFI on the QuickJS binary, hardware-enforced shadow-stack, a seccomp sandbox to prevent shell execution, and a build of QuickJS where I had stripped all functionality in…

Maybe we can remove mitigations. Every exploit you see is: First, find a vulnerability (the difficult part). Then, drill through five layers of ultimately ineffective "mitigations" (the tedious but almost always doable part). Probabilistic mitigations work against probabilistic attacks, I guess - but exploit writers aren't random, they are directed, and they find the weaknesses.

Most mitigations just flat out do not attempt to help against "arbitrary read/write". The LLM didn't just find "a vuln" and then work through the mitigations, it found the most powerful possible vulnerability.

Lots of vulnerabilites get stopped dead by these mitigations. You almost always need multiple vulnerabilities tied together, which relies on a level of vulnerability density that's tractable. This is not just busywork.

Re: The coming industrialisation of exploit generation with LLMs

#155

Post by staticassertion »

koakuma-chan wrote: post #153
saagarjha wrote: post #103

Earlier quoted context omitted.

Usually rewriting something in Rust requires nontrivial choices on the part of the translator that I’m not sure are currently within the reach of LLMs.

I heard this before, that apparently there are things you cannot implement in Rust. Like, apparently you cannot implement certain data structures in Rust. I think this is bullshit. Rust supports raw pointers, etc. You can implement whatever you want in Rust.

Presumably they are saying that you'd end up using a lot of `unsafe`. Of course, that's still much better than C, but I assume that their point isn't "You can't do it in Rust" it's "You can't translate directly to safe rust from C".

Re: The coming industrialisation of exploit generation with LLMs

#156

Post by johnisgood »

koakuma-chan wrote: post #153
saagarjha wrote: post #103

Earlier quoted context omitted.

Usually rewriting something in Rust requires nontrivial choices on the part of the translator that I’m not sure are currently within the reach of LLMs.

I heard this before, that apparently there are things you cannot implement in Rust. Like, apparently you cannot implement certain data structures in Rust. I think this is bullshit. Rust supports raw pointers, etc. You can implement whatever you want in Rust.

Yes, you are looking for https://rcoh.me/posts/rust-linked-list-basically-impossible/.

Re: The coming industrialisation of exploit generation with LLMs

#157

Post by koakuma-chan »

staticassertion wrote: post #155
koakuma-chan wrote: post #153

Earlier quoted context omitted.

I heard this before, that apparently there are things you cannot implement in Rust. Like, apparently you cannot implement certain data structures in Rust. I think this is bullshit. Rust supports raw pointers, etc. You can implement whatever you want in Rust.

Presumably they are saying that you'd end up using a lot of `unsafe`. Of course, that's still much better than C, but I assume that their point isn't "You can't do it in Rust" it's "You can't translate directly to safe rust from C".

> Of course, that's still much better than C

Exactly. "can't translate to safe Rust" is not a good faith argument.

Re: The coming industrialisation of exploit generation with LLMs

#158

Post by koakuma-chan »

koakuma-chan wrote: post #157
staticassertion wrote: post #155

Earlier quoted context omitted.

Presumably they are saying that you'd end up using a lot of `unsafe`. Of course, that's still much better than C, but I assume that their point isn't "You can't do it in Rust" it's "You can't translate directly to safe rust from C".

> Of course, that's still much better than C Exactly. "can't translate to safe Rust" is not a good faith argument.

If anything, writing unsafe code in Rust is also fun. It has many primitives like `MaybeUninit` that make it fun.

Re: The coming industrialisation of exploit generation with LLMs

#159

Post by warkdarrior »

rightbyte wrote: post #108
saagarjha wrote: post #104

Earlier quoted context omitted.

Awesome! What’s your strategy for migration of the entire world’s infrastructure to whatever you’re thinking about?

My strategy is to not use "the entire world's infrastructure" which makes it redundant. If enough people cancel their leftpad-as-a-Service subscription the server can be unplugged. (Yes I am somewhat hyperbolic and yes I see use for internet connected servers and clients. I argue against the SaaS driven centralization.)

> I argue against the SaaS driven centralization.

How does that help with the topic at hand? (LLM-assisted vulnerability research)

Are the decentralized systems that you prefer more secure/less buggy/less exploitable by LLMs?

Re: The coming industrialisation of exploit generation with LLMs

#160

Post by nitwit005 »

nl wrote: post #63

One of the interesting things to me about this is that Codex 5.2 found the most complex of the exploits. The reflects my experience too. Opus 4.5 is my everyday driver - I like using it. But Codex 5.2 with Extra High thinking is just a bit more powerful. Also despite what people say, I don't believe progress in LLM performance is slowing down at all - instead we are having more trouble generating tasks that are hard…

It didn't find the exploits, it wrote code that made use of them. You can see them feeding it exploit descriptions, and samples of making use of them in their log files: https://github.com/SeanHeelan/anamnesis-release/blob/master/...
Post reply on HN