Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The coming industrialisation of exploit generation with LLMs

sean.heelan.io

Post reply on HN
Print view View original

131–140 of 174 posts

Re: The coming industrialisation of exploit generation with LLMs

#131

Post by rramadass »

saagarjha wrote: post #109

> The exploits generated do not demonstrate novel, generic breaks in any of the protection mechanisms. They take advantage of known flaws in those protection mechanisms and gaps that exist in real deployments of them. These are the same gaps that human exploit developers take advantage of, as they also typically do not come up with novel breaks of exploit mitigations for each exploit. I actually think this result is…

> formal modeling of exploits, which is currently a very immature field.

Can you elaborate more on this with pointers to some resources?

Re: The coming industrialisation of exploit generation with LLMs

#134

Post by pnathan »

KellyCriterion wrote: post #126
larodi wrote: post #124

two points - 1) it becomes increasingly more dangerous to dl stuff from the internet and just run it, even its opensource, given normally people don't read all of it. for weird repos I'd recomment to do automated analysis with opus 4.5 or the gpt 5.2 indeed. 2) if we assume adversaries are using LLMs to churn exploits 24/7, which we should absolutely do, perhaps the time where we turn the internet off whenever is not…

...well, just dont download random stuff from the internet and run it on your important machines then? :-)) You are right: 30 years ago, it was safe to go to vendor XY page and download his latest version and it was more or less waterproof. Today with all these mirror sites, very often better SEO ranking than the original, its quite dangerous: In my former bank we had a colleague who installed a browser add-in that h…

> 30 years ago, it was safe to go to vendor XY page and download his latest version and it was more or less waterproof.

You _are_ joking, right? I distinctly remember all sorts of dubious freewarez sites with slightly modified installers. 1997-2000 era. And anti-virus was a thing in MS-DOS even.

Re: The coming industrialisation of exploit generation with LLMs

#135

Post by KellyCriterion »

pnathan wrote: post #134
KellyCriterion wrote: post #126

Earlier quoted context omitted.

...well, just dont download random stuff from the internet and run it on your important machines then? :-)) You are right: 30 years ago, it was safe to go to vendor XY page and download his latest version and it was more or less waterproof. Today with all these mirror sites, very often better SEO ranking than the original, its quite dangerous: In my former bank we had a colleague who installed a browser add-in that h…

> 30 years ago, it was safe to go to vendor XY page and download his latest version and it was more or less waterproof. You _are_ joking, right? I distinctly remember all sorts of dubious freewarez sites with slightly modified installers. 1997-2000 era. And anti-virus was a thing in MS-DOS even.

back then we were sharing Shareware or Freeare or PD-Ware by swapping disks and copying magazine disks :-D

but, you are old enough - so mean pages like fosi.da.ru back then? ;-)

Re: The coming industrialisation of exploit generation with LLMs

#137

Post by QuadmasterXLII »

tptacek wrote: post #48
QuadmasterXLII wrote: post #42

Earlier quoted context omitted.

These exploits were costing $50 of API credit each. If you receive 5001 issues from $100 in API spend on bug hunting and one of the issues cost $50 and the other 5000 cost one cent each, and they’re all visually indistinguishable using perfect grammar and familiar cyber security lingo; hard to find the dianond.

The point of the post is that the harness generates a POC. It either works or it doesn't.

https://hackerone.com/reports/3100073 includes a POC too- still slop

Re: The coming industrialisation of exploit generation with LLMs

#138

Post by jdefr89 »

Vulnerability Researcher/Reverse Eng here... Aspects about it generating an API for read/write primitives are simply it regurgitating tons of APIs that exist already. Its still cool, but its not like it invented the primitives or any novel technique. Also, this toy JS is similar to binaries you'd find in a CTF. Of course it will be able to solve majority of those. I am curious though.. Latest OpenAI models don't seem to want to generate any real exploit code. Is there a prompt jail break or something being used here?

Re: The coming industrialisation of exploit generation with LLMs

#139

Post by jdefr89 »

simonw wrote: post #5

> In the hardest task I challenged GPT-5.2 it to figure out how to write a specified string to a specified path on disk, while the following protections were enabled: address space layout randomisation, non-executable memory, full RELRO, fine-grained CFI on the QuickJS binary, hardware-enforced shadow-stack, a seccomp sandbox to prevent shell execution, and a build of QuickJS where I had stripped all functionality in…

Most modern kill chains involve chaining together that many bugs... I know because it's my job and its become demoralizing.

Re: The coming industrialisation of exploit generation with LLMs

#140

Post by atomic128 »

GaggiX wrote: post #121
ahartmetz wrote: post #119

Earlier quoted context omitted.

Maybe we can remove mitigations. Every exploit you see is: First, find a vulnerability (the difficult part). Then, drill through five layers of ultimately ineffective "mitigations" (the tedious but almost always doable part). Probabilistic mitigations work against probabilistic attacks, I guess - but exploit writers aren't random, they are directed, and they find the weaknesses.

The vulnerability was found by Opus: "This is true by definition as the QuickJS vulnerability was previously unknown until I found it (or, more correctly: my Opus 4.5 vulnerability discovery agent found it)."

Number 6, explained 3 years ago:

https://github.com/nobodyisnobody/docs/blob/main/code.execut...

Original publication in 2017:

https://m101.github.io/binholic/2017/05/20/notes-on-abusing-...

Post reply on HN