Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TIL you can de-obfuscate code with ChatGPT

twitter.com

Post reply on HN
Print view View original

51–60 of 111 posts

Re: TIL you can de-obfuscate code with ChatGPT

#51

Post by rednerrus »

It's been my experience that ChatGPT gets things wrong sometimes. It's also been my experience that if you say "X isn't working as expected." it will do its best to fix the issues. It usually does a pretty good job of fixing it.

I had it write a handful of scripts for me yesterday. It got about 90% of it right on the first pass and 99% of it right on the second pass. You still need to have some understanding of what you're doing so you can see when things are wrong but man if it doesn't save you a lot of time.

Re: TIL you can de-obfuscate code with ChatGPT

#52

Post by Sakos »

twic wrote: post #43
garrettjoecox wrote: post #22

Earlier quoted context omitted.

Are you just now realizing these LLM’s can’t factually determine if what they output is correct?

No, i'm just now realizing that HN users can’t factually determine if what these LLM's output is correct.

Turns out we're all just ChatGPTs after all.

Re: TIL you can de-obfuscate code with ChatGPT

#53

Post by Lutger »

crizzlenizzle wrote: post #4

Is it correct though? I’ve been toying around with ChatGPT for a few weeks now and I encountered a few situations in which ChatGPT was like 90% accurate at best. Things like suggesting snippets of configuration files or plugin research. It’s good to get an idea and get started somewhere, but I certainly cannot trust it blindly.

90% accurate sounds impressive, and it is, but its still 100% incorrect almost always.

Re: TIL you can de-obfuscate code with ChatGPT

#54

Post by Sakos »

mlatu wrote: post #21
teeray wrote: post #12

Every commenter is jumping in asking “is it correct?” Even if it’s not 100%, if it’s at least reasonably close, it could be a tremendous force-multiplier against obfuscation for someone with some familiarity with roughly what the code is trying to do.

the input was generated by chatgpt. and is non-functional. so, ROUGHLY, what is it trying to do?

I'm wondering why people are so quick to defend ChatGPT.

Re: TIL you can de-obfuscate code with ChatGPT

#55

Post by moyix »

jerf wrote: post #17
teeray wrote: post #12

Every commenter is jumping in asking “is it correct?” Even if it’s not 100%, if it’s at least reasonably close, it could be a tremendous force-multiplier against obfuscation for someone with some familiarity with roughly what the code is trying to do.

Correctness is a big deal here. This is a security context and we can assume that the obfuscators are active attackers against legible code, not just people passively hoping that their obfuscated code is obfuscated. If this becomes a popular technique, then code obfuscation tools will simply pivot to writing code that ChatGPT gets wrong when asked to unobfuscate it. I can't even imagine that would be a particularly h…

I don't really see this as a problem. Once you have a first cut deobfuscation from this you can refine it with other methods, like comparing input/output examples between the original and the deobfuscated version, or even use something more sophisticated like symbolic execution [1] or differential fuzzing [2] to systematically look for divergence between the behavior of the two. You could even feed these back in to ChatGPT and ask it to redo the deobfuscation given a failing test case.

Such testing won't be able to prove that the two are equivalent (unless it's exhaustive) but with decent coverage of the original you can get some good confidence. The goal of deobfuscation is usually understanding, so I'm not sure you need strong guarantees of perfect semantic equivalence with no human intervention/judgment.

And of course, existing deobfuscators have bugs and aren't guaranteed to preserve semantics either.

[1] https://en.wikipedia.org/wiki/Symbolic_execution

[2] https://en.wikipedia.org/wiki/Differential_testing

Re: TIL you can de-obfuscate code with ChatGPT

#57

Post by wittycardio »

mlatu wrote: post #49
worldsavior wrote: post #27

Earlier quoted context omitted.

So what?

the "obfuscated" "version" doesnt even work. alex literally asked chatgpt to come up with a math problem and its solution, both from whole cloth. and you ask so what. well, everyone should ask "so what?" to alex.

[dead]

Re: TIL you can de-obfuscate code with ChatGPT

#58

Post by folkrav »

zahrc wrote: post #31
crizzlenizzle wrote: post #4

Is it correct though? I’ve been toying around with ChatGPT for a few weeks now and I encountered a few situations in which ChatGPT was like 90% accurate at best. Things like suggesting snippets of configuration files or plugin research. It’s good to get an idea and get started somewhere, but I certainly cannot trust it blindly.

In my experience, ChatGPT often comes up with pseudo syntax.

It often happens that ChatGPT will confidently give you something that _looks_ like what you're asking for despite it being awfully wrong - sometimes you can make it "understand" its mistake and correct it, sometimes not. It's usually not that far off, but trusting it blindly is just out of the question.

Re: TIL you can de-obfuscate code with ChatGPT

#59

Post by amrb »

pathartl wrote: post #44
amrb wrote: post #37

Earlier quoted context omitted.

Silly idea tho would it be helpful in creating the server side code for an online only game? I've seen a project for battlefield 3 tho already have the feeling it's a team effort at minimum?

It's not some magic bullet. It helps a ton with trying to give names to obfuscated function and variable names, but you have to be intelligent enough to know what the code's actually doing. It probably helps RE teams a lot, but until it can easily run across an entire codebase it's just another tool in the toolbox.

Agreed its can lacking logic till to spell it out, example was this ctf challenge and it just could understand the hash collision till I gave it the full write up [0]

w.r.t codebases I may look at some of the free models (as this gets around the cost problem) and try to feed it prompts as a block of code plus meaningful references to same under the token limit.

[0] https://github.com/victor-li/pwnable.kr-write-ups/blob/maste...

Re: TIL you can de-obfuscate code with ChatGPT

#60

Post by jerf »

moyix wrote: post #55
jerf wrote: post #17

Earlier quoted context omitted.

Correctness is a big deal here. This is a security context and we can assume that the obfuscators are active attackers against legible code, not just people passively hoping that their obfuscated code is obfuscated. If this becomes a popular technique, then code obfuscation tools will simply pivot to writing code that ChatGPT gets wrong when asked to unobfuscate it. I can't even imagine that would be a particularly h…

I don't really see this as a problem. Once you have a first cut deobfuscation from this you can refine it with other methods, like comparing input/output examples between the original and the deobfuscated version, or even use something more sophisticated like symbolic execution [1] or differential fuzzing [2] to systematically look for divergence between the behavior of the two. You could even feed these back in to C…

You seem to have just blipped by the deobfuscation from ChatGPT being actively wrong.

I meant what I said. I expect ChatGPT would happily output a substring search algorithm for the accept loop of an HTTP server if you just put enough "haystack" and "needle" words in the obfuscated code. How are you supposed to "refine" that into the truth?

To the extent that there is an answer, the answer is, completely ignore the ChatGPT output and use existing tools. Which is to say, ChatGPT would be worse than useless at that point.

I'm not saying ChatGPT will be slightly off, and maybe the obfuscator can kick it to be another 5 or 10% wrong. I'm saying, it is likely trivial to update the obfuscator to make ChatGPT utterly wrong, in every detail, up to and including the entire fundamental nature of the code.

Post reply on HN