Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TIL you can de-obfuscate code with ChatGPT

twitter.com

Post reply on HN
Print view View original

61–70 of 111 posts

Re: TIL you can de-obfuscate code with ChatGPT

#61

Post by macNchz »

aliqot wrote: post #46
nprateem wrote: post #35

Earlier quoted context omitted.

I know, all those frusty old grumpyboots who actually want the thing to return factually accurate answers and valid code. Just be happy with plausible sounding answers people. Sheesh!

I've had a few instances where it returned bad code or was unable to solve a challenge, however most were fixed by better prompts, or by clarifying prompts. In a way I think there is a two-way "learning" process going on here. I'm training it how to give me what I ask for, and it trains me how to ask for what I want it to give me.

The tricky bit IMO is when you’re at the threshold of being able to identify errors it makes. I tested some situations a while ago where I asked for some physics calculations functions. I’m an experienced programmer but haven’t really done anything with physics since high school 20 years ago. The code returned looked plausible and would run, but going through it line by line and looking up the real formulas it turned out to be super wrong.

Re: TIL you can de-obfuscate code with ChatGPT

#62

Post by [deleted] »

Jeema101 wrote: post #13
crizzlenizzle wrote: post #4

Is it correct though? I’ve been toying around with ChatGPT for a few weeks now and I encountered a few situations in which ChatGPT was like 90% accurate at best. Things like suggesting snippets of configuration files or plugin research. It’s good to get an idea and get started somewhere, but I certainly cannot trust it blindly.

Don't think so. There's clearly the beginning of a while loop near the top of the obfuscated version. There's no loops at all in the 'de-obfuscated' version.

[deleted]

Re: TIL you can de-obfuscate code with ChatGPT

#63

Post by alexandrius »

nprateem wrote: post #35
dsabanin wrote: post #30

I have to say, I find all the comments dismissing ChatGPT hilarious. I read them in a funny grandpa voice. However, we should look past the insignificant details. The main achievement is that we now have a really capable unstructured text-to-computer interface. We can hook it up to anything and it will give us answers with whatever properties we desire, in whatever shape we can think of.

I know, all those frusty old grumpyboots who actually want the thing to return factually accurate answers and valid code. Just be happy with plausible sounding answers people. Sheesh!

Right? Devs are so butthurt that, are dismissing this all together. It’s called denial.

Re: TIL you can de-obfuscate code with ChatGPT

#64

Post by azatom »

Two types of comments:

- "the result doesn't even work..." yeah, even to be a rubberduck is amazing, what thll you expect, got your payment too?

- "wow, amazing...": not really, best case it is a google without (direct) advertisement, found the original code/very similar parts..

tryied with own obfuscated code, not from the net... can not get anything from it

Re: TIL you can de-obfuscate code with ChatGPT

#65

Post by seanhunter »

crizzlenizzle wrote: post #4

Is it correct though? I’ve been toying around with ChatGPT for a few weeks now and I encountered a few situations in which ChatGPT was like 90% accurate at best. Things like suggesting snippets of configuration files or plugin research. It’s good to get an idea and get started somewhere, but I certainly cannot trust it blindly.

Yeah an example I was shown was python code to process some data. It was 30 lines of correct-looking trivial boilerplate code, except for one regex to do the actual processing. The regex was hopelessly wrong.

Clearly if you didn't know how to write the other 29 lines of code there's no way you are going to be able to debug the regex.

Re: TIL you can de-obfuscate code with ChatGPT

#66

Post by whamlastxmas »

folkrav wrote: post #58
zahrc wrote: post #31

Earlier quoted context omitted.

In my experience, ChatGPT often comes up with pseudo syntax.

It often happens that ChatGPT will confidently give you something that _looks_ like what you're asking for despite it being awfully wrong - sometimes you can make it "understand" its mistake and correct it, sometimes not. It's usually not that far off, but trusting it blindly is just out of the question.

I was having ChatGPT give me wildly wrong answers and when I asked for a source it provided me fake websites and confidently quoted information from those sites that have never existed

Re: TIL you can de-obfuscate code with ChatGPT

#67

Post by whamlastxmas »

garrettjoecox wrote: post #22
mlatu wrote: post #10

i would rather ask it to give me the python source of a program capable to deobfuscate THAT string. anything else is just naive, wishfull thinking and a waste of time. you will have to deobfuscate the code manually anyways but at least you got on HN EDIT: OMG... the obfuscated code was generated by chatgpt AND DOESNT EVEN WORK

Are you just now realizing these LLM’s can’t factually determine if what they output is correct?

It actively makes shit up in my experience. Like outright lies about basic facts.

Re: TIL you can de-obfuscate code with ChatGPT

#68

Post by vlunkr »

seanhunter wrote: post #65
crizzlenizzle wrote: post #4

Is it correct though? I’ve been toying around with ChatGPT for a few weeks now and I encountered a few situations in which ChatGPT was like 90% accurate at best. Things like suggesting snippets of configuration files or plugin research. It’s good to get an idea and get started somewhere, but I certainly cannot trust it blindly.

Yeah an example I was shown was python code to process some data. It was 30 lines of correct-looking trivial boilerplate code, except for one regex to do the actual processing. The regex was hopelessly wrong. Clearly if you didn't know how to write the other 29 lines of code there's no way you are going to be able to debug the regex.

The optimistic way to look at it though is that it wrote the boring 29 lines that you didn't want to write and got you straight to the actual problem that needs solving.

Re: TIL you can de-obfuscate code with ChatGPT

#69

Post by moyix »

jerf wrote: post #60
moyix wrote: post #55

Earlier quoted context omitted.

I don't really see this as a problem. Once you have a first cut deobfuscation from this you can refine it with other methods, like comparing input/output examples between the original and the deobfuscated version, or even use something more sophisticated like symbolic execution [1] or differential fuzzing [2] to systematically look for divergence between the behavior of the two. You could even feed these back in to C…

You seem to have just blipped by the deobfuscation from ChatGPT being actively wrong . I meant what I said. I expect ChatGPT would happily output a substring search algorithm for the accept loop of an HTTP server if you just put enough "haystack" and "needle" words in the obfuscated code. How are you supposed to "refine" that into the truth? To the extent that there is an answer, the answer is, completely ignore the…

When it gets it entirely wrong that will be trivially detected by an I/O example, no? So I don't see that as dangerous, just inconvenient (it sometimes doesn't work, but you know when it doesn't work). You can also use an existing semantics-preserving deobfuscator and then use that as the input to an LLM deobfuscator instead of the original.

If you're saying that obfuscators can eventually adapt, then sure. So can deobfuscators. This particular problem is kind of inherently an arms race.

Re: TIL you can de-obfuscate code with ChatGPT

#70

Post by ok123456 »

buster wrote: post #42
shever73 wrote: post #24

https://twitter.com/AlexAlexandrius/status/16178852870484254... So, all I've learned is that ChatGPT knows the obfuscated and de-obfuscated versions of code that it itself has generated.

Which is not runnable in the first place. Interesting though, as this shows exactly the problem. It looks legit, but it's some generated fake text.

It tends to invent whole fake packages.
Post reply on HN