I’ve tried using LLMs to deobfuscate libraires like fingerprintjs-pro to understand what specific heuristics implementation details they use to detect bots.
They mostly fail. A human reverse engineer will still do better.
OpenAI is good at unminifying code
141–150 of 321 posts
Re: OpenAI is good at unminifying code
#142Author of HumanifyJS here! I've created specifically a LLM based tool for this, which uses LLMs on AST level to guarantee that the code keeps working after the unminification step: https://github.com/jehna/humanify
More tools should be built on ASTs, great work! I'm still waiting for the AST level version control tbh
Unison supposedly has an AST-aware version control system: https://www.unison-lang.org/
Re: OpenAI is good at unminifying code
#143Earlier quoted context omitted.
I don’t think they’re saying that minifying provides no additional space savings, but rather that those additional savings are small and not worth the tradeoffs.
Not even that is true in my knowledge. For example a particular benchmark [1] demonstrates that many popular libraries benefit much from minification even after gzip compression, with the saving ranging from 35% to 75%. Sure, a small library would be fine without any minification or even compression, but otherwise minification is clearly beneficial. [1] https://github.com/privatenumber/minification-benchmarks
I think you have to look at this in the context of an entire bundle or project, and then you have to weigh it against the download speeds you’re generally expecting for the users of your site or app.
I agree that as a blanket statement “gzip is enough” is not technically correct, but I think it’s largely correct in spirit, in that people tend to reach for minification by default, without really thinking about what they’re gaining.
If minifying saves you 200 KB overall, for example, and you expect your average user to have a 200 Mbps connection, you’re saving a grand total of 8 ms on page load, which is an imperceptible difference on its own. In exchange, you’re getting worse debugging, and worse error reporting.
Re: OpenAI is good at unminifying code
#144Author of HumanifyJS here! I've created specifically a LLM based tool for this, which uses LLMs on AST level to guarantee that the code keeps working after the unminification step: https://github.com/jehna/humanify
More tools should be built on ASTs, great work! I'm still waiting for the AST level version control tbh
[deleted]
Re: OpenAI is good at unminifying code
#145Earlier quoted context omitted.
I see your point, but I think there's more to it. It's kind of like saying "all humans can do is perceive and produce sound, any other ability is just a side-effect". We might be focusing too much on their mechanism for "perception" and overlooking other capabilities they've developed.
Sure, but that claim wouldn't be true for humans, right? So it's a nonsequiteur. The relevant claim would be: all humans can do is move around in their environments, adapt the world around them through action, observe using adaptive sensory motor systems, grow and adapt their brains and bodies in response to novel and changing environments, abstract sensory motor techniques into symbolic concepts, vocalize this using…
> all they can do is, in fact, sample from a compression of historical texts using a weighted probability metric.
I don't think that's all they can do.
I think they know more than what is explicitly stated in their training sets.
They can generalize knowledge and generalize relationships between the concepts that are in the training sets.
They're currently mediocre at it, but the results we observe from SOTA generative models are not explainable without accepting that they can create an internal model of the world that's more than just a decompression algorithm.
I'm going to step away from LLMs for a moment, but: How are video generator models capable of creating videos with accurate shadows and lighting that is consistent in the entire frame and consistent between frames?
You can't do that simply by taking a weighted average of the sections of videos you've seen in your training set.
You need to create an internal 3D model of the objects in the scene, and their relative positions in space across the length of the video. And no one told the model explicitly how to do that, it learned to do it "on its own".
I think the same principle applies to LLMs.
Re: OpenAI is good at unminifying code
#146Earlier quoted context omitted.
Unfortunately the comments that could be generated are exactly the ones that should never be written. You want the comment to explain why, the information missing from the code.
This is something I always disagreed with. In my experience, I rather read a short comment explaining what is the purpose of a block of code, than trying to decipher it. Yes, code "should speak for itself", but reading a comment is almost always faster than reading blocks of code. And then there is also documentation (if you include it in what you define as comment). I much rather go through a website, with a search…
It's just a rule of thumb, like anything else. In most code, "why" is the hard part; I see that you are incrementing that account by a penny from out of the blue, but why? When you are in code where "what" is the hard part, like an implementation of a book algorithm or some tricky performance optimization, then by all means comment that.
Really all this rule amounts to is
// Increment by a penny
accountValue += 1
Re: OpenAI is good at unminifying code
#147Earlier quoted context omitted.
Unfortunately the comments that could be generated are exactly the ones that should never be written. You want the comment to explain why, the information missing from the code.
This is something I always disagreed with. In my experience, I rather read a short comment explaining what is the purpose of a block of code, than trying to decipher it. Yes, code "should speak for itself", but reading a comment is almost always faster than reading blocks of code. And then there is also documentation (if you include it in what you define as comment). I much rather go through a website, with a search…
> reading a comment is almost always faster than reading blocks of code
Not to a competent programmer when reading well-written code.
This also means that you read what the code does, rather than what a comment says the code does. Otherwise you will be blind to bugs. Any experienced developer will tell you that code very often doesn't do what the original programmer thought it did.
Re: OpenAI is good at unminifying code
#148Should the title say ChatGPT or gpt-4 (the model) instead of OpenAI (the company)?
[deleted]
Re: OpenAI is good at unminifying code
#149Earlier quoted context omitted.
Not even that is true in my knowledge. For example a particular benchmark [1] demonstrates that many popular libraries benefit much from minification even after gzip compression, with the saving ranging from 35% to 75%. Sure, a small library would be fine without any minification or even compression, but otherwise minification is clearly beneficial. [1] https://github.com/privatenumber/minification-benchmarks
I think you have to look at this in the context of an entire bundle or project, and then you have to weigh it against the download speeds you’re generally expecting for the users of your site or app. I agree that as a blanket statement “gzip is enough” is not technically correct, but I think it’s largely correct in spirit, in that people tend to reach for minification by default, without really thinking about what th…
Minification would be indeed useless under that set of assumptions, but the real world is much more variable and you need a comfortable margin. For example, mobiles rarely have that large bandwidth sustained all the time.
Comprehensively speaking, the minification is only a small step in building a performant website or web application. You have way more things to do, for example choosing a correct image compression format and method would have much more impact in general. But not everyone can be expected to understand them in depth, so we have best practices. Doing the minification therefore qualifies as a good best practice, even though it would be just a single one out of many others.
Re: OpenAI is good at unminifying code
#150Earlier quoted context omitted.
> JS minification is fairly mechanical and comparably simple, so the inversion should be relatively easy. Just because a task is simple doesn't mean its inverse need be. Examples: - multiplication / prime factorization - deriving / integrating - remembering the past / predicting the future Code unobfuscation is clearly one of those difficult inverse problems, as it can be easily exacerbated by any of the following pr…
Of course, it is not generalizable! In my experience though, most minifiers do only the following: - Whitespace removal, which is trivially invertible. - Comment removal, which we never expect to recover via unminification. - Renaming to shorter names, which is tedious to track but still mechanical. And most minifiers have little understanding of underlying types anyway, so they are usually very conservative and rare…
I feel you’re downplaying the obfuscatory power of name-mangling. Reversing that (giving everything meaningful names) is surely a difficult problem?