Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse engineering the obfuscated TikTok VM

github.com

Post reply on HN
Print view View original

121–130 of 131 posts

Re: Reverse engineering the obfuscated TikTok VM

#121

Post by Thorrez »

lazyeye wrote: post #107
fidotron wrote: post #40

Earlier quoted context omitted.

If you believe this you underestimate how adversarial the software world really is. TikTok will be on the receiving end of botnets by everything from commercial entities, state backed groups and criminals. They won't be betting that this stops that entirely, but it adds a layer of friction that is easy for them to change on a continuous basis. These things are also very good for leaving honeypots in where if someone…

Nah..I agree with the parent comment, there is simply no legitimate reason for a social media app to employ this level of obsfucation.

If you ran a social media site and app, and had a problem of many different groups employing bots to post tons of content for nefarious purposes to your site, what would you do?

Re: Reverse engineering the obfuscated TikTok VM

#122

Post by dns_snek »

wpietri wrote: post #67
dns_snek wrote: post #36

Earlier quoted context omitted.

The balance of evidence suggests otherwise. If they cared about spam bots they would take action when spammers are handed to them on a silver platter. The kinds of spammers who will leave 30 identical comments advertising illegal services, not some weird moderation corner case. If you ever end up on a video that's related to drugs, there will be entire chains of bots just advertising to each other and TikTok won't fi…

A large company is much less cohesive than you realize. You can't reliably reason about the goals of one part because another part isn't consistent. This particular difference could easily be explained by insufficient funding to moderation, which is endemic in social media.

I've said this twice already, it's not that another part "isn't consistent" (I would agree that this is to be expected), they're CONSISTENTLY acting in the opposite manner than is being speculated here and I subscribe to the "purpose of a system is what it does" world view.

Re: Reverse engineering the obfuscated TikTok VM

#123

Post by lazyeye »

Thorrez wrote: post #121
lazyeye wrote: post #107

Earlier quoted context omitted.

Nah..I agree with the parent comment, there is simply no legitimate reason for a social media app to employ this level of obsfucation.

If you ran a social media site and app, and had a problem of many different groups employing bots to post tons of content for nefarious purposes to your site, what would you do?

I guess Id probably be doing something similar to what all the other social media apps are doing (unless of course, I had something to hide...)

Re: Reverse engineering the obfuscated TikTok VM

#124

Post by wpietri »

dns_snek wrote: post #122
wpietri wrote: post #67

Earlier quoted context omitted.

A large company is much less cohesive than you realize. You can't reliably reason about the goals of one part because another part isn't consistent. This particular difference could easily be explained by insufficient funding to moderation, which is endemic in social media.

I've said this twice already, it's not that another part "isn't consistent" (I would agree that this is to be expected), they're CONSISTENTLY acting in the opposite manner than is being speculated here and I subscribe to the "purpose of a system is what it does" world view.

If you really subscribed to POSIWID, you wouldn't be making arguments like "That's probably not the goal", as that's nonsensical from the POSIWID perspective.

The nominal goal of the code could well be bots at the same time the POSIWID purpose is about the exec impressing his superiors and the developers feeling smart and indulging their pet technical interests. Similarly, the nominal goal of the abuse reporting system would include spam, even if the POSIWID analysis would show that the true current purpose is to say they're doing something while keeping costs low.

So again, I don't think you have a lot of understanding of how large companies work. Whereas I, among other things, ran an anti-abuse engineering team at Twitter back in the day, so I'm reasonably familiar with the dynamics.

Re: Reverse engineering the obfuscated TikTok VM

#125

Post by mike_hearn »

godelski wrote: post #115
rfoo wrote: post #29

Earlier quoted context omitted.

Google has been doing this since forever for recaptcha. And, to be fair, it seems to be fairly effectively for bot detection. https://github.com/neuroradiology/InsideReCaptcha > bots seem to be going a very different route If the "very different route" means running a headless browser, then it's a success for this tech. Because the bot must run a blackbox JS now, and this gives people a whole new street of ways to ru…

Okay... but those bots exist... and in high numbers... By "very different route" I mean "measure to effectively stop the bots" (or dramatically reduce). It seems like if they're using a headless browser then they're still being quite effective in accomplishing their goals.

Google's obfuscating VM based anti-bot system (BotGuard) was very effective. Source: I wrote it. We used it to completely wipe out numerous botnets that were abusing Google's products e.g. posting spam, clickfraud, phishing campaigns. BotGuard is still deployed on basically every Google product and they later did similar systems for Android and iOS, so I guess it continues to work well.

AFAIK Google was the first to use VM based obfuscation in JavaScript. Nobody was using this technique at the time for anti-spam so I was inspired primarily by the work Nate Lawson did on BluRay.

What most people didn't realize back then is that if you can force your adversary to run a full blown web browser there are numerous tricks to detect that the browser is being automated. When BotGuard was new most of those tricks were specific to Internet Explorer, none were already known (I had to discover them myself) and I never found any evidence that any of them were rediscovered outside of Google. The original bag of tricks is obsolete now of course, nobody is using Internet Explorer anymore. I don't know what it does these days.

The VM isn't merely about protecting the tricks, though. That's useful but not the main reason for it. The main reason is to make it easier to generate random encrypted programs for the VM, and thus harder to write a static analysis. If you can't write a static analysis for the program supplied by your adversary you're forced to actually execute it and therefore can't write a "safe" bot. If the program changes in ways that are designed to detect your bot, done well there's no good way to detect this and bring the botnet to a safe halt because you don't know what the program is actually doing at the semantic level. Therefore the generated programs can detect your bot and then report back to the server what it found, triggering delayed IP/account/phone number bans. It's very expensive for abusers to go through these bans but because they have to blindly execute the generated programs they can't easily reduce the risk. Once the profit margin shrinks below the margin from abusing a different website, they leave and you win.

Re: Reverse engineering the obfuscated TikTok VM

#126

Post by hoseja »

Scaevolus wrote: post #100
like_any_other wrote: post #91

Earlier quoted context omitted.

Unless the scrapers are DDoSing the site, I refuse to consider the downloading of publicly posted data as malicious. It shows how captured the conversation has become by corporate interests, that viewing or storing data posted free of charge, publicly, by their users, in a way not approved by that corporation, is seen as malicious, and the only morally allowed way to view it is to use their spyware-laden client.

What if the user has disabled downloads of a video? Should the creator (and copyright owner) of a piece of media not be allowed even token attempts to prevent copying?

Here's a great way to prevent people copying your precious video: don't post in on the internet.

Re: Reverse engineering the obfuscated TikTok VM

#127

Post by esseph »

supriyo-biswas wrote: post #50
esseph wrote: post #48

Earlier quoted context omitted.

If you explain what you disagree with, maybe people would learn something. If you joint "their ranks", you've simply gone down the bit of the "social media + expertise" bell curve, where the more expertise you get on a particular topic, the less you want to engage about it in public. This is not unique to any one field or realm of knowledge.

This is all correct, just that I'm lamenting the decline of technical discussions over uninformed positions and polarizing diatribes at a place I loved to come to be informed. /soapbox

You're the one doing the informing on this topic, congrats, you've forum-peaked.

Re: Reverse engineering the obfuscated TikTok VM

#128

Post by Thorrez »

lazyeye wrote: post #123
Thorrez wrote: post #121

Earlier quoted context omitted.

If you ran a social media site and app, and had a problem of many different groups employing bots to post tons of content for nefarious purposes to your site, what would you do?

I guess Id probably be doing something similar to what all the other social media apps are doing (unless of course, I had something to hide...)

What are the other social media apps doing? Are you sure they're not using obfuscated VMs as well?

I'm guessing a lot of them use reCAPTCHA, and according to this comment, reCAPTCHA uses an obfuscated VM:

https://news.ycombinator.com/item?id=43748994

Re: Reverse engineering the obfuscated TikTok VM

#129

Post by rfoo »

ctoth wrote: post #101
poincaredisk wrote: post #63

Earlier quoted context omitted.

I'm surprised by this. As a professional reverse engineering I've actually found LLMs to be terrible at deobfuscation of JS (especially in the context of JS malware). But maybe my requirements are higher and it's actually OK for occasional use against weak packers?

Have you seen this? https://github.com/jehna/humanify What they do is ground the LLM to the AST with Babel to ensure you still get the same shape of AST out of your deobfuscation pass. Probably this tool could be cleaned up, made to work with multiple llm and parser backends, have its prompts improved, &c.

This is great idea! But it's more about having LLMs to give function & variables names, instead of having LLM to deobfuscate. The (traditional) deobfuscations (e.g. unpack, de-flatten, de-virtualization etc) were done by 100% precise human made Babel plugins and is totally unrelated to a LLM.

Re: Reverse engineering the obfuscated TikTok VM

#130

Post by lazyeye »

Thorrez wrote: post #128
lazyeye wrote: post #123

Earlier quoted context omitted.

I guess Id probably be doing something similar to what all the other social media apps are doing (unless of course, I had something to hide...)

What are the other social media apps doing? Are you sure they're not using obfuscated VMs as well? I'm guessing a lot of them use reCAPTCHA, and according to this comment, reCAPTCHA uses an obfuscated VM: https://news.ycombinator.com/item?id=43748994

Yep I'd probably go with a reCaptcha like everybody else except TikTok then.
Post reply on HN