41–50 of 416 posts
Unrelated, but has anyone written a Chrome/Firefox extension to browse the web sending out Googlebot or Facebook user agent? I wonder if you can bypass paywalls or see things that aren't generally presented to regular users
Safari has this built in: https://osxdaily.com/2013/01/16/change-user-agent-chrome-saf...
Same thing has happened to me: https://twitter.com/moonscript/status/1124888489298808834 The network address range falls under Facebook's ownership, so I don't think it's someone spoofing. I do think it's very possible someone found a way to trigger crawl requests in large quantity. Alternatively, I would not be surprised it's just a bug on facebook's end.
They've done this before to me, too. First I tried `iptables -j DROP`, which made the machine somewhat usable, but didn't help with the traffic. After trying a few things, I tried `-j TARPIT`, and that appeared to make them back off. Of course, sample size of 1, etc. It could have been coincidental.
You can add artificial wait times to responses, or you can just route all of the 'bad' traffic to one machine, which becomes oversubscribed (be sure to segregate your stats!). All bad actors fighting over the same scraps creates proportional backpressure. Just adding 2 second delays to each request won't necessarily achieve that if multiple user agents are hitting you at once.
If you can verify it's actually coming from Facebook's IP address (as many other comments here suggest), you should absolutely get in touch with them, though I'm not sure how. Perhaps their security team. That's a very serious bug that they'd be glad to catch -- if it's affecting you it's surely affecting others. Otherwise it's a bot/malware/etc. spoofing Facebook and gone wrong, which sucks. And yeah just block it b…
Was it a real FB crawler or a random buggy homemade bot masquerading as one? Check here: https://developers.facebook.com/docs/sharing/webmasters/craw...
"who do people already like to hate? I'll pretend to be them."
Was it a real FB crawler or a random buggy homemade bot masquerading as one? Check here: https://developers.facebook.com/docs/sharing/webmasters/craw...
https://www.ultratools.com/tools/ipv6InfoResult?ipAddress=2a...
Maybe someone at Facebook is testing a project that didn't work as expected?
What I've heard happen is someone puts a governor on the system, and much later your new big customer starts having errors because your aggregate traffic exceeds capacity planning you did 2 years ago and forgot about.
It would be nice to understand how you came to the conclusion that this was a Facebook bot.
for example https://www.ultratools.com/tools/ipv6InfoResult?ipAddress=2a...
Did you check the ip and not just the UA? Does it matches with the one from fb?
This is actually a common tactic from malicious actors - pretend to be other bots to get through to websites. If you reverse lookup the IP address you can see if it part of the facebook network. UA is very untrustworthy
https://www.ultratools.com/tools/ipv6InfoResult?ipAddress=2a...