Facebook was used as a proxy by web scraping bots

#1

Post by avastel » Thu, Nov 05, 2020, 6:15 PM UTC

Facebook was used as a proxy by web scraping bots
datadome.co

Re: Facebook was used as a proxy by web scraping bots

#2

Post by AznHisoka » Thu, Nov 05, 2020, 6:33 PM UTC

How is DataDome different from Cloudflare? The latter offers bot protection for free if you are already a Cloudflare customer

Re: Facebook was used as a proxy by web scraping bots

#3

Post by iampims » Thu, Nov 05, 2020, 6:38 PM UTC

AznHisoka wrote: post #2

How is DataDome different from Cloudflare? The latter offers bot protection for free if you are already a Cloudflare customer

They’re a French/EU company, that alone can be an advantage for some businesses.

Re: Facebook was used as a proxy by web scraping bots

#4

Post by ericcholis » Thu, Nov 05, 2020, 6:38 PM UTC

AznHisoka wrote: post #2

How is DataDome different from Cloudflare? The latter offers bot protection for free if you are already a Cloudflare customer

My understanding is that it's more "advanced". Take that at face value, I've not used the service.

Re: Facebook was used as a proxy by web scraping bots

#5

Post by kevin1985 » Thu, Nov 05, 2020, 6:41 PM UTC

iampims wrote: post #3

AznHisoka wrote: post #2

How is DataDome different from Cloudflare? The latter offers bot protection for free if you are already a Cloudflare customer

They’re a French/EU company, that alone can be an advantage for some businesses.

Pretty sure they have an office in New York as well. Site shows POPs all over the map: datadome.co

Re: Facebook was used as a proxy by web scraping bots

#6

Post by martimarkov » Thu, Nov 05, 2020, 6:48 PM UTC

The website doesn’t open for me.

Re: Facebook was used as a proxy by web scraping bots

#7

Post by cblconfederate » Thu, Nov 05, 2020, 6:49 PM UTC

You can't create your own link previewer, cloudflare will put a captcha in front of every website. All I want is a a freaking tag. They don't seem eager to fix it either, their proposed solution is to contact every website owner (seriously) to ask them to whitelist you[1].

Frankly, i wish facebook or cloudflare offered their previewer as a free service, since most websites have them whitelisted.

1. https://community.cloudflare.com/t/attention-required-messag...

Re: Facebook was used as a proxy by web scraping bots

#8

Post by tyingq » Thu, Nov 05, 2020, 6:51 PM UTC

That's pretty interesting, Facebook as a "web scale / hundreds of pages per second" batch web page summarizer. I imagine you could build a pretty decent general purpose search engine that way...free crawler.

Re: Facebook was used as a proxy by web scraping bots

#9

Post by MDinBk » Thu, Nov 05, 2020, 6:52 PM UTC

interesting!

Re: Facebook was used as a proxy by web scraping bots

#10

Post by llacb47 » Thu, Nov 05, 2020, 7:20 PM UTC

martimarkov wrote: post #6

The website doesn’t open for me.

Check your DNS, if you are running a pihole or something similar you will need to disable or allow this site.