Live data from Hacker News

Facebook was used as a proxy by web scraping bots

datadome.co

81–90 of 125 posts

Re: Facebook was used as a proxy by web scraping bots

#81

I came from a third world country and internet was pretty expensive. For some reason, my provider made Facebook completely free. So in my free college days I used the Facebook developer echo API to make a HTTP proxy so I can browse internet for free. It was terrible, was only HTTP 1, so no web sockets, videos stopped randomly etc, but hey I could read Reddit.

What API is this? I am curious as to what this API was meant to be used for.

Re: Facebook was used as a proxy by web scraping bots

#82
post #80
post #77

Earlier quoted context omitted.

May I ask which country?

There are a lot of countries where Facebook is “free”, for example India and Philippines.

I'm guessing that in these countries the cost of Internet traffic is dominated by the undersea cables at their borders.

Facebook has been paying to have new undersea cables laid. This is done as part of a consortium, but those cables only have 6-12 strands in them (the repeaters are bulky) so owning even just one whole strand of fiber in an undersea cable is still an obscene amount of bandwidth for a single company that isn't in the business of reselling bandwidth.

Re: Facebook was used as a proxy by web scraping bots

#83

Earlier quoted context omitted.

Well, I think you're being overly alarmist, at least in the short term: DRM on Video has not really caught on, at least on-line; Non-Chrome browsers continue to have a significant share (mostly on Desktops); and ad blocking remains rather effective. In the long run, I'm definitely worried: Capitalist economies tend to see a concentration of capital, generally and in most sectors individually. And this seems to be a r…

> DRM on Video has not really caught on, at least on-line This seems like a weird statement. All of the paid streaming services use DRM on Video, so all major browsers include the requisite black-box DRM modules. I'm actually surprised YouTube has not added Widevine DRM for all videos yet, but I'm sure it'll happen if RIAA/MPAA get annoyed enough with youtube-dl and the like. > PS - By AMP, do you mean Amazon Prime?…

I've never heard of this AMP thing. Is it really that popular?

Valid point about paid streaming services - which I don't use.

Re: Facebook was used as a proxy by web scraping bots

#86
post #79

Earlier quoted context omitted.

> sources is a very hard problem We're talking about virtually unknown blogs that get 1 http request from my server's IP, which is not blacklisted anywhere. It's not hard at all , i just think cloudflare's tech s not that good

Well if you have an easy solution that you think would work, why don't you put up a website, commission a DDOS attack from a skilled actor and try to demonstrate mitigation? Companies pay big money to CloudFlare. If a simpler and cheaper solution is workable, they'll pay you instead.

Just like telling if it's raining is easy but stopping rain once has started is hard, the claim is that it's not hard to detect if a site is being ddosed.

Re: Facebook was used as a proxy by web scraping bots

#87
post #80

Earlier quoted context omitted.

There are a lot of countries where Facebook is “free”, for example India and Philippines.

I'm guessing that in these countries the cost of Internet traffic is dominated by the undersea cables at their borders. Facebook has been paying to have new undersea cables laid. This is done as part of a consortium, but those cables only have 6-12 strands in them (the repeaters are bulky) so owning even just one whole strand of fiber in an undersea cable is still an obscene amount of bandwidth for a single company t…

In The Philippines, my understanding is that they have ample bandwidth via Korea and other countries in the region. But the reason they have such expensive terrible internet is because of a lack of net neutrality and deregulation.

The cellphone duopoly sells "YouTube passes", that entitle you to get unthrottled YouTube for brief periods of time.

Re: Facebook was used as a proxy by web scraping bots

#88

Earlier quoted context omitted.

I'm guessing that in these countries the cost of Internet traffic is dominated by the undersea cables at their borders. Facebook has been paying to have new undersea cables laid. This is done as part of a consortium, but those cables only have 6-12 strands in them (the repeaters are bulky) so owning even just one whole strand of fiber in an undersea cable is still an obscene amount of bandwidth for a single company t…

In The Philippines, my understanding is that they have ample bandwidth via Korea and other countries in the region. But the reason they have such expensive terrible internet is because of a lack of net neutrality and deregulation. The cellphone duopoly sells "YouTube passes", that entitle you to get unthrottled YouTube for brief periods of time.

I wonder if a proxy could be made to encode data as video to put in a YouTube livestream. You'd still need an uplink but the upload bandwidth usage is a fraction of the download one for typical Internet usage.

Re: Facebook was used as a proxy by web scraping bots

#89
post #37

Earlier quoted context omitted.

I’ve long said Cloudflare is a dangerous threat to the open internet and as well as some privacy tools like TOR. But it doesn’t always get much traction on here because both the founder and employees of cloudflare are quite popular users on HN. Some have given me brief half assed counter answers that conveniently miss other harder questions like a good PR person does (and which you seem to have gotten in your reply).…

> But it doesn’t always get much traction on here because both the founder and employees of cloudflare are quite popular users on HN. I don't think it gets much traction because you're barking up the wrong tree. Also, suggesting that YC is out to silence you and that nobody actually has a counter argument isn't very good for traction, either. Until my website can't get taken off by a $5 rental of an internet-of-shit…

[deleted]

Re: Facebook was used as a proxy by web scraping bots

#90

I came from a third world country and internet was pretty expensive. For some reason, my provider made Facebook completely free. So in my free college days I used the Facebook developer echo API to make a HTTP proxy so I can browse internet for free. It was terrible, was only HTTP 1, so no web sockets, videos stopped randomly etc, but hey I could read Reddit.

This is exactly the reason I love reading this site
Post reply on HN