I came from a third world country and internet was pretty expensive. For some reason, my provider made Facebook completely free. So in my free college days I used the Facebook developer echo API to make a HTTP proxy so I can browse internet for free. It was terrible, was only HTTP 1, so no web sockets, videos stopped randomly etc, but hey I could read Reddit.
Facebook was used as a proxy by web scraping bots
81–90 of 125 posts
Re: Facebook was used as a proxy by web scraping bots
#82Earlier quoted context omitted.
May I ask which country?
There are a lot of countries where Facebook is “free”, for example India and Philippines.
Facebook has been paying to have new undersea cables laid. This is done as part of a consortium, but those cables only have 6-12 strands in them (the repeaters are bulky) so owning even just one whole strand of fiber in an undersea cable is still an obscene amount of bandwidth for a single company that isn't in the business of reselling bandwidth.
Re: Facebook was used as a proxy by web scraping bots
#83Earlier quoted context omitted.
Well, I think you're being overly alarmist, at least in the short term: DRM on Video has not really caught on, at least on-line; Non-Chrome browsers continue to have a significant share (mostly on Desktops); and ad blocking remains rather effective. In the long run, I'm definitely worried: Capitalist economies tend to see a concentration of capital, generally and in most sectors individually. And this seems to be a r…
> DRM on Video has not really caught on, at least on-line This seems like a weird statement. All of the paid streaming services use DRM on Video, so all major browsers include the requisite black-box DRM modules. I'm actually surprised YouTube has not added Widevine DRM for all videos yet, but I'm sure it'll happen if RIAA/MPAA get annoyed enough with youtube-dl and the like. > PS - By AMP, do you mean Amazon Prime?…
Valid point about paid streaming services - which I don't use.
Re: Facebook was used as a proxy by web scraping bots
#84Re: Facebook was used as a proxy by web scraping bots
#85So this was a very long web page to say: Facebook forgot to rate limit their web scraper on a per user basis, but we told them and they fixed it.
Re: Facebook was used as a proxy by web scraping bots
#86Earlier quoted context omitted.
> sources is a very hard problem We're talking about virtually unknown blogs that get 1 http request from my server's IP, which is not blacklisted anywhere. It's not hard at all , i just think cloudflare's tech s not that good
Well if you have an easy solution that you think would work, why don't you put up a website, commission a DDOS attack from a skilled actor and try to demonstrate mitigation? Companies pay big money to CloudFlare. If a simpler and cheaper solution is workable, they'll pay you instead.
Re: Facebook was used as a proxy by web scraping bots
#87Earlier quoted context omitted.
There are a lot of countries where Facebook is “free”, for example India and Philippines.
I'm guessing that in these countries the cost of Internet traffic is dominated by the undersea cables at their borders. Facebook has been paying to have new undersea cables laid. This is done as part of a consortium, but those cables only have 6-12 strands in them (the repeaters are bulky) so owning even just one whole strand of fiber in an undersea cable is still an obscene amount of bandwidth for a single company t…
The cellphone duopoly sells "YouTube passes", that entitle you to get unthrottled YouTube for brief periods of time.
Re: Facebook was used as a proxy by web scraping bots
#88Earlier quoted context omitted.
I'm guessing that in these countries the cost of Internet traffic is dominated by the undersea cables at their borders. Facebook has been paying to have new undersea cables laid. This is done as part of a consortium, but those cables only have 6-12 strands in them (the repeaters are bulky) so owning even just one whole strand of fiber in an undersea cable is still an obscene amount of bandwidth for a single company t…
In The Philippines, my understanding is that they have ample bandwidth via Korea and other countries in the region. But the reason they have such expensive terrible internet is because of a lack of net neutrality and deregulation. The cellphone duopoly sells "YouTube passes", that entitle you to get unthrottled YouTube for brief periods of time.
Re: Facebook was used as a proxy by web scraping bots
#89Earlier quoted context omitted.
I’ve long said Cloudflare is a dangerous threat to the open internet and as well as some privacy tools like TOR. But it doesn’t always get much traction on here because both the founder and employees of cloudflare are quite popular users on HN. Some have given me brief half assed counter answers that conveniently miss other harder questions like a good PR person does (and which you seem to have gotten in your reply).…
> But it doesn’t always get much traction on here because both the founder and employees of cloudflare are quite popular users on HN. I don't think it gets much traction because you're barking up the wrong tree. Also, suggesting that YC is out to silence you and that nobody actually has a counter argument isn't very good for traction, either. Until my website can't get taken off by a $5 rental of an internet-of-shit…
Re: Facebook was used as a proxy by web scraping bots
#90I came from a third world country and internet was pretty expensive. For some reason, my provider made Facebook completely free. So in my free college days I used the Facebook developer echo API to make a HTTP proxy so I can browse internet for free. It was terrible, was only HTTP 1, so no web sockets, videos stopped randomly etc, but hey I could read Reddit.