31–40 of 294 posts
Earlier quoted context omitted.
I don't see us getting a massive innovation in search on the internet now that Google has such a massive foothold, and companies like Cloudflare stop innovation from happening. How are we "stopping search innovation"?
For instance there is no way for distributed search engines to work with CloudFlare. No, "contact me and we'll help" is not always a solution.
Earlier quoted context omitted.
Crawling a Cloudflare powered website is basically impossible without needing to do some bodges as to how to crawl it. How can you expect someone to crawl a bunch of websites if they are actively blocked from accessing it? Now, you might say users can whitelist bots in their robots.txt file but then again will the person creating the engine individually ask companies to allow them to crawl? Also, slightly unrelated b…
If you are building a search engine and getting blocked you can always contact me and I'll make sure that the teams that work on bot detection and DDoS are aware. We would like to know because we should not be blocking a legit crawler like this.
Cloudflare is both a great thing and a terrible thing that has happened to the internet in recent years. Great in that they have a fantastic UI to add your site in, basically shielding the average user from attacks. Bad from a standpoint of that now only Google, Bing, and maybe other big search engines have the capabilities to actually crawl the internet now. I don't see us getting a massive innovation in search on t…
There was once cs professor who claimed that the internet does not scale around 2000. Either things choke up or you need huge investments into networks. It turns out that he was right, sort of. The vanilla attach server into the internet, server-to-client IP-network is pretty much dead. It has been replaced with CDN's , private delivery networks, cache on top of cache. Cloudfare, Amazon, Google and MS are the connect…
Earlier quoted context omitted.
https://news.ycombinator.com/item?id=27153635
This will scale wonderfully!
But let's suppose someone is building a new cool search engine and our ML stuff is blocking them. Then... contact us/me.
Earlier quoted context omitted.
Crawling a Cloudflare powered website is basically impossible without needing to do some bodges as to how to crawl it. How can you expect someone to crawl a bunch of websites if they are actively blocked from accessing it? Now, you might say users can whitelist bots in their robots.txt file but then again will the person creating the engine individually ask companies to allow them to crawl? Also, slightly unrelated b…
If you are building a search engine and getting blocked you can always contact me and I'll make sure that the teams that work on bot detection and DDoS are aware. We would like to know because we should not be blocking a legit crawler like this.
And what about crawlers with even more limited scopes (targeting only a handful of sites) that they can’t possibly be called search engines? Are they ever considered legit?
Earlier quoted context omitted.
I believe the idea here is you need to buy actual FIDO U2F keys and they could then be revoked on a per-key basis if you're caught abusing them as they're signed by a 3rd party so can't just be emulated. Meaning you need to buy more. Makes it expensive at least.
How can you revoke on a per-key basis without at the same time being able to track keys uniquely?
Edit: I was wrong. Cloudflare claims they could track people, but it would require tracking via cookies. [1] The hardware security keys have an "attestation key pair" that is shared among all units in one production batch (which contains at least 100K units). [2]
1: https://blog.cloudflare.com/introducing-cryptographic-attest...
2: https://www.w3.org/TR/webauthn-2/#sctn-attestation-privacy
Earlier quoted context omitted.
For instance there is no way for distributed search engines to work with CloudFlare. No, "contact me and we'll help" is not always a solution.
Please explain the problem (here or via email to me).