61–70 of 379 posts
Suppose a crawler or rival search engine doesn’t respect robots.txt, reddit can’t stop them. Make it a bit trickier, yes, but not stop them.
It is evidence that they didn't have permission if you sue them.
Hopefully this paves the way for antitrust action, but I won't hold my breath. Reddit's justification for this is profoundly wrong. Their "public content policy" is absurd doublespeak, and counter to everything the open internet is and hopes to be. You cannot simultaneously call yourself "open" and "public" while refusing access to automated clients. Every client is automated. They even go so far as to say that "craw…
Or are you somehow suggesting that it’s google’s fault that Reddit took this step? I don’t see any indication that’s the case.
Earlier quoted context omitted.
I agree with you in theory, but in practice someone has to pay for all this magic.
We did. As in we, the Internet, existed for a long time without anyone making money and we paid for the privilege. Websites were built and hosted at owner's expense, for years, with no expectation that they be financially rewarded. Sure some would run donation drives, or work with sponsors relevant to the community in question, but a whole ton, mine included, just cost me a lot of money over many years. Those website…
This is great. It means I won't see Reddit content popping up all over search results in other engines. Can Medium do the same? And perhaps Quora?
site:reddit.com works for kagi for new posts this week?
Basically all 'independent' search engines piggyback off Google or Bing https://help.kagi.com/kagi/search-details/search-sources.htm... >Our search results also include anonymized API calls to all major search result providers worldwide
Incorrect: https://www.mojeek.com/about/why-mojeek
Reddit has been ripe for disruption for years. It's just waiting on an inflection point and someone to take it behind the barn.
Does this mean there will be a future where everyone is running their own crawler? I suppose.
Suppose a crawler or rival search engine doesn’t respect robots.txt, reddit can’t stop them. Make it a bit trickier, yes, but not stop them.
This is great. It means I won't see Reddit content popping up all over search results in other engines. Can Medium do the same? And perhaps Quora?
I deeply regret every minute spent on and kilobyte of text contributed to reddit.