So I went Slashdot, Digg, Reddit. I stopped spending any time on Reddit 5 years ago. Not worth it.
Google is the only search engine that works on Reddit now, thanks to AI deal
281–290 of 379 posts
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#282It's weird to say that reddit "works" with google. Every page they serve to google is stuffed full of hidden unrelated content, so any reddit result in google is unlikely to actually contain what you were searching for.
Google really should blacklist reddit entirely for this practice, but sadly as bad as reddit is it's still a much higher quality result than average for google.
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#283We need laws that make it so that giant platforms like Reddit have no exclusive rights to content submitted by users. It would be ridiculous for only Google to be able to train AI on YouTube or Reddit content for example.
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#284Earlier quoted context omitted.
They changed it to disallow so that scrapers can't just claim the robots.txt gave them permission.
Independent scrapers can launder the data between Reddit and AI consumers. The only folks this hurts is users seeking info via search engines and folks willing to kowtow to rules that are potentially low impact to evade. Next steps would be (from an adversarial perspective) browser extensions that stream back data for ingestion similar to Recap for Pacer [1]. [1] https://free.law/recap/faq (full disclosure: assisting…
Scrapings laundering. Do we have a term for this?
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#285Earlier quoted context omitted.
Up until 2016 (I think, +/- 1 year), if you could remember 3 uncommon words in a comment, you could find any reddit post instantly on Google. I'd want to follow up on a thread from weeks ago, and it was magic. Number one result. Then one day that just stopped working, and even adding site:*.reddit.com didn't fix it. At the time, I think, I didn't realize that it was mostly Google's fault, I thought maybe Reddit had c…
it's so bad it's crazy, you can legit not find stuff on the internet anymore, it's the same with youtube, I search something and get like 20 or so results and then everything else is hidden. it started when youtube removed the ability to search for videos older than 5 years, if I had to guess? cost saving, have every old video in cheaper storage... but it sort of fragments youtube, every couple of years you only get…
One day I was searching live videos of a local band from youtube and when sorting by upload date the oldest video was from 2010. I knew for a fact there had to be older videos so I got a youtube API key and searched via the API, ended up finding multiple videos starting from 2006. Learned that youtube is full of videos that are basically impossible to find with the regular search.
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#286Earlier quoted context omitted.
it's even worse. it's not theirs (it's the users'), they are merely hosting it and using it (ToS gives them a fancy irrevocable license I guess). so they can do whatever they want with it and the actual owners/authors have no chance to really influence Reddit at all to make it crawlable. (the GDPR-like data takeout is nice, but ... completely useless in these cases where the value is in the composition and aggregatio…
> the GDPR like data takeout is nice Is there a way to export my history? How?
I tried it many months back when they glitch-killed my decade-plus account. (Yes, I'm still bitter over the kafkaesque injustice.)
Anyway, you basically submit a request and then later they will email you a link to a zip file that contains a free dozen CSV files with unescaped newlines. One for all the comments you made, one for up/down-votes, one for blocked users, etc.
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#287Earlier quoted context omitted.
> the GDPR like data takeout is nice Is there a way to export my history? How?
https://www.reddit.com/settings/data-request (and there's some help article for it that I didn't read, but google found this first https://support.reddithelp.com/hc/en-us/articles/36004304835... that's how I got to the link)
Thank you, it worked
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#288Earlier quoted context omitted.
> There is nothing preventing search companies paying the same $60 Million to license content. Yes, actually, there is - having $60m to throw around. "Barriers to entry often cause or aid the existence of monopolies and oligopolies" [0]. Monopolies and oligopolies are definitionally the opposite of free market forces. This is quite literally Econ 101. [0] - https://en.wikipedia.org/wiki/Barriers_to_entry
Microsoft can throw around $60M, and Bing is used by most of the "alternative" search engines. It doesn't solve the problem, but if money is the only thing preventing search engines from accessing Reddit, then what goes for Google also goes for Microsoft.
Cool, so the top two players are now the only players and everyone else is now trapped feeding off of them
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#289Earlier quoted context omitted.
it's so bad it's crazy, you can legit not find stuff on the internet anymore, it's the same with youtube, I search something and get like 20 or so results and then everything else is hidden. it started when youtube removed the ability to search for videos older than 5 years, if I had to guess? cost saving, have every old video in cheaper storage... but it sort of fragments youtube, every couple of years you only get…
One day I was searching live videos of a local band from youtube and when sorting by upload date the oldest video was from 2010. I knew for a fact there had to be older videos so I got a youtube API key and searched via the API, ended up finding multiple videos starting from 2006. Learned that youtube is full of videos that are basically impossible to find with the regular search.
One of my past times is looking up anime opening reactions for fun/to hear people listen to bands I enjoy that sometimes do anime ops, searching that is so scary, you see the same 4 people or every month or so you get 1 or get 1 removed, you can't tell me there's not more than 4 people on the internet that upload opening reactions... when anime is a billion+ dollar industry, if you know particular channels you can see daily uploads on plenty of channels, that are simply search banned for whatever reason by the algorithm.
But yeah, the most outrageous one is older videos, I do believe the reason is that they are using some long term cloud storage that is cheaper for older videos so they removed the ability to search by date.
Additionally, I don't believe the API fully fixes it, because Bing has a wrapper for youtube and searches do not really vary
Re: Google is the only search engine that works on Reddit now, thanks to AI deal
#290They changed robots.txt a month or so ago. For the first 19 years of life, reddit had a very permissive robots.txt. We allowed all by default and then only restricted certain poorly behaved agents (and Bender's Shiny Metal Ass(tm)) But I can understand why they made the change they did. The data was being abused. My guess is that this was an oversight -- that they will do an audit and reopen it for search engines aft…
how was it being abused. You still clicked on the information and saw the reddit ads? Now they won't get any of that from "rival" sites to google. I guess they figured the 60 million was more than that ad revenue. Seems greedy but I don't think it's illegal like others are suggesting.