Live data from Hacker News

Google is the only search engine that works on Reddit now, thanks to AI deal

404media.co

291–300 of 379 posts

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#291

Earlier quoted context omitted.

[flagged]

No, because people don't come to Lemmy to read AI-generated posts. It's called social media, not robot time with language models.

there are plenty of reddits outside of the "popular ones" that have invaluable information in one place and I can ask questions. It still has value for me, and not much value for axis of evil bots to spread political disinformation like popular subs and on twitter.

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#292
post #219

Earlier quoted context omitted.

Quite sure they are also enforcing these with some technical measures to limit scraping.

As was LinkedIn, who was forced to rate stop limiting / IP-banning scrapers for public pages.

Really? That seems strange.

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#293

Earlier quoted context omitted.

Independent scrapers can launder the data between Reddit and AI consumers. The only folks this hurts is users seeking info via search engines and folks willing to kowtow to rules that are potentially low impact to evade. Next steps would be (from an adversarial perspective) browser extensions that stream back data for ingestion similar to Recap for Pacer [1]. [1] https://free.law/recap/faq (full disclosure: assisting…

Scrapings laundering. Do we have a term for this?

Yes, right in the law - "fair use"

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#294
post #157
post #148

This problem is only going to get worse. for my thegreatestbooks.org site i used to just get indexed/scraped by google and bing. now it's like 50+ AI bots scraping my entire site just so they can train a LLM to answer questions my site answers without having a user ever visit my site. I just checked cloudflare and in the past 24 hours I've had 1.2 million bot/automated requests

There's a new setting in Cloudflare to block AI/scraper bots. https://blog.cloudflare.com/declaring-your-aindependence-blo...

Anyone have any experience with this? Is there nothing but upside in blocking these bots

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#295

Earlier quoted context omitted.

Kagi uses at least Google and Mojeek edit: > Realistically, there are only 2 search engines now. https://seirdy.one/posts/2021/03/10/search-engines-with-own-...

> Realistically, there are only 2 search engines now. From the article: Many alternatives to GBY [Google, Bing, and Yandex] exist, but almost none of them have their own results; This seems to assert that ~0 other search providers do any crawling at all. Ever. Are we sure that's the case? (they could crawl but never ever return those results == more odd).

Aside: Does anyone know how the GBY term became a thing and why it includes Yandex but not Baidu?

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#296

Earlier quoted context omitted.

> There is nothing preventing search companies paying the same $60 Million to license content. Yes, actually, there is - having $60m to throw around. "Barriers to entry often cause or aid the existence of monopolies and oligopolies" [0]. Monopolies and oligopolies are definitionally the opposite of free market forces. This is quite literally Econ 101. [0] - https://en.wikipedia.org/wiki/Barriers_to_entry

And yet "free market forces" are often the reason why monopolies and oligopolies arise...? Monopolies are entirely consistent with free market economics. After all, if there's clearly a "best product" for a particular niche, it's entirely rational (free market actor) behavior for everyone to use the same product, leading to its monopoly in that market segment. I don't understand why people think this isn't/won't be/s…

> Monopolies are entirely consistent with free market economics

Not in the least. Literally in the first semester, Economics 101 type class that any business/economics/etc student would take, it would be covered clearly that monopolies are violations of free markets.

A free market isn't a euphemism for anarchy or "no rules", it's a specific economic term. The things it is free of include artificial price floors or ceilings, barriers to entry, anti-competitive practices, etc. In other words, monopolies, oligopolies, cartels, monopsonies, etc are all violations of a free market. You do not have a free market if there is a monopoly supplier.

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#297
post #5

# Welcome to Reddit's robots.txt # Reddit believes in an open internet, but not the misuse of public content. # See https://support.reddithelp.com/hc/en-us/articles/26410290525844-Public-Content-Policy Reddit's Public Content Policy for access and use restrictions to Reddit content. # See https://www.reddit.com/r/reddit4researchers/ for details on how Reddit continues to support research and non-commercial use. # pol…

> We believe in something that we will now proceed to violate.

I will never take a statement given by a company that blatantly lies like this at face value going forward. What a bunch of clowns.

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#298

This is an interesting development. How many other sites might have leverage to charge to be indexed? I don't want to live in a world where you have to use X search engine to get answers from Y site - but this seems like the beginning of that world. From an efficiency perspective - it's obviously better for websites to just lease their data to search engines then both sides paying tons of bandwidth and compute to get…

> but possibly could lead the old, cool, hobbiest & un-monetized web being reinvented?

It still exists. It just isn't that popular.

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#299

Earlier quoted context omitted.

Having a Monopoly != Anti-Competitive. Having Barriers to Entry != Anti-competitive Yes, large players have advantages of Economies of Scale. Just because you can't run an Airline because you don't have money to buy an Airplane isn't anti-competitive. Today Microsoft, Apple, OpenAI, Google, Amazon all can afford those piddly $60m to license from reddit. Not Anti-competitive at all. But saddened by how much corporate-…

If you check citations, you'd find the sentence preceding my excerpt on barriers to entry: "Because barriers to entry protect incumbent firms and restrict competition in a market, they can contribute to distortionary prices and are therefore most important when discussing antitrust policy." Antitrust policy then links to a page on competition law: "Competition law is the field of law that promotes or seeks to maintai…

I took care of the down vote for you. I dunno when the privilege is earned, but keep up the quality comments and you’ll be able to downvote incurious ideologues like this in no time!

Re: Google is the only search engine that works on Reddit now, thanks to AI deal

#300

Earlier quoted context omitted.

If you check citations, you'd find the sentence preceding my excerpt on barriers to entry: "Because barriers to entry protect incumbent firms and restrict competition in a market, they can contribute to distortionary prices and are therefore most important when discussing antitrust policy." Antitrust policy then links to a page on competition law: "Competition law is the field of law that promotes or seeks to maintai…

Once again buying an Airplane and starting an Airline business has probably the highest barrier to entry. Yet the Airline industry is the most competitive.

Hm. Chip fabrication?
Post reply on HN