Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

351–360 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#351

Earlier quoted context omitted.

Do you have a source for this claim? Cloudflare gives me endless captchas on almost any website.

Yes, I bought a cheap residential proxy service and used it to scrape a site.

Ironic that I'd need something like that to stop getting blocked from a residential IP

Re: google.com/goto: Google's anti-scraping update

#352

Earlier quoted context omitted.

Didn't Google take it down because of copyright? With the rise of paywalls it became a backdoor around them, but Google doesn't actually have a license to republish the copyrighted text they scraped.

It's not a backdoor is the sites intentionally served Google a version without the paywall so the indexing got "better" than what real users got. The sites kind of dug their own hole here, Google wouldn't even sit on the content unless it was offered up to them in that way.

Unfortunately copyright law allows them to give something to one person without giving it to others _and_ to sue to the former person to block them from giving it to the others.

Re: google.com/goto: Google's anti-scraping update

#354

So why are we angry about that ? I mean the end result for the users are exactly the same, it matters only for bots. Google have such a (justified) bad reputation that whatever they do, people assume it’s entishification. I don’t believed it is on that matter.

Aside from the list of current concerns that others have covered well...Google is very good at "boiling frogs". That is, rolling out unpopular things in phases.

This could be, for example, step one in the return of AMP, but with a new twist. Where google conveniently returns the content of your website, without directly sending the user to your website. With whatever changes it chooses to make.

Re: google.com/goto: Google's anti-scraping update

#355

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

> For actual web search I actually like using Yandex You might want to reconsider this. https://meduza.io/image/attachments/images/007/699/276/large...

You think Google doesn't do censorship? Or any mainstream search engine, for that matter?

It's the usual "I prefer censorship not targeted at/tailor made for me".

Re: google.com/goto: Google's anti-scraping update

#356

Earlier quoted context omitted.

I switched to DuckDuckGo around 2018 or so, which was when I realized that Google's results quality has deteriorated so much that I won't lose anything of value by doing so. This thread is how I learn about the atrocities that Google Search is committing these days. Which is an elaborate way to say: you don't have to live like this.

I don't. At this point I just ask the oracle in natural language. There's so many to choose from. Why spend hours wading through only vaguely related things? You can ask it for a direct answer or you can ask it to teach you about the topic.

Sometimes there is joy in discovering things for yourself. Also, the oracle currently relies on humans publishing new information, what will you do when the incentive is gone and the oracle has nothing to harvest?

Re: google.com/goto: Google's anti-scraping update

#357

I really like the idea of turning the search index into a public utility. It is one of those natural monopoly coordination problem things. Just quasi nationalise it for economic efficiency. Ofc google can still sell adds against their own ui (like everyone else). Hopefully this move will move that idea closer to reality.

With the current administration? They would manipulate the results to ridiculous extent

So would the previous and the next one, this is not situational.

Re: google.com/goto: Google's anti-scraping update

#358

Earlier quoted context omitted.

For Google, the business side of you paying them doesn't work out as obvious as it might seem. Advertisers are sold the idea that 1000 clicks/impression is X dollars. The average impression might be priced at 1000/X$. But you are worth much more than that for two reasons. - you are a person willing to pay 5$ - Google can sell a “collectivized” product. The same way that eg collective farmer crop insurance creates act…

The advert paradox, anyone willing to spend money to avoid ads is inherently worth more to advertisers, since they're willing to spend money. The more money you're willing to pay to avoid ads the more you're worth to advertisers. That's why tools like uBlock and YouTube morphe are the only answer, negotiating with terrorists never works.

I don't understand your logic. Why would someone that never sees my ads be valuable to me? Why would I care? Everyone spends money on something, the fact you spend it on adblockers doesn't make you automatically more valuable to advertisers. I have never heard of this "advert paradox" concept you mention and I have worked in industry for a while now. All this is just conjecture on your part and doesn't really correspond to how it works.

Re: google.com/goto: Google's anti-scraping update

#359

Earlier quoted context omitted.

> For actual web search I actually like using Yandex You might want to reconsider this. https://meduza.io/image/attachments/images/007/699/276/large...

You think Google doesn't do censorship? Or any mainstream search engine, for that matter? It's the usual "I prefer censorship not targeted at/tailor made for me".

Bold to assume Russian propaganda is not aimed at the west.

Re: google.com/goto: Google's anti-scraping update

#360
What are you going to steal from Google?

The Internet has been dead for years and, after the Scrapocalypse, the small living remnants are behind a login wall.

Google can not provide you anything you couldn't find on either your local ZIM archive or the Media you consume.

Post reply on HN