Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

211–220 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#212

So why are we angry about that ? I mean the end result for the users are exactly the same, it matters only for bots. Google have such a (justified) bad reputation that whatever they do, people assume it’s entishification. I don’t believed it is on that matter.

Aside from the privacy concerns and inconveniences this brings to Google users, it matters for users (of services) that rely on SerpApi too. Kagi for example has started showing these goto URLs in search results.

Re: google.com/goto: Google's anti-scraping update

#213
post #31
post #4

If I hadn't already ditched Google long ago, this would be more than enough to chase me away.

They are forcing my further and further towards kagi. I do hope the fawning on here is at least partly justified.

I’m not fawning over it but it’s

- usually better than Google

- has actually useful features (smush BS listicle articles together, weight/block sites)

- if you don’t use any searches that month, they’ll just straight up refund the month subscription automatically

- doesn’t annoy me constantly

Re: google.com/goto: Google's anti-scraping update

#214
post #39
post #31

Earlier quoted context omitted.

They are forcing my further and further towards kagi. I do hope the fawning on here is at least partly justified.

Kagi is what I settled on a couple of years ago. I do think sometimes the fawning is over the top, but it is a solid search engine and tended to give me a bit better results than Google out of the box. The real win, though, is that you can give various sites a weight, so the search results will prefer or avoid sites according to your desires. Once I had that going, my search results tended to be much better than Goog…

Yep, this. I was delighted they implemented this simple yet powerful idea. I don't even remember when I have needed to go to a second page of search results in Kagi and spammy clone sites which copy stack exchange verbatim are banned into oblivion where they belong. Fuck up once doing that shit, and you are forever out of my search results.

And then there is the very handy assistant, that I can jump into using follow-up questions to the quick AI queries one can choose to use or not to use by appending a "?". Again a simple idea, which empowers the user.

Re: google.com/goto: Google's anti-scraping update

#215
post #89

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

> There are still illegal streaming sports and movie sites everywhere (who knew) and all other seedy corners of the internet that have been neatly erased by Google This is such a strange position to take. Apple doesnt allow all sorts of apps on the appstore, but that is never said as "Apple is erasing illegal streaming". Google is simply not showing the results. They are not taking down, or banning, or doing anything…

If you use an iphone then yes, apple is indeed "erasing" illegal streaming in the same sense that google is. We can be nostalgic for the old internet while also recognizing that at least google is well within their rights not to serve up results that fall outside the bounds of the law. (Apple not so much. When you gatekeep the hardware platform I think you're ethically obligated to act as a common carrier. Unfortunately the law doesn't require that.)

Re: google.com/goto: Google's anti-scraping update

#216
post #174

Earlier quoted context omitted.

I think they always knew what link you clicked on (when most of the web has a ga4 tag on the other side of the link)

You can block Google Analytics with your browser. You can't block needing to ask Google for the URL of the result you want to visit.

If you're that concerned, use another search engine

Re: google.com/goto: Google's anti-scraping update

#217

Can someone explain why this matters? Not being flippant I just don’t understand why this would be important.

It's to make it more difficult for scrapers. Google have been able to do all the tracking they've wanted for decades.

Think about it: if you want to set up something like jina.ai you need to build your own index or piggyback on Google. My guess is they use residential proxies to fire requests to Google then scrape the URLs.

Opaque URLs now give Google another gateway to detect circumvention of their anti-bot controls, helping them monetise their search index rather than allowing providers like jina.ai to succeed.

Re: google.com/goto: Google's anti-scraping update

#218
post #33

I've been using DuckDuckGo for years now, ever since it became noticeable that two different people searching for the same search term would get two different results back from Google. Meaning they were no longer completely reliable: they might show one person a result that they hide from the other person by burying it on page 3 where few people ever look. DDG's search results have been poorer recently than they used…

The age of internet search is over. The age of Cloudflare has begun. It wouldn't be possible to build a search engine now if they wanted to... and there wouldn't be anything to search for anyway. The non-corporate internet withered into dust and blew away in the wind. If you could find what you want, how would they ever sell you what they want you to buy? And I'm not just talking merchandise, though that too. Your po…

I think curated directories from prehistory might make a comeback eventually, maybe combined with a search engine that indexes only the whitelisted websites. ironically, we can use LLMs to filter out AI slop by measuring the signal-to-noise ratio, which is atrocious for AI slop and ESL slop that preceded it.

Re: google.com/goto: Google's anti-scraping update

#219

Earlier quoted context omitted.

> That wouldn’t work at all with this url format - the url can only be resolved by requesting the goto link. What is "that" you say wouldn't work? The extension can work fine by requesting every goto link upfront. As far as your google searches go, this is just as private as before. I would recommend you avoid judging a comment because of a metric it neither said nor implied.

Previously the clear target URL of a search result was added to the result-link (or div?) as an additional data-tag (aria or whatever, can't remember), and thus was able to be used with a userscript or similar to replace the Googleified tracking redirect link. And it was also part of the googliefied tracking URL, either way you were able to see the clear target URL. That no longer is possible, you must ask google.com…

But the extension doesn't need to use that exact mechanism, and the person that linked it didn't mention specific mechanisms. The purpose of the extension is putting the real URLs back, and it can still do that.

And it can still prevent google from knowing which search results you click on, even though OP didn't mention that feature.

Re: google.com/goto: Google's anti-scraping update

#220

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

[dead]
Post reply on HN