Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

Post reply on HN
Print view View original

461–470 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#461

Post by celsoazevedo »

nine_k wrote: post #110
lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…

Kagi seems to heavily rely on Yandex for results. I don't use it, so can't compare, but are results much better than using Yandex directly?

Re: google.com/goto: Google's anti-scraping update

#462

Post by joeblubaugh »

Dylan16807 wrote: post #219
hurfdurf wrote: post #173

Earlier quoted context omitted.

Previously the clear target URL of a search result was added to the result-link (or div?) as an additional data-tag (aria or whatever, can't remember), and thus was able to be used with a userscript or similar to replace the Googleified tracking redirect link. And it was also part of the googliefied tracking URL, either way you were able to see the clear target URL. That no longer is possible, you must ask google.com…

But the extension doesn't need to use that exact mechanism, and the person that linked it didn't mention specific mechanisms. The purpose of the extension is putting the real URLs back, and it can still do that. And it can still prevent google from knowing which search results you click on, even though OP didn't mention that feature.

You can read the extension source, and description. It is obvious that what it does is strip url parameters and decode well known google encodings.

Re: google.com/goto: Google's anti-scraping update

#463

Post by darknavi »

mentalpiracy wrote: post #71
darknavi wrote: post #40

Earlier quoted context omitted.

> Sometimes, these redirect URLs take a perceivable amount of time to load, which is very irritating. Great. On top of my on-going battle with Windows + Firefox + DNS/TLS resolution sometimes stalling for seconds at a time, another few second server-side stall is introduced. I swear that every day modern computing scenarios get slower and slower instead of snappier and snappier.

Wow - this exact same bug has been happening to me too. I gave up on troubleshooting it after the first few attempts came up with nothing, assumed it was just unique to me.

I had no idea others hit this, I just assumed it was some wonky setup I have locally. I hope we both figure it out one day haha (or Firefox does)

Re: google.com/goto: Google's anti-scraping update

#465

Post by ranger_danger »

mitxela wrote: post #290
ranger_danger wrote: post #247

Earlier quoted context omitted.

Do you have a source for this claim? Cloudflare gives me endless captchas on almost any website.

Yes, I bought a cheap residential proxy service and used it to scrape a site.

Sorry, I don't consider that a bypass, or easy.

Re: google.com/goto: Google's anti-scraping update

#466

Post by altmanaltman »

mitxela wrote: post #367
altmanaltman wrote: post #358

Earlier quoted context omitted.

I don't understand your logic. Why would someone that never sees my ads be valuable to me? Why would I care? Everyone spends money on something, the fact you spend it on adblockers doesn't make you automatically more valuable to advertisers. I have never heard of this "advert paradox" concept you mention and I have worked in industry for a while now. All this is just conjecture on your part and doesn't really corresp…

If you have $20 to spend avoiding ads, you have $30 to spend on cool products so the advertiser is willing to spend up to $24 to make you not block ads about a $30 product you'll buy (that costs $5 to make)

That does not make sense at all. Why would you care about people who want to avoid ads? The real data shows very little people actually do that. How much you spend on adblocker is irrelevant to your profile for a business. They will get you some other way if you ad block on digital.

Re: google.com/goto: Google's anti-scraping update

#468

Post by josephcsible »

on_the_train wrote: post #166

I'm confused. Google has done this for several years already. I have a Firefox extension installed that reverts it - it's several years old.

They changed how they do it, such that it's no longer possible for extensions to revert it.

Re: google.com/goto: Google's anti-scraping update

#469

Post by josephcsible »

Dylan16807 wrote: post #219
hurfdurf wrote: post #173

Earlier quoted context omitted.

Previously the clear target URL of a search result was added to the result-link (or div?) as an additional data-tag (aria or whatever, can't remember), and thus was able to be used with a userscript or similar to replace the Googleified tracking redirect link. And it was also part of the googliefied tracking URL, either way you were able to see the clear target URL. That no longer is possible, you must ask google.com…

But the extension doesn't need to use that exact mechanism, and the person that linked it didn't mention specific mechanisms. The purpose of the extension is putting the real URLs back, and it can still do that. And it can still prevent google from knowing which search results you click on, even though OP didn't mention that feature.

> But the extension doesn't need to use that exact mechanism

How do you figure? How else could it possibly work now?

Re: google.com/goto: Google's anti-scraping update

#470

Post by MentalM »

swiftcoder wrote: post #393
Scharkenberg wrote: post #347

Earlier quoted context omitted.

The quintessential HN comment questioning the obvious. Yandex does Putin's bidding much much further than any western search engine ever meddled with their results for political purposes. Yandex employees' lives would be actively at risk if they didn't. Sometimes I really wish a hard reality check would meet the people making such comments from the comfort of their programming chair or from thousands of kilometers aw…

> Yandex does Putin's bidding much much further than any western search engine ever meddled with their results for political purposes. Yandex employees' lives would be actively at risk if they didn't Does media manipulation score higher/lower on your moral index than the widespread domestic spying the US has forced tech companies to be complicit in? Not that media manipulation by the US government is exactly an unhea…

> But it is at best wildly hypocritical to suggest that US tech firms are not subject to government interference

It is very different degree of subjection where we talk about country that have laws, politics, сourts, journalism, all that stuff, and country, where if you dare even to complain about some wild highly illegal stuff that the kgb demands corporation that you working for to do, you end up getting 30 years in prison for treason, falling out of the window, getting poisoned by some chemical weapon, and even then you will be considering yourself a lucky man because nothing very, very bad had happened to your family.

Post reply on HN