Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

Post reply on HN
Print view View original

361–370 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#361

Post by Aldipower »

lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

Just tried Yandex again, after a long time, and indeed it was a blast from the past, in a positive sense! So many real search results, which transports the feel of an authentic mirror of the web contents. Default search engine from now on! Thanks for the hint.

Re: google.com/goto: Google's anti-scraping update

#362

Post by somenameforme »

egorfine wrote: post #322
lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

> For actual web search I actually like using Yandex You might want to reconsider this. https://meduza.io/image/attachments/images/007/699/276/large...

Meduza gonna Meduza. Go do the search yourself. I get nothing like what Meduza claims.

Re: google.com/goto: Google's anti-scraping update

#363

Post by ormax3 »

skinfaxi wrote: post #249
Jakob wrote: post #137

Earlier quoted context omitted.

Until 10 years ago, i used Dash for that. It’s still around https://kapeli.com/dash It’s instant, works offline, auto-updates, and includes all the websites you listed, and allows for custom ones too.

What do you use now?

https://devdocs.io/

Re: google.com/goto: Google's anti-scraping update

#364

Post by mitxela »

ranger_danger wrote: post #276
mitxela wrote: post #245

Earlier quoted context omitted.

That's just Google not caring about your ISP because you live in the global south. Everyone gets this. US citizens get the exact same problem but Google whitelists their networks.

> because you live in the global south I don't, but thanks for confidently assuming.

At least Google thinks you do. And Google kind of has the power to decide where the global south is, at least as far as it pertains to captchas.

Re: google.com/goto: Google's anti-scraping update

#365

Post by dartharva »

nine_k wrote: post #110
lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…

...you.. do know classic Google Search is still an option, right? Just add `&udm=14` to the search URL.

Also Google AI mode is accessible on google.com/ai.

Re: google.com/goto: Google's anti-scraping update

#366

Post by mitxela »

dwedge wrote: post #351
mitxela wrote: post #290

Earlier quoted context omitted.

Yes, I bought a cheap residential proxy service and used it to scrape a site.

Ironic that I'd need something like that to stop getting blocked from a residential IP

It's probably cheaper to switch your ISP to one that your target site does not block, than to pay someone else to buy that connection and proxy all your traffic through them.

Re: google.com/goto: Google's anti-scraping update

#367

Post by mitxela »

altmanaltman wrote: post #358
RobotToaster wrote: post #330

Earlier quoted context omitted.

The advert paradox, anyone willing to spend money to avoid ads is inherently worth more to advertisers, since they're willing to spend money. The more money you're willing to pay to avoid ads the more you're worth to advertisers. That's why tools like uBlock and YouTube morphe are the only answer, negotiating with terrorists never works.

I don't understand your logic. Why would someone that never sees my ads be valuable to me? Why would I care? Everyone spends money on something, the fact you spend it on adblockers doesn't make you automatically more valuable to advertisers. I have never heard of this "advert paradox" concept you mention and I have worked in industry for a while now. All this is just conjecture on your part and doesn't really corresp…

If you have $20 to spend avoiding ads, you have $30 to spend on cool products so the advertiser is willing to spend up to $24 to make you not block ads about a $30 product you'll buy (that costs $5 to make)

Re: google.com/goto: Google's anti-scraping update

#368

Post by pmontra »

RobotToaster wrote: post #330
athrowaway3z wrote: post #229

Earlier quoted context omitted.

For Google, the business side of you paying them doesn't work out as obvious as it might seem. Advertisers are sold the idea that 1000 clicks/impression is X dollars. The average impression might be priced at 1000/X$. But you are worth much more than that for two reasons. - you are a person willing to pay 5$ - Google can sell a “collectivized” product. The same way that eg collective farmer crop insurance creates act…

The advert paradox, anyone willing to spend money to avoid ads is inherently worth more to advertisers, since they're willing to spend money. The more money you're willing to pay to avoid ads the more you're worth to advertisers. That's why tools like uBlock and YouTube morphe are the only answer, negotiating with terrorists never works.

I'm willing to pay time or money to avoid ads, not to be spied and harassed by companies making me offers to buy something that almost certainly I don't want to buy. Those companies should pay me a percent of what I make them save by not buying ads to serve me.

And eventually when I want to buy something I research it and buy it and nobody had to pay anything to Google.

Re: google.com/goto: Google's anti-scraping update

#369

Post by dwedge »

vhcr wrote: post #86
zenoprax wrote: post #65

Earlier quoted context omitted.

Can they even see the domain? Assuming you're not using the ISP's DNS of course.

Unless you're using DNS over HTTPS they can see the unencrypted DNS traffic. There's also Encrypted Client Hello, but they can also see which IP you're connecting to.

Also if the domain responds on the bare IP or there's nothing else they've seen on that IP it's a fair assumption. This is without even wondering if their router sends telemetry

Re: google.com/goto: Google's anti-scraping update

#370

Post by egorfine »

somenameforme wrote: post #362
egorfine wrote: post #322

Earlier quoted context omitted.

> For actual web search I actually like using Yandex You might want to reconsider this. https://meduza.io/image/attachments/images/007/699/276/large...

Meduza gonna Meduza. Go do the search yourself. I get nothing like what Meduza claims.

Yeah I did while I was in Kiev at the time. Did not screenshot unfortunately but yeah that's what Yandex did show for many "Bucha" search terms.
Post reply on HN