Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

Post reply on HN
Print view View original

471–480 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#471

Post by nine_k »

noisy_boy wrote: post #145
nine_k wrote: post #110

Earlier quoted context omitted.

The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…

> Worth the $5 for me Steep at 300 searches per month or about 10 per day. If you search often, it is too expensive. If you rarely search, not worth 5 bucks. They are plainly trying to push people to their $10 unlimited plan. I would have appreciated if they allowed, say, 600 searches per month for $5 or so.

It's relatively rare as of late that I need to specifically find something on the internet, and DDG does not suffice, so I rarely exhaust the 300 Kagi searches.

Most of the time I need to find out something, and in that case, Claude / Gemini / Grok / Perplexity / you name it work better. Most of my queries are technical by nature, not interesting for data mining on me personally.

Re: google.com/goto: Google's anti-scraping update

#472

Post by nine_k »

celsoazevedo wrote: post #461
nine_k wrote: post #110

Earlier quoted context omitted.

The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…

Kagi seems to heavily rely on Yandex for results. I don't use it, so can't compare, but are results much better than using Yandex directly?

I'd say that the ranking, and thus the relevance, is noticeably higher. If I know what I want to find, Kagi is good at surfacing exactly that, not something vaguely related.

Re: google.com/goto: Google's anti-scraping update

#473

Post by codedokode »

somenameforme wrote: post #362
egorfine wrote: post #322

Earlier quoted context omitted.

> For actual web search I actually like using Yandex You might want to reconsider this. https://meduza.io/image/attachments/images/007/699/276/large...

Meduza gonna Meduza. Go do the search yourself. I get nothing like what Meduza claims.

Yandex censors the results, it is required by law, so I do not understand what are you arguing against. To be specific, any URLs, which are blacklisted and banned in Russia, must be omitted from search results. Which includes BBC and other Western media and explains the difference in images because in the Google's results the images come from BBC and Voice of America.

Also, if you try to search for "download Chrome" (in Russian) then the first result in Yandex leads to a scammy website: https://ibb.co/HDRJGgZD The real link is the second one, but I remember a year ago or so there was no official link at all. You can also note that official link to Google has a grey text saying: "the owner of the resource violates Russian law" (Yandex is required to show this notification).

Yandex has also been caught "accidentally" removing the site of Ekaterina Duntsova who was planning to nominate for presidential election in 2024, and showed fake/phishing sites instead.

So, Yandex is only good for searching torrents/pirated movies (which you can watch on rutube) and nothing more.

Re: google.com/goto: Google's anti-scraping update

#474

Post by mplanchard »

goosejuice wrote: post #412
noisy_boy wrote: post #145

Earlier quoted context omitted.

> Worth the $5 for me Steep at 300 searches per month or about 10 per day. If you search often, it is too expensive. If you rarely search, not worth 5 bucks. They are plainly trying to push people to their $10 unlimited plan. I would have appreciated if they allowed, say, 600 searches per month for $5 or so.

$10 is nothing if you're searching that often to no longer be the product. https://proton.me/blog/what-is-your-data-worth-to-google

Agree the $10 plan is so worth it as to be a complete non-question as to whether to renew whenever it comes up.

Also, super cool link, thanks for the share.

Re: google.com/goto: Google's anti-scraping update

#476

Post by rpdillon »

MallocVoidstar wrote: post #390
mitxela wrote: post #181

Earlier quoted context omitted.

Kagi works by scraping Google and other engines, so this should still worry you. I still support the use of Kagi though, as a market signal to Google that we'd even pay them for their product if it wasn't dogshit.

Kagi don't scrape, they pay other engines for API access: https://help.kagi.com/kagi/search-details/search-sources.htm... >Our search results also include anonymized API calls to all major search result providers worldwide

They pay scraping sites for results, including SERP API. When I pointed this out, last time Kagi was discussed on Hacker News, an employee of Kagi said that they're trying to build their own internal index, but he didn't provide details.

Re: google.com/goto: Google's anti-scraping update

#477

Post by wongarsu »

egorfine wrote: post #322
lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

> For actual web search I actually like using Yandex You might want to reconsider this. https://meduza.io/image/attachments/images/007/699/276/large...

Absolutely. Where google is shaping results "to please corpo-political masters", Yandex is obviously influenced by the Kreml

I use Kagi as my daily driver. But there are some kinds of queries where my interests and Yandex's interests happen to align. Mostly anything corporate America wouldn't like, like evil copyright-infringing websites

Re: google.com/goto: Google's anti-scraping update

#478

Post by II2II »

mitxela wrote: post #457
Avicebron wrote: post #434

Earlier quoted context omitted.

You know how many things are asking for "~$10/mo". People don't have the purchasing power they used to, and everything has a subscription model. Firefox with DDG is good enough.

As they said, you wouldn't pay $10/month for search. This is why free pay-with-your-data services won. Nobody wants to pay for what they use, so companies extract value in other ways and you can't complain about that if you're not willing to pay what it costs.

> Nobody wants to pay for what they use, so companies extract value in other ways and you can't complain about that if you're not willing to pay what it costs.

I'm likely old school here, but I never liked "paying for what I use" in the online world because it required revealing some personally identifying information (i.e. to make payment). My attitude is almost certainly irrelevant these days, with the scope of online tracking, sharing of data, and the near necessity of disclosing one's identity to sites other than one's ISP, but it was an attitude that was established in my mind nearly 30 years ago.

(I also don't trust the pay for privacy model. First of all, you can never assume that paying for something implies your data won't be sold. That was even true before the public Internet. Then, even with statements about privacy, you cannot assume that will be true in the future. If the company is sold, management changes, or the company decides the need/want additional revenues, that will change. And that assumes they don't use weasel words about what privacy means, or the ultimate "we reserve the right to changes this agreement at any time".)

Re: google.com/goto: Google's anti-scraping update

#479

Post by egorfine »

wongarsu wrote: post #477
egorfine wrote: post #322

Earlier quoted context omitted.

> For actual web search I actually like using Yandex You might want to reconsider this. https://meduza.io/image/attachments/images/007/699/276/large...

Absolutely. Where google is shaping results "to please corpo-political masters", Yandex is obviously influenced by the Kreml I use Kagi as my daily driver. But there are some kinds of queries where my interests and Yandex's interests happen to align. Mostly anything corporate America wouldn't like, like evil copyright-infringing websites

I sort of agree with you.

Re: google.com/goto: Google's anti-scraping update

#480

Post by Dylan16807 »

josephcsible wrote: post #469
Dylan16807 wrote: post #219

Earlier quoted context omitted.

But the extension doesn't need to use that exact mechanism, and the person that linked it didn't mention specific mechanisms. The purpose of the extension is putting the real URLs back, and it can still do that. And it can still prevent google from knowing which search results you click on, even though OP didn't mention that feature.

> But the extension doesn't need to use that exact mechanism How do you figure? How else could it possibly work now?

I said that in my first comment.

You do a google search. The extension resolves every link on the page immediately via the goto urls. This doesn't leak any information to google because they obviously know which links they sent you. Now the links are resolved and you can copy and click them and get clean URLs, without sending any information about which ones you're copying or clicking.

Post reply on HN