Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

Post reply on HN
Print view View original

391–400 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#391

Post by Aldipower »

topato wrote: post #303
mitxela wrote: post #181

Earlier quoted context omitted.

Kagi works by scraping Google and other engines, so this should still worry you. I still support the use of Kagi though, as a market signal to Google that we'd even pay them for their product if it wasn't dogshit.

I’m pretty sure Kagi’s entire pitch is that they’re an alternate and original dataset. Theirs is the only search engine whose data isn’t based upon google or bing’s databases. To anyone who didn’t find Kagi useful in the past, I strongly recommend you try it again, as they have made huge strides in usability and quality of surfaced data in just the past year…. Huge strides since even a few months ago too, with more a…

Kagi is mostly based on Bing and Google results, officially by API and not scraped, and ranked by there own mechanism I think.

Re: google.com/goto: Google's anti-scraping update

#393

Post by swiftcoder »

Scharkenberg wrote: post #347
swiftcoder wrote: post #275

Earlier quoted context omitted.

> campaigns for a Russian product on HN are annoying, and appaling if you have any morals How is Yandex more morally-bankrupt than Google in your analysis?

The quintessential HN comment questioning the obvious. Yandex does Putin's bidding much much further than any western search engine ever meddled with their results for political purposes. Yandex employees' lives would be actively at risk if they didn't. Sometimes I really wish a hard reality check would meet the people making such comments from the comfort of their programming chair or from thousands of kilometers aw…

> Yandex does Putin's bidding much much further than any western search engine ever meddled with their results for political purposes. Yandex employees' lives would be actively at risk if they didn't

Does media manipulation score higher/lower on your moral index than the widespread domestic spying the US has forced tech companies to be complicit in?

Not that media manipulation by the US government is exactly an unheard of event (at least as far back as the whole WMDs in Iraq scandal).

> Sometimes I really wish a hard reality check would meet the people making such comments from the comfort of their programming chair or from thousands of kilometers away

Look, I'm American myself, and have as keen a dislike of Russia's foreign and domestic policies as the next guy. But it is at best wildly hypocritical to suggest that US tech firms are not subject to government interference, in a manner that may be detrimental to their customers.

Re: google.com/goto: Google's anti-scraping update

#394

Post by miki123211 »

andwur wrote: post #143
nine_k wrote: post #110

Earlier quoted context omitted.

The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…

Good workaround! It's just sad that to achieve the previous behavior we now need to burn significantly more compute, and in turn energy, and with far worse performance and an inverted UX. The most frustrating part is that they have all of the data, and oodles of compute, available to surface the same very functional experience they originally offered, but they would prefer the image of being visionaries rather than t…

I think the "original experience", whatever it may have been (and I strongly suspect there's more nostalgia here than actual worse results) would be very quickly overrun by modern SEO methods / tactics.

Kagi can be "better" than Google (for some very narrow definitions of better) because they basically don't have to think about SEO at all. They're small enough that nobody does SEO against Kagi. If their algorithm is sufficiently different that typical Google tactics don't work on it, even if it's much more naive because of that, Kagi wins.

Re: google.com/goto: Google's anti-scraping update

#395

Post by fred_ »

ruszki wrote: post #332
nine_k wrote: post #110

Earlier quoted context omitted.

The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…

I'm using Kagi every day, and unfortunately, it is not true anymore. I think what happens is that the blanket banning of IP ranges because of LLMs, and flat out incompetence and laziness, affects them heavily. I get results, or just a few unrelated ones many times every day, while Google happily returns what I searched for. I think Kagi's index is shrinking, or they really made their algorithm worse. I slowly shifted…

Same here. Been paying for Kagi for probably a year or so, but I find that it hasn’t really been able to keep up with at least my personal expectations. It feels quite slow (which I don’t remember it always being), and the search result quality is not always the best. On top of that, the 10$/mo is a little steep for me, given that their assistant and maps are quite sub-par in my opinion. I would have preferred them to focus more on the search experience, and less on the other products. Or give me a cheaper plan for only unlimited search without the other products.

Re: google.com/goto: Google's anti-scraping update

#396

Post by chadgpt3 »

dotancohen wrote: post #387
chadgpt3 wrote: post #340

Earlier quoted context omitted.

As far as I'm aware, when a country sets out to exterminate* a race, that's a holocaust. Am I wrong? * Often the torture is the point, and they don't completely eradicate the race because then they wouldn't be able to keep torturing them

Israel never set out to exterminate, Israel set out to get her hostages back - including babies under a year old. You know who did set out to exterminate? Hamas, who in their very own charter state their intention to eradicate the Jews: https://avalon.law.yale.edu/20th_century/hamas.asp I suggest reading this article which explains the propaganda campaigns that followed the October 7th attacks on Israel: https://www.…

Please make the same analysis for Germany in regard to the Gleiwitz radio station

Re: google.com/goto: Google's anti-scraping update

#397

Post by mitxela »

Aldipower wrote: post #391
topato wrote: post #303

Earlier quoted context omitted.

I’m pretty sure Kagi’s entire pitch is that they’re an alternate and original dataset. Theirs is the only search engine whose data isn’t based upon google or bing’s databases. To anyone who didn’t find Kagi useful in the past, I strongly recommend you try it again, as they have made huge strides in usability and quality of surfaced data in just the past year…. Huge strides since even a few months ago too, with more a…

Kagi is mostly based on Bing and Google results, officially by API and not scraped, and ranked by there own mechanism I think.

Officially Kagi pays SerpAPI to scrape Google for them, since Google doesn't have an API they'll let Kagi use.

Re: google.com/goto: Google's anti-scraping update

#398

Post by benhurmarcel »

noisy_boy wrote: post #145
nine_k wrote: post #110

Earlier quoted context omitted.

The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…

> Worth the $5 for me Steep at 300 searches per month or about 10 per day. If you search often, it is too expensive. If you rarely search, not worth 5 bucks. They are plainly trying to push people to their $10 unlimited plan. I would have appreciated if they allowed, say, 600 searches per month for $5 or so.

The 300 searches a month work for me because I only use Kagi when I think it will give better results than others. And most searches are trivial so any free engine will do (like you have the website name and want the url).

Re: google.com/goto: Google's anti-scraping update

#399

Post by narrator »

lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

Yandex also easily finds copyrighted stuff and other various types of wrongthink banned by Google.

Re: google.com/goto: Google's anti-scraping update

#400

Post by squidbeak »

BoingBoomTschak wrote: post #384
moffkalast wrote: post #359

Earlier quoted context omitted.

Bold to assume Russian propaganda is not aimed at the west.

[flagged]

Naivete and scorn of autism in one short sentence - you're very efficient.

Yandex.com is the international front end to yandex, and not a portal for 'a handful of western autists'.

But on Russia's granular propaganda and disinformation efforts, you ought to read Peter Pomerantsev's excellent "This is Not Propaganda" before making silly sweeping statements.

Post reply on HN