Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

381–390 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#381

> The url parameter uses a custom, Google-specific encoding. So that will be figured out, making the whole exercise void. Google knows this. What is the real goal here?

"Figuring it out" doesn't help if it's encrypted with a key only Google holds or a reference to some database record that you don't have. In either case, the only way to resolve this is to ask Google, which means they can track it as a click (and rate limit etc.).

Re: google.com/goto: Google's anti-scraping update

#382

Earlier quoted context omitted.

Have you tried Yandex and evaluated the quality of its results, or do you just feel obligated to continuously put it down because it's russian?

I have, and it has never been good for my purposes. In English, I just can't find things I need, in Russian, I find things I wish to never see again, tbh. What is your experience with it? Edit: The best search for my purposes historically has been on Pinterest. Unfortunately, not anymore. I normally search for visual artifacts (photos, art, schematics, etc), out of all search engines Google is still the only one that…

Have you tried its image search? It's pretty good compared to its competitors. Google's image search fell off as they severely nerfed it.

Re: google.com/goto: Google's anti-scraping update

#383
post #2

Direct URLs in Google search results have been replaced with redirect URLs in the form of www.google.com/goto?url= . The base64 data appears to consist of a very basic protobuf structure, containing a long string of bytes in field 2 which presumably identify the URL. Sometimes, these redirect URLs take a perceivable amount of time to load, which is very irritating.

The biggest problem with this is if the target URL doesn't load but also doesn't quickly error out, like e.g. many .gov sites in Europe (seems like they are just dropping traffic from non-US IPs).

Now you can't load the page and can't easily (using only the browser UI) get a link to paste into archive.org or archive.is to read the page.

Re: google.com/goto: Google's anti-scraping update

#384

Earlier quoted context omitted.

You think Google doesn't do censorship? Or any mainstream search engine, for that matter? It's the usual "I prefer censorship not targeted at/tailor made for me".

Bold to assume Russian propaganda is not aimed at the west.

[flagged]

Re: google.com/goto: Google's anti-scraping update

#385

Brave search (free) or Kagi (paid) are able to replace Google and not feel like I'm missing out. Brave has its own independent index which is cool.

+1 for Kagi, even if Maps and Shopping and Images aren’t as good as Google (tbh their Images search is good enough 8/10 times) they’ve got the actual web search stuff working quite well.

Re: google.com/goto: Google's anti-scraping update

#386
post #356

Earlier quoted context omitted.

I don't. At this point I just ask the oracle in natural language. There's so many to choose from. Why spend hours wading through only vaguely related things? You can ask it for a direct answer or you can ask it to teach you about the topic.

Sometimes there is joy in discovering things for yourself. Also, the oracle currently relies on humans publishing new information, what will you do when the incentive is gone and the oracle has nothing to harvest?

> Sometimes there is joy in discovering things for yourself.

Agreed. And webrings still exist in certain corners of the internet.

> the oracle currently relies on humans publishing new information

New? When I ask it how to do something with a piece of software it's perfectly capable of answering me by means of direct interrogation of the source code.

I'll grant you that it owes ~all of its knowledge of the world at large to having been bootstrapped using the more or less complete body of humanity's published works. However I think that was merely the quickest path to bootstrap it as opposed to the only one.

Re: google.com/goto: Google's anti-scraping update

#387

Earlier quoted context omitted.

Your Holocaust inversion is not only morally wrong, it is also factually incorrect.

As far as I'm aware, when a country sets out to exterminate* a race, that's a holocaust. Am I wrong? * Often the torture is the point, and they don't completely eradicate the race because then they wouldn't be able to keep torturing them

Israel never set out to exterminate, Israel set out to get her hostages back - including babies under a year old.

You know who did set out to exterminate? Hamas, who in their very own charter state their intention to eradicate the Jews: https://avalon.law.yale.edu/20th_century/hamas.asp

I suggest reading this article which explains the propaganda campaigns that followed the October 7th attacks on Israel: https://www.adl.org/resources/article/hamas-its-own-words

Only one side is calling for genocide, and it is the Arab side - propaganda and disinformation campaigns aside.

Re: google.com/goto: Google's anti-scraping update

#388

Earlier quoted context omitted.

I was waiting to see how long it took for someone to "make this about Palestine" - their strategy is to inundate all discussion with the topic. Over half the journalists in the Gaza Strip are affiliated with Hamas. Many of them infiltrated Israel in 2023, and some of them held Israeli hostages in their homes, including children hostages. https://www.terrorism-info.org.il/en/abou

So Israel killing "bad journalists" is fine, but Russian are killing only the good ones? Nice world to live in. You should know that any country can declare a journalist a foreign spy, a terrorist, but does it give them right to kill? Without due process, without trial? Because someone says "they are Hamas". It's Godwin time.

Hamas claims that the journalists are Hamas, not Israel. And I'm not declaring any journalist "good" or "bad". I'm stating that a Hamas member is a valid target. Him moonlighting as a journalist does not change that.

Re: google.com/goto: Google's anti-scraping update

#389

Brave search (free) or Kagi (paid) are able to replace Google and not feel like I'm missing out. Brave has its own independent index which is cool.

I use kagi exclusively because it works. I get the results I need every time, and it's not annoying. I haven't used Google in years. Simply no need. I really hope people keep paying kagi so they stick around.

I hope you also know they're using Yandex as one of their indexes.

https://kagifeedback.org/d/5445-reconsider-yandex-integratio...

Uruky is the way. 2x cheaper also.

Re: google.com/goto: Google's anti-scraping update

#390
post #110

Earlier quoted context omitted.

The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…

Kagi works by scraping Google and other engines, so this should still worry you. I still support the use of Kagi though, as a market signal to Google that we'd even pay them for their product if it wasn't dogshit.

Kagi don't scrape, they pay other engines for API access: https://help.kagi.com/kagi/search-details/search-sources.htm...

>Our search results also include anonymized API calls to all major search result providers worldwide

Post reply on HN