Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

Post reply on HN
Print view View original

91–100 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#91

Post by userbinator »

I was suspicious when they started obfuscating URLs in their own browser, then on their SERPs, and now this...

For many years, I had my filtering proxy rewrite the URLs in the way mentioned in the article.

Almost exactly a year ago, Google stopped working without JS. I stopped using Google.

Now they're upping the game, and as the article (which is a bit of marketing itself) admits, those who have the resources can still blast through these obstacles while those who don't are locked out.

Since the article brings up "AI scrapers", I'll just point it out as being the latest scare-tactic for coercing people to give up the privacy, anonymity, and (browser) freedom of an open interoperable Internet.

Re: google.com/goto: Google's anti-scraping update

#92

Post by bitpush »

stackghost wrote: post #8
nullbio wrote: post #7

The enshittification continues. Nothing good ever comes from businesses desperately trying to protect their moats rather than making their products better so they don't need to.

Thank Prabhakar Raghavan for that. After destroying Yahoo’s search product he failed up and did the same at Google. In reward for making Google Search materially worse in every way except ad revenue, he failed up again and was promoted to a cushy do-nothing role. Everything wrong with the tech industry, embodied in a single person.

Do you have any source outside of the Ed Zitron article (which is highly specualtive?)

Re: google.com/goto: Google's anti-scraping update

#93

Post by vish045 »

adtac wrote: post #68

why is this such a bad thing? it's not really any different from using a uuid as a user facing key, which basically everyone does. and trying to protect your moat isn't automatically a bad thing. they clearly feel it's helping competition, so they're closing a hole. competition is good doesn't mean help your competitors . - coming from someone who's been using fastmail as my personal for ~10 years because i don't wan…

I guess someone made a website which google crawled and adding a senf made uuid to it is like google trying to own it rather than just being a true search engine just having index to it.

Re: google.com/goto: Google's anti-scraping update

#94

Post by sofixa »

lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

> Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our corpo-political masters".

As long as you don't search anything related to Russia itself, or to Russian interests elsewhere (like their invasion of Ukraine). Then it's very heavily biased, priority is given to state ran propaganda mills, independent media is hidden from results, etc.

And before anyone starts thinking about whatabouting, Yandex are based in a country where journalists are openly and publicly assasinated to intimadate. It would be delusional to expect any sort of press and related (like search engine or aggregation) freedom, or to try to compare this to anything in any other developed country.

Re: google.com/goto: Google's anti-scraping update

#95

Post by leobg »

bitpush wrote: post #89
lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

> There are still illegal streaming sports and movie sites everywhere (who knew) and all other seedy corners of the internet that have been neatly erased by Google This is such a strange position to take. Apple doesnt allow all sorts of apps on the appstore, but that is never said as "Apple is erasing illegal streaming". Google is simply not showing the results. They are not taking down, or banning, or doing anything…

The pain being made is: If you use a search engine as your eyes to see what exists “on the internet”, then, absolutely, whatever Google hides from its results or fails to index is “erased” from “your” experience of the internet.

Re: google.com/goto: Google's anti-scraping update

#96

Post by what »

InsideOutSanta wrote: post #48
demibabs wrote: post #10

Can someone explain why this matters? Not being flippant I just don’t understand why this would be important.

It's primarily relevant because it makes scraping search results much more expensive, solidifying Google's effective monopoly on Internet search. Google has previously tried to prevent scraping of search results using legal means, but courts correctly think that scraping of Google's search results should be legal, just as Google's scraping of the whole Internet is legal. This is Google's reaction to that.

You can opt out from Google scraping you though? In theory you can opt out of anyone scraping you (if people were well behaved). Google should get to opt out of being scraped too.

Re: google.com/goto: Google's anti-scraping update

#97

Post by what »

JohnFen wrote: post #22
scottyah wrote: post #11

Earlier quoted context omitted.

Why? I don't really see why this is bad, other than if your company blocks url shorteners/redirects. What are you losing?

Because I wouldn't want Google to know what links I click on. When I used to use Google, and they added such redirects that included the destination URL as a parameter, I'd edit the link to make it just that URL before resolving it. This new scheme would make that impossible.

They already knew which links you clicked on though…

Re: google.com/goto: Google's anti-scraping update

#99

Post by ma2kx »

At least they seem still provide results for my searxng instance. I mean sure, they are horrible but duckduckgo just blocks most queries (and I'm the only person using the ip / seraxng instance)...

Next i'll do is to implement tavilly, exa, tinyfish etc. as search engines for searxng. No agents, no mcp, just their search api endpoint.

Re: google.com/goto: Google's anti-scraping update

#100

Post by crummy »

rmunn wrote: post #70
pibaker wrote: post #62

Earlier quoted context omitted.

I don't know how global consistency is actually useful, but I can easily think of ways it is less convenient. For example, DST starts and ends on different dates in the UK and the US. Which dates should google return when someone from either country searches for just "DST end date?" Someone lives in Orange County and searches for "Orange County Sheriffs," which one of the eight Orange Counties should google return? T…

I wouldn't mind that one. That a man in the US and a woman in Japan would get different results for a search for "sushi restaurant" is perfectly reasonable (even if the woman in Japan was searching in English). It's when two people in the same neighborhood got different results for the same search that I said "wait a minute, they're personalizing search results now for ad-targeting purposes" and ditched them. The pot…

I understand the potential for abuse, but it is useful when I search for python and get programming languages and someone else gets snakes.
Post reply on HN