Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

google.com/goto: Google's anti-scraping update

autom.dev

Post reply on HN
Print view View original

451–460 of 545 posts

Re: google.com/goto: Google's anti-scraping update

#451

Post by sofixa »

sugarkjube wrote: post #238
sofixa wrote: post #94

Earlier quoted context omitted.

> Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our corpo-political masters". As long as you don't search anything related to Russia itself, or to Russian interests elsewhere (like their invasion of Ukraine). Then it's very heavily biased, priority is given to state ran propaganda mills, independent media is hidden from results, etc. And before anyone start…

Reminds me of this old joke. A Soviet citizen and an American are sitting next to each other on a commercial flight. The American turns to the Russian and says, "I have to hand it to you — your state propaganda is very impressive. You really know how to shape what people think." The Soviet smiles, nods, and replies, "Thank you, but it's really nothing compared to American propaganda." The American looks shocked and s…

Nobody is denying the existence of American propaganda, but it's an absurd false equivalency to pretend it's comparable to Russia - the country where there are multiple Wikipedia articles with lists of murdered journalists and there is a day of remembrance for murdered journalists. The country which sends assassins to kill dissidents abroad. Especially when we have the Mitrokhin archives since the 1990s in which a KGB archivist painstakingly tried to warn us about the firehose of shit method. Do you know the conspiracy theories about AIDS were started by the KGB? This is the kind of shit we're talking about, not Americans jacking themselves off on their anthem thanking troops for their service.

Actually sorry, it's beyond absurd.

Re: google.com/goto: Google's anti-scraping update

#453

Post by oblio »

radiator wrote: post #446
oblio wrote: post #289

Earlier quoted context omitted.

[flagged]

> There are "politics and ideology" and there is right and wrong. That is exactly how six-year-olds think.

Thankfully I'm not a 6 year old, and you're not my dad.

If anything, yours is the kind of retort a 12 year old would find clever.

Re: google.com/goto: Google's anti-scraping update

#454

Post by sourcecodeplz »

rkagerer wrote: post #158

I did an interview with Google around 20 years ago, where they posed a challenge involving tracking which specific search results people click. It's obvious in hindsight the solution required rewriting all the urls to redirect through their servers. Note this was in the days before they already did so as a matter of course. I failed to gain traction on the problem, because to me the very idea of doing such a thing wa…

i tried visiting kagi now, and there is no way to search if you dont signup first?

also this page doesnt have a submit button and pressing enter doesnt submit the form either:

https://kagi.com/html

Re: google.com/goto: Google's anti-scraping update

#455

Post by emaro »

Lord_Zero wrote: post #298
emaro wrote: post #183

Earlier quoted context omitted.

I do agree it's kinda expensive. I'm lucky enough that my employer pays the $25 plan for met because it includes access to many LLMs.

I pay the $10 and use assistant multiple times a day and never needed the fatter models since for coding I have copilot and Gemini

True. I got this as my first LLM subscription (was a long time skeptic, less so now but still a bit), but now I have cursor too so maybe I can even downgrade. Thanks for the reminder. :)

Re: google.com/goto: Google's anti-scraping update

#456

Post by _blk »

RobotToaster wrote: post #330
athrowaway3z wrote: post #229

Earlier quoted context omitted.

For Google, the business side of you paying them doesn't work out as obvious as it might seem. Advertisers are sold the idea that 1000 clicks/impression is X dollars. The average impression might be priced at 1000/X$. But you are worth much more than that for two reasons. - you are a person willing to pay 5$ - Google can sell a “collectivized” product. The same way that eg collective farmer crop insurance creates act…

The advert paradox, anyone willing to spend money to avoid ads is inherently worth more to advertisers, since they're willing to spend money. The more money you're willing to pay to avoid ads the more you're worth to advertisers. That's why tools like uBlock and YouTube morphe are the only answer, negotiating with terrorists never works.

Sad but true. I paid for Prime to be ad free, easily worth 3$/month but sure enough now part of their catalog has become "only available with ads" or ads in their FireTV is before you hit the prime app (and then again when you do). So now their whole product is crap and I'm looking for a new streaming stick solution [anyone?] Sounds like it's time for a newcomer/disruptor like Angel Studios to release one.

Natural next step of the ad lobby is obviously to pay the lobby to make it illegal to bypass the money making machine if they haven't done so yet. I don't doubt that capitalism still works, it just takes time to "boil the frogs" dead [enough] to stop them from buying such crappy products.

Re: google.com/goto: Google's anti-scraping update

#457

Post by mitxela »

Avicebron wrote: post #434
brookst wrote: post #421

Earlier quoted context omitted.

Pretty much sums up why Google’s business model changed. Everyone wants good search, few are willing to be even $10/mo.

You know how many things are asking for "~$10/mo". People don't have the purchasing power they used to, and everything has a subscription model. Firefox with DDG is good enough.

As they said, you wouldn't pay $10/month for search. This is why free pay-with-your-data services won. Nobody wants to pay for what they use, so companies extract value in other ways and you can't complain about that if you're not willing to pay what it costs.

Re: google.com/goto: Google's anti-scraping update

#458

Post by iamnothere »

lubujackson wrote: post #82

As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…

Yes, it’s been impossible to trust Google since the “Jigsaw” project, and especially when they started blanking out search pages on trending topics with messages like “these results are changing quickly”. It’s clear that they have moved from being a neutral arbiter to pushing a perspective.

I refuse to use a search engine that sees me as cattle to be steered towards “preferred” information sources. I specifically enjoy reading fringe material and will continue to seek and find it, thank you very much.

Re: google.com/goto: Google's anti-scraping update

#459

Post by JohnTHaller »

googapologist wrote: post #151
JohnTHaller wrote: post #128

Note: Article published by Autom.dev which, from a quick read of their homepage, seems like it scrapes Google search results in violation of Google's terms of service and sells those results to customers via an API. That's just my quick read of it, though, so this could be wrong.

Note: Irrelevant. The reported behavior exists, does it not? Anecdata: I've observed this behavior for several weeks already as a regular user without a Google account and there are countless comments from regular users reporting the same behavior.

It's relevant to know when the source of information is biased due to a conflict of interest. The source's business is apparently scraping Google (and Bing, etc).

Re: google.com/goto: Google's anti-scraping update

#460

Post by celsoazevedo »

Aldipower wrote: post #391
topato wrote: post #303

Earlier quoted context omitted.

I’m pretty sure Kagi’s entire pitch is that they’re an alternate and original dataset. Theirs is the only search engine whose data isn’t based upon google or bing’s databases. To anyone who didn’t find Kagi useful in the past, I strongly recommend you try it again, as they have made huge strides in usability and quality of surfaced data in just the past year…. Huge strides since even a few months ago too, with more a…

Kagi is mostly based on Bing and Google results, officially by API and not scraped, and ranked by there own mechanism I think.

Kagi was using Yandex, it comes up on every Kagi post here. Seems that they've removed that info from their site now though...
Post reply on HN