511–520 of 545 posts
If you're trying to fight scrapers on a small site, delay links can only flatten bursts. If bots can only scrape at human speed per IP, they can just scrape 100x as many sites at the same time. Once every bot operator does that, total traffic will return to the original level.
As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…
> For actual web search I actually like using Yandex You might want to reconsider this. https://meduza.io/image/attachments/images/007/699/276/large...
For everyone to make the search themselves
Earlier quoted context omitted.
ISPs do not see query strings since Google uses HTTPS. ISPs can only see the domain and IP address you are connecting to.
Can they even see the domain? Assuming you're not using the ISP's DNS of course.
There's even a reverse pointer query that can be made to get a domain to IP from a legacy ARPA TLD, although it isn't 100% robust.
This text appears on this page https://www.autom.dev/blog/google-search-goto-links
> The real URL is in the Location header on /goto. Request that URL. Do not follow the redirect.
And this text appears on this page https://www.autom.dev/blog/google-goto-url-fix
> Do not follow the redirect. Read Location.
That's what a redirect is, reading the value of the location header and then requesting it. How do you not follow the redirect by reading the location header? Once you've made the request to the /goto url, with GET or HEAD, to get the location header, google knows you're interested in whatever it is putting in the location header and can assume you're going to go there, if you're letting the User Agent (curl or the browser) go there for you or not.
Earlier quoted context omitted.
I'm using Kagi every day, and unfortunately, it is not true anymore. I think what happens is that the blanket banning of IP ranges because of LLMs, and flat out incompetence and laziness, affects them heavily. I get results, or just a few unrelated ones many times every day, while Google happily returns what I searched for. I think Kagi's index is shrinking, or they really made their algorithm worse. I slowly shifted…
Same here. Been paying for Kagi for probably a year or so, but I find that it hasn’t really been able to keep up with at least my personal expectations. It feels quite slow (which I don’t remember it always being), and the search result quality is not always the best. On top of that, the 10$/mo is a little steep for me, given that their assistant and maps are quite sub-par in my opinion. I would have preferred them t…
It definitely is. The extension-based redirect in Safari is also unreliable. Furthermore, many of my searches (and others', apparently) unexplainably defaulted to Groningen in the Netherlands, even though I'm nowhere near the place.
I ended up not renewing my annual subscription. Since then, I vibecoded a small userscript that replicates domain ranking for DuckDuckGo – plus DDG already lets you turn off ads natively.
Earlier quoted context omitted.
Kagi don't scrape, they pay other engines for API access: https://help.kagi.com/kagi/search-details/search-sources.htm... >Our search results also include anonymized API calls to all major search result providers worldwide
Is Marginalia the only real competing search engine ?
Earlier quoted context omitted.
> But the extension doesn't need to use that exact mechanism How do you figure? How else could it possibly work now?
I said that in my first comment. You do a google search. The extension resolves every link on the page immediately via the goto urls. This doesn't leak any information to google because they obviously know which links they sent you. Now the links are resolved and you can copy and click them and get clean URLs, without sending any information about which ones you're copying or clicking.
As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…
I made https://froogle.fyi out of a desire to get that old school search back. Source: https://github.com/scosman/froogle It hasn’t replaced Kagi as my daily driver but the UX is fun.
What does this mean? Isn’t the location header the redirect? Am I not following the redirect by requesting the location header url?
Google has gone so bad over the past few years. You only get like 8 results per page. I remember there was a time that I wonder how a site get reached if it ranked on the second page, when I can set the number of results to be 50. The censorship is also really bad, and google doesn't even tell you the results are censored, returning totally nonsense results while other search engines work normally.
I've been supporting Brave search which returns 20 results per page and has other features. It used to be not good a few years ago, but now the results are often better than google's.