501–510 of 543 posts
ISPs can presumably correlate the Google query string with the request following the response to the goto and so make a search index? I guess they would charge too much. Do any large ISPs use visit data to feed into a search index?
ISPs do not see query strings since Google uses HTTPS. ISPs can only see the domain and IP address you are connecting to.
Earlier quoted context omitted.
> Nobody wants to pay for what they use, so companies extract value in other ways and you can't complain about that if you're not willing to pay what it costs. I'm likely old school here, but I never liked "paying for what I use" in the online world because it required revealing some personally identifying information (i.e. to make payment). My attitude is almost certainly irrelevant these days, with the scope of onl…
I mean, there’s more privacy protection on payment data than there is for email address and browsing habits. Plus most companies just push out payment handling to stripe. So most websites have little idea who actually transacted with them to begin with (email dependent).
Earlier quoted context omitted.
$10 is nothing if you're searching that often to no longer be the product. https://proton.me/blog/what-is-your-data-worth-to-google
Agree the $10 plan is so worth it as to be a complete non-question as to whether to renew whenever it comes up. Also, super cool link, thanks for the share.
That's confidence in the product right there. I'd argue any subscription which won't do this is designed to exploit people who don't need or use a subscription.
As much as I am sad that Google died like 15 years ago, I am past the mourning phase. That was when they announced they were shifting from returning websites to "returning answers" and it has been a long slide into shittification I do enjoy using their free AI. For actual web search I actually like using Yandex. It reminds me of old Google, returning reasonable results and much less "shaping results to please our cor…
The real "old-school Google, but modern" is Kagi, with the caveat of being paid. (Worth the $5 for me.) But LLMs can be commanded. This is may bookmark alias for invoking the spirit of old Gog=ogle from within the new, AI-based Google: https://google.com/search?q=You%20are%20Google%20Search%20fr... (Cleaned / decoded: 'You are Google Search from 2004. Given a search request, provide 10 links to relevant pages, each w…
I wonder if this would also work on other sites getting hammered with bots. Allow each anonymous user 1 "real" page load then turn the rest into encrypted links that the web server can decrypt. If a session cookie with reputation exists, stop screwing with the links.
Kind of annoying but it'd allow tracking if the same agent/bot is churning through IPs/User Agents.
While a lot of people are concerned with local model performance, I wonder how feasible is it now to run a local indexed web search? Surely running an old school Google is possible with the beefy AI rigs today. I know the problem will be crawling which would be bottlenecked by the ISP but I use Google to search SO, Wikipedia, programming language docs, Github issues, and AWS docs. I think a feasible workflow would be…
Impossible. The majority of websites firewall automated crawler traffic (because of the rise of the bots), only making exceptions for the largest search engines. There is no possibility of starting a new crawler.
I imagine it'd take quite a bit of shape for the index and be hard to keep it up to date unless you restrict what it indexes.
Earlier quoted context omitted.
> Worth the $5 for me Steep at 300 searches per month or about 10 per day. If you search often, it is too expensive. If you rarely search, not worth 5 bucks. They are plainly trying to push people to their $10 unlimited plan. I would have appreciated if they allowed, say, 600 searches per month for $5 or so.
Pretty much sums up why Google’s business model changed. Everyone wants good search, few are willing to be even $10/mo.
While a lot of people are concerned with local model performance, I wonder how feasible is it now to run a local indexed web search? Surely running an old school Google is possible with the beefy AI rigs today. I know the problem will be crawling which would be bottlenecked by the ISP but I use Google to search SO, Wikipedia, programming language docs, Github issues, and AWS docs. I think a feasible workflow would be…
SearXNG configured as in the OpenWebUI docs is pretty cool. My "Hello World" with a new agent framework is teaching it to use SearXNG. Hook this in as a tool and the agent can answer a lot of questions. SearXNG is more of a metasearch, the dude who wrote it pops in on here and is working on a cool sounding project that is more like a local personal search engine, I forget the name, but I've been meaning to check it o…
While a lot of people are concerned with local model performance, I wonder how feasible is it now to run a local indexed web search? Surely running an old school Google is possible with the beefy AI rigs today. I know the problem will be crawling which would be bottlenecked by the ISP but I use Google to search SO, Wikipedia, programming language docs, Github issues, and AWS docs. I think a feasible workflow would be…
Wayback Machine full archive is like less than 50PB. Let's say you could strip multimedia and remove every patterned data to compress that into about a petabyte. The per-bit cheapest disk right now is consumer Seagate 24TB(SI; 21.8TiB usable) at ~$500, or around $1200k for just the disks. Doable if you had couple million dollars to burn. Cheaper than private jets new.
I imagine if set some constraints you could get index size down quite a bit but I still suspect it'd be hard keeping up with content churn.
Edit: Looks like enwiki bz2 is coming in around 46Gi which isn't too bad considering the amount of content it contains.