131–140 of 396 posts
I still remember helping a friend finding informations on the accounting balance of Rome's, Italy, public transport, and finding the most relevant link buried deep at around page 20. The first 15 pages were almost completely news websites with completely irrelevant news to the search query but they would consistently rank much higher.
It's unfortunate that with all the immense value that search engines provide the idea of paying a small monthly or annual fee to use a search engine is incomprehensible for most people.
Earlier quoted context omitted.
> And since you control what URLs need to be crawled, you protect yourself against rogue clients sending arbitrary URLs. I'm not worried about the URLs, but the content of the URLs sent back. Say the server tells a client to crawl a CNN article. The "hacked" client sends a fake CNN article back.
Get 3 people to scrape it and see if there are significant differences. Some might, because of A/B testing or news updating, but even updating news will get a positive similar page and those that don't should probably fall into an exceptions category until it can be determined what can be done about it. Maybe a flag in the URL to give you a static page or just accept that it changes often enough that even faked pages…
One feature that I really wish more search engines would have is the ability to blocklist certain domains, particularly ones whose results are never relevant or helpful to the query itself (Pinterest, Quora, etc). It could even be used as a factor in the site’s search rankings.
I think kagi does that, I use while was in beta, you also can assign a priority to sites, like normal or boost. Kagi is a paid service and doesn’t shows you ads.
It's really useful!
Earlier quoted context omitted.
Are your sure they're combating it? It seems like they've given up.
Yeah... but if Google is giving up, who would ever be able to do it?
Google's search results are so bad I can't really alledge incompetence here but have to wonder whether there's some different motivation. Maybe it's that low quality search results tend to be plastered with ads, which they get a cut on.
It's unfortunate that with all the immense value that search engines provide the idea of paying a small monthly or annual fee to use a search engine is incomprehensible for most people.
Perhaps a better approach would be building an open source www index or even a full current cache - as an enabler for people to build their own search engines? Right now it is extremely difficult to build your own web crawler that would compete with Google. And that is not because of the technology, but because multiple sites will prevent your bot from accessing them if you're not Google or Bing - either through robo…
Why is https://commoncrawl.org/ not enough?
If I were to make a search engine, I'd definitely give users more control over their results. Block crap sites, vote up your favourite sites, vote down questionable sites, maybe different context profiles, because if you're searching for Java in the context of vacation or news events you want different results than if you're searching for it in a programming context.
There's so much that search could do better than what Google is doing, but I'm not doing it because it's way too much work, and it requires serious resources to index everything.
> Instead of looking at how long people spend on a site, we would encourage users to give explicit feedback on rankings and use this to improve our ranking system. While they're not wrong about how the way Google determines ranking has its issues, this way has its own set of problems. If you explicitly use user ratings as part of your rankings in some way, people can punish sites they don't like, ala review bombing o…
Bombing is one thing, but you also have a whole SEO industry now that will exploit any way possible to get to the top of the rankings. The moment you have community rankings on search, and your search gets popular, you land in a war zone with bots trying to mangle those. Reddit is kind of good dealing with that, but it is very resource intensive.
I use duck duck go. Recently someone showed me their screen where they were using google to do a search. I was absolutely aghast. The last time I used google when you searched for something you saw a simple text list of sites (which is how DDG still works). Instead the google results were… a disaster. You had to scroll through some much garbage before finding actual search results - a list of sites. It was like googl…
I do too. However, I am responsible for a couple of sites that Bing absolutely refuses to even index. Google has no issues with them, not for 10 years running. Those sites are effectively invisible in Bing and therefore DuckDuckGo. I’m not talking about low ranking, the entire domain is ignored. My own personal website for example is not even listed in the search engine I use. Microsoft support see the issue but have…