Live data from Hacker News

The world needs a non-profit search engine

daoudclarke.net

51–60 of 396 posts

Re: The world needs a non-profit search engine

#51

One feature that I really wish more search engines would have is the ability to blocklist certain domains, particularly ones whose results are never relevant or helpful to the query itself (Pinterest, Quora, etc). It could even be used as a factor in the site’s search rankings.

> It could even be used as a factor in the site’s search rankings.

They did that for a while without considering the obvious downside - the incentive to mass blacklist your competitors.

Re: The world needs a non-profit search engine

#52
post #2

> The technology for organizing the world’s knowledge should be owned by everyone. This is what really nails it for me. There's far too much black box in pretty much every major search engine out there. Maybe it's by design "so that people can't game it". Even so, it's not working very well. I'm excited for the next 10 years to see what we (humans) come up with to solve the state of the internet, because something's…

>.....to see what we (humans) come up with to solve the state of the internet, because something's gonna give at some point.

We (humans)? Are the fish up to something? Never did trust fish, especially regarding SEO stuff.

Re: The world needs a non-profit search engine

#53

I use duck duck go. Recently someone showed me their screen where they were using google to do a search. I was absolutely aghast. The last time I used google when you searched for something you saw a simple text list of sites (which is how DDG still works). Instead the google results were… a disaster. You had to scroll through some much garbage before finding actual search results - a list of sites. It was like googl…

> I use duck duck go. You use a proxied Bing. If you want a proxied Google (what I prefer), you can use https://startpage.com

Oh boy. I loved ixquick/startpage until 2020(?) when it suddenly stopped working and demanded I turn on JS.

I did no such thing but looked around and found the company had been sold to an advertiser/tracker. It used to work much, much better than ddg:(

Re: The world needs a non-profit search engine

#54

One feature that I really wish more search engines would have is the ability to blocklist certain domains, particularly ones whose results are never relevant or helpful to the query itself (Pinterest, Quora, etc). It could even be used as a factor in the site’s search rankings.

I think kagi does that, I use while was in beta, you also can assign a priority to sites, like normal or boost. Kagi is a paid service and doesn’t shows you ads.

Re: The world needs a non-profit search engine

#55
post #20

Never thought real-time searching will be so cool https://mwmbl.org

That’s a terrible name

I disliked it too intially but it had grown on me after a couple of days. As to why the name was chosen and how to pronounce it are in the FAQ [1]

[1] https://github.com/mwmbl/mwmbl#how-do-you-pronounce-mwmbl

Re: The world needs a non-profit search engine

#56
First thing I try is:

Star trek imdb

First result is startrek.com, second result is Star Trek into Darkness IMDb but 3rd is xkcd.

It then goes off into Q and William Shatner Wikipedia links and Muppet Movie IMDB in Russian.

I tried putting a plus in front of IMDB and quoting Star Trek. It doesn't seem to be able to find Star Trek on IMDB. I admire the concept, and it is extremely fast.

Re: The world needs a non-profit search engine

#57
> Instead of looking at how long people spend on a site, we would encourage users to give explicit feedback on rankings and use this to improve our ranking system.

While they're not wrong about how the way Google determines ranking has its issues, this way has its own set of problems. If you explicitly use user ratings as part of your rankings in some way, people can punish sites they don't like, ala review bombing on Yelp, Steam, etc.

Not saying it's necessarily a bad idea because of that, but I hope they don't fall victim to the mentality of, "let's just trust the users" as an ironclad rule, because that doesn't always work out well.

Re: The world needs a non-profit search engine

#58

Please add it to [1] since Firefox (absurdly!) doesn't seem to let us add arbitrary search engines anymore. 1. https://addons.mozilla.org/en-US/firefox/extensions/category...

It does, I just tried it with 102.1. You can add a "keyword" to use a search engine, and once you've done that, you can add it as a search engine to the url+search bar (the main one on top, I don't know how it's actually called) as well, and you can set it as your default search engine.

Re: The world needs a non-profit search engine

#59

Earlier quoted context omitted.

After a certain scale I think you can let clients do double-work and let the most common crawl data, among different clients, win. And since you control what URLs need to be crawled, you protect yourself against rogue clients sending arbitrary URLs. There certainly are a lot of elegant ways to reduce spam for this particular problem imo.

> And since you control what URLs need to be crawled, you protect yourself against rogue clients sending arbitrary URLs. I'm not worried about the URLs, but the content of the URLs sent back. Say the server tells a client to crawl a CNN article. The "hacked" client sends a fake CNN article back.

Get 3 people to scrape it and see if there are significant differences.

Some might, because of A/B testing or news updating, but even updating news will get a positive similar page and those that don't should probably fall into an exceptions category until it can be determined what can be done about it. Maybe a flag in the URL to give you a static page or just accept that it changes often enough that even faked pages won't last long?

Re: The world needs a non-profit search engine

#60

I use duck duck go. Recently someone showed me their screen where they were using google to do a search. I was absolutely aghast. The last time I used google when you searched for something you saw a simple text list of sites (which is how DDG still works). Instead the google results were… a disaster. You had to scroll through some much garbage before finding actual search results - a list of sites. It was like googl…

I do too. However, I am responsible for a couple of sites that Bing absolutely refuses to even index. Google has no issues with them, not for 10 years running. Those sites are effectively invisible in Bing and therefore DuckDuckGo. I’m not talking about low ranking, the entire domain is ignored.

My own personal website for example is not even listed in the search engine I use. Microsoft support see the issue but have no explanation and escalate and quote “quality requirements” for weeks now. A human review yielded positive results regarding quality. Meanwhile, I see lots of SEO spam on DDG when searching for generic technical terms.

Post reply on HN