Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The world needs a non-profit search engine

daoudclarke.net

Post reply on HN
Print view View original

381–390 of 396 posts

Re: The world needs a non-profit search engine

#381

Post by rntksi »

freediver wrote: post #306
rntksi wrote: post #293

Earlier quoted context omitted.

Here's a search engine that I'm subscribed to: https://kagi.com In the 20-30 searches that I do in a day, I still have to google about half of them. Either because it's stuff Google does well (currency conversion, for example), or Kagi just doesn't get what I'm trying to search. I remember starting out with the Internet searching on Altavista and Yahoo and Lycos. The information that was present was nowhere near as n…

> In the 20-30 searches that I do in a day, I still have to google about half of them. Either because it's stuff Google does well (currency conversion, for example), or Kagi just doesn't get what I'm trying to search. Currency conversion is not technically a search. It is question answering and Kagi capabilities are still being built. Google only has a 20 year headstart. Can you report all such cases to kagifeedback.…

Thanks, currency conversion was an example off the top of my head only. I am active on orionfeedback and kagifeedback, I find that they're really prompt and effective in answering to feedback.

The other examples are a bit harder to describe and I can't quite describe how Google gets it right. I think I might need more time to describe it out, as it involves search in another language.

Re: The world needs a non-profit search engine

#382

Post by [deleted] »

y42 wrote: post #9

I totally like the idea but I dare to doubt that this would solve the SEO problem. Website owners who are participating in those notorious affiliate programs or earn money with ads will still use the search engine to drag people onto their generic websites, using methodolgies to fit the search engines ranking mechanisms, no matter if they are public or not. SEO and all it's results seem to be immanent to the system.

[deleted]

Re: The world needs a non-profit search engine

#383

Post by account42 »

Nextgrid wrote: post #315
y42 wrote: post #9

I totally like the idea but I dare to doubt that this would solve the SEO problem. Website owners who are participating in those notorious affiliate programs or earn money with ads will still use the search engine to drag people onto their generic websites, using methodolgies to fit the search engines ranking mechanisms, no matter if they are public or not. SEO and all it's results seem to be immanent to the system.

> notorious affiliate programs Great, detect the affiliate links and downrank them. Ban it completely if the problem is severe or the products are fake/etc. > earn money with ads Similar, detect them and downrank. If a particular ad network is knowingly malicious, ban it entirely.

The best thing about this solution is that its an instant competitive advantage that Google will not be able (or willing) to copy.

Re: The world needs a non-profit search engine

#385

Post by Brybry »

liftm wrote: post #289
Brybry wrote: post #251

Earlier quoted context omitted.

20% to 40% of internet users use an adblocker depending on whose statistics you use.[1][2][3] SEO is basically another form of advertising, so evidence suggests to me that people would use something like this. [1] https://en.wikipedia.org/wiki/Ad_blocking#:~:text=users [2] https://earthweb.com/how-many-people-use-ad-blockers/ [3] https://www.insiderintelligence.com/insights/ad-blocking

A quick, entirely unrepresentative look at the user count for uBlock Origin and uBlacklist in Firefox make me somewhat less optimistic than you are. Or is there a more popular way of blocking sites from search results than uBlacklist out there, which I simply don't know about?

uBlock Origin and Adblock Plus are the most popular extensions on Firefox, with each around 5 to 5.5 million average daily users over a whole year[1].

I believe the addon page is average daily users over a week or 6 days?[2]

Firefox monthly active users (unique over 28 days) is around 200 to 210 million worldwide (26 to 22 million in the USA)[3].

It's probably wrong to compare those numbers but a very naive 200 / 28 = 7.41 million average unique users per day. 5 / 7.41 = 67% (probably wrong)

[1] https://addons.mozilla.org/blog/firefoxs-most-popular-innova...

[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1168642

[3] https://data.firefox.com/dashboard/user-activity

Re: The world needs a non-profit search engine

#386

Post by hrbf »

Semaphor wrote: post #376
hrbf wrote: post #353

Earlier quoted context omitted.

Sure. A very simple one would be https://ipbl.herrbischoff.com , a public blocklist page referencing a resource used by a couple dozen users. The HTML doesn't get a lot simpler than that and is entirely valid markup. It's (unsurprisingly) low ranked in Google but it's there. Not so in Bing, it's simply not there. The page exists since March 2021. Bing Webmaster Tools reads "Discovered but not crawled. URL cannot appe…

This is fascinating, thanks. Have you experimented on allowing the bing ad bot that you have blocked? If they have some kind of retaliatory non-crawling?

Interesting theory. But the IPBL doesn’t even have a robots.txt and a different, larger site from a German celebrity I host does have the same directives and is indexed, although incompletely.

My working theory is that Bing’s selection algorithm is biased towards large and already popular sites. In the server logs, I don’t see Bing even attempting to crawl the sites I mentioned, except requesting robots.txt and the root page. Bing appears to be excruciatingly slow to update anything but high traffic sites.

Again, Microsoft Support was unable to explain this behavior even after manual, human review found everything to be in order.

I tried deleting robots.txt entirely and got only Chinese crawlers and SEO bots, but still no Bing crawl. All organic traffic comes from blogs linking directly and Google.

Re: The world needs a non-profit search engine

#387

Post by hrbf »

javajosh wrote: post #347
ItsMonkk wrote: post #339

Earlier quoted context omitted.

Yep, you have outlined the problems with grants. As they look forward, and due so in a one-time lump sum, they hold considerable risk, and thus the providers of the grant money need to ensure they pick correctly, but this just leaves us with a centrally controlled economy and bureaucratic nightmare. Instead, you should seek to discover the intrinsic value as it is occurring, and therefore no risk to the granters. I b…

> a modified Vickrey auction So, fund people based on github stars. I could think of worse solutions, honestly! Oh, something else I remembered: Princeton's Institute for Advanced Study! This is an interesting idea. Basically they tell a scientist "You've changed human history, the least we can do is give you an income for the rest of your life that has no strings attached. You can teach, or not, research, or not." I…

Given that I have a GitHub repository with 26k+ stars, what do you think an equivalent amount of money per month that should work out to?

Re: The world needs a non-profit search engine

#388

Post by DisjointedHunt »

6510 wrote: post #226
DisjointedHunt wrote: post #209

Earlier quoted context omitted.

I respect that. I guess my point is "Grow and change with a decent understanding of what the present state enables" A search engine like Google isn't just a search engine as the author describes it. It is a very integral part of the economy of the internet and just labelling a simplistic interpretation of the present state as "Evil" with an academically poor write up of what a viable alternative is does little good.

I want to write articles and read articles written for me by others. Ideally as few as possible should profit from this process. As google is now a turd, not just no longer capable of delivering this service but actively destroying the good part of the web by refusing to index it. It is my attention, it doesn't belong to anyone else. My access to information and educated opinion is a far more integral part of the gre…

Google is a catalogue. There is no physical analogy that comes close.

Your point about wanting to read articles written for you by others is certainly possible. The very fact that such a desirable outcome drives you to Google and nowhere else should suggest the complexity of the problem they’re solving and how there isn’t really anything else out there doing so well.

>Ideally as few as possible should profit from this process.

Why?

When you accepted a job offer in the software industry, did you stipulate that your mission is to write code for your employer and you will be charging as little as possible for that privilege? Minimum wage should get you by just fine, right?

I hate fully grown adults behaving as though anyone except them making a profit is somehow evil.

Re: The world needs a non-profit search engine

#389

Post by DisjointedHunt »

Nextgrid wrote: post #311
DisjointedHunt wrote: post #161

Earlier quoted context omitted.

And that money goes to support many industries beyond search itself. The author really needs to get off a computer for a minute and understand the economics of the web as it stands today and how the free ad supported model supports millions of peoples livelihoods before jumping to "This is all bullshit"

> the free ad supported model supports millions of peoples livelihoods That doesn't necessarily make it right. Online scamming also supports a lot of people's livelihoods.

One is a business model, one is a crime and fraud.

I’d expect you to know the difference.

Re: The world needs a non-profit search engine

#390

Post by abenga »

amelius wrote: post #259
rocketbop wrote: post #203

Earlier quoted context omitted.

Personally I don't ask my network or friends for options on such things because people tend to have a positive bias towards things we have invested money in. Interestingly summarized on a Lifehacker some time ago. https://lifehacker.com/the-psychology-of-a-fanboy-why-you-ke... I like my friends but I don't want to get my news or politics or shopping advice from them.

The question is: is that worse than the advice from marketeers?

Marketers' information would have a known bias that you can mentally correct for.
Post reply on HN