Live data from Hacker News

The world needs a non-profit search engine

daoudclarke.net

261–270 of 396 posts

Re: The world needs a non-profit search engine

#261
post #260

With all the great progress in large language models lately, and them being excellent text compressors, I've started to wonder if you couldn't just replace a search engine with a like 100mb file full of weights that let you query essentially google scale results except all locally.

YALM is 200GB and require 200GB of GPU Memory to run....

Re: The world needs a non-profit search engine

#262
post #260

With all the great progress in large language models lately, and them being excellent text compressors, I've started to wonder if you couldn't just replace a search engine with a like 100mb file full of weights that let you query essentially google scale results except all locally.

YALM is 200GB and require 200GB of GPU Memory to run....

Yeah you picked the hugest SOTA one of them all, but there's smaller ones like this https://bellard.org/libnc/gpt2tc.html that run well even on CPUs that might run well fine tuned specifically on search results (or at least just code queries).

Re: The world needs a non-profit search engine

#263
post #119

Earlier quoted context omitted.

> and wrap it in ads, and derive content from it i am probably missing something but can you give an example where this happens?

E.g. on Google if you search for "how to tie a tie", a little info box may pop up with step by step instructions. This content is taken from some website, but that website gets no page hits or ad revenue. Instead, Google gets to serve ads on the search engine results page. (I don't know if this happens for this specific example, but Google does this for some searches)

Ok, I just tried but don't see that info box but that is exactly what i was asking for. Thank you very much, did not think of that.

Re: The world needs a non-profit search engine

#264

Earlier quoted context omitted.

Allow users to blacklist sites. Share blacklisted sites. Have the option of instead of hiding blacklisted sites entirely - show them in a different column. Have 4 different types of list. Whitelist - highest scoring Not listed - these websites are not ranked. Yellowlist - show but keep in a separate column Blacklist - don't show https://search.brave.com/help/goggles is interesting.

I found the thought of adding a "yellowlist" funny, as some people are starting to find the use of "whitelist" and "blacklist" racist.

The proper terms now are allowlist and blocklist

Re: The world needs a non-profit search engine

#265
post #235

Earlier quoted context omitted.

99%+ of people wouldn't use this, they would just try the search engine, see that the results suck, then stop using it. It could be useful for power users, though.

99%+ of people in the medical industry did not sanitize hands or equipment 100 years ago. 99%+ of people in the tech industry currently do not care to do the extra steps required for data neutrality, and privacy. 99%+ of people are lazy to the point of harming themselves and others. 1%- of people examine how the 99%+ do things and pioneer harm reduction tactics in spite of everyone constantly reminding them that no o…

99% of analogies are correct

Re: The world needs a non-profit search engine

#266
post #155

Disclaimer: I work for Google, though far away from Search. Regardless of search engine design, there's HUGE money in SEO. Any successful search engine will be gamed. Do you have the developer power to go red-queen against all the large companies in the world?

For clarification, “red queen” means a conflict between two or more entities where the cost of engagement grows, but the relative advantage does not change. Simply put, search engines have been at war with SEO for over 30-years, which has significantly raised the bar not only being a search engine, but producing content; not to mention knowing how to search for information. With the introduction of machine generated…

To your last point, I don’t quite agree. Google’s incentives are misaligned such that keeping you on Google.com just a little longer is better than not because you are more likely to click on an ad.

But also yes the users and the UI both fail. When I used to search for something I would type in something like “gutter clog clean” but slowly started noticing that Google likes longer sentences like “how do I clean a clog in my gutters?”. In pursuit of making Knowsmore (from Ralph Breaks the Internet), Google lost the power user features. Search would be infinitely better if they actually fucking respected literal mode and stopped trying to treat me like an idiot with no attention span. Having search results that contain one out of like 8 words in my query and asking me if I want to include others and then when I say I do still showing me results without them is broken UI and not a user problem.

Re: The world needs a non-profit search engine

#267

This seems really really naive. Do you really think a non-profit is going to fight the hordes of spammers, scammers, seo masses, mechanical turk hordes, etc that are going to game your system?

Not sure I missed the sarcasm here. Why would a non-profit not fight spam etc.? Why would a for-profit fight spam do so? I see the largest internet companies, including Facebook, Twitter, and also Google, fight spam and other harmful content only to the degree absolutely necessary to stay somewhat usable. Which makes sense because it's costly and does not generate profit. I would expect a non-profit, however, to focu…

The SEO operators have a profit incentive. The people who run the non-profit search engine lack a profit incentive.

There are very few incentives better than profit..This is going to be a US v/s USSR fight during the cold war..

Re: The world needs a non-profit search engine

#269

One feature that I really wish more search engines would have is the ability to blocklist certain domains, particularly ones whose results are never relevant or helpful to the query itself (Pinterest, Quora, etc). It could even be used as a factor in the site’s search rankings.

Have a look at Brave Goggles: https://search.brave.com/goggles

Re: The world needs a non-profit search engine

#270

Earlier quoted context omitted.

Allow users to blacklist sites. Share blacklisted sites. Have the option of instead of hiding blacklisted sites entirely - show them in a different column. Have 4 different types of list. Whitelist - highest scoring Not listed - these websites are not ranked. Yellowlist - show but keep in a separate column Blacklist - don't show https://search.brave.com/help/goggles is interesting.

99%+ of people wouldn't use this, they would just try the search engine, see that the results suck, then stop using it. It could be useful for power users, though.

Why should we care about 99% of people? They are people who upload all their personal data to social networks agreeing to "we can do whatever we want" terms, they pay with a card, use Apple's and Microsoft's spyware (which marketing people call "telemetry") ridden operating systems, and install Chrome that sends a signal to Google every time they open a new tab and sends data about every form on every website they visit (which Google developers call "crowdsourcing" in the code [1]).

Make a customizable and privacy-respecting search for us, power users.

Also, I have noticed that Firefox internal search works good for sites you had visited. So when I want to visit a page I have seen earlier, I can go straight to it skipping Google.

Also, you can click on any search box and add it with a prefix, so that you can search MDN or Wikipedia directly, again, without informing Google.

[1] https://source.chromium.org/chromium/chromium/src/+/main:com...

Post reply on HN