Ask HN: Is there a search engine which excludes the world's biggest websites? • Hacker News Board

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#201

Post by egberts1 » Sun, May 17, 2020, 10:21 AM UTC

COVIDisntCORONA wrote: post #134

tacon wrote: post #130

Earlier quoted context omitted.

>SARS-CoV-2: strain of SARS-CoV Excuse the incivility, but no. SARS-CoV-2 is not a strain or type of SARS-CoV. The viruses share ancestors, but SARS-CoV-2 did not come directly from SARS-CoV. SARS-CoV and SARS-CoV-2 are in the category of beta coronaviruses[0]. "The whole genome-based phylogenetic analysis presented that two Bat SARS-like CoVs (ZXC21 and ZC45) were the closest relatives of SARS-CoV-2."[1] [0] https:/…

Excuse the incivility once again, but no. While we're on the topic of linguistic pedantary, strain isn't exclusive to direct mutations from a parent genome. Strains, like much of biological taxonomy, are a human abstraction to make communication of the idea of -- in this case -- "a virus sharing similar properties to coronaviruses that cause severe acute respiratory syndrome" -- albeit this is a very simplified defin…

Wow. A civil exchange on proper use of terms by two disparate but related technological domain experts. It’ll go on forever.

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#202

Post by egberts1 » Sun, May 17, 2020, 10:24 AM UTC

What we all really need is a long-tail Bloom filter search engine on the search engines themselves.

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#203

Post by EmilioMartinez » Sun, May 17, 2020, 12:27 PM UTC

chaos_a wrote: post #7

https://wiby.me/ exists to solve this exact problem. I've found some pretty neat/odd websites on it in the past.

Just added wiby.me/surprise to the bookmarks bar. Amusingly, the icon keeps changing every time I use it.

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#204

Post by tomaskafka » Sun, May 17, 2020, 12:40 PM UTC

dorkwood wrote: post #89

noad wrote: post #4

This is a great question, I also want a way to search the internet but exclude all major media domains as well as any company over a certain size. So I just want to search through old blogs, SO, non-corporate social media, weird forums, etc. There are so many cool things I remember reading on the web like 10-20 years ago that still exist that are so buried now on Google they might as well not exist. Nowadays searchin…

For years, my trick for finding interesting content was to go to, say, the 7th page of Google results, and start there. This doesn't work anymore -- it's SEO-optimised listicle blog posts all the way down. My trick now is to use Twitter to discover interesting people, and follow them there. Granted, it's not a search engine, but it's at least given me the ability to discover weird things again.

Many times I have thought about what could happen if Twitter asked you to recommend up to 3-5 people you value, and write a tweet-sized (or shorted) recommendation.

Over time you would get a 'pagerank for people' and could do awesome stuff with that, like 'You don't know XYZ, but 3 people you trust trust her, and this is what they tell about her:' ...

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#205

Post by halfdan » Sun, May 17, 2020, 2:17 PM UTC

tomaskafka wrote: post #204

dorkwood wrote: post #89

Earlier quoted context omitted.

For years, my trick for finding interesting content was to go to, say, the 7th page of Google results, and start there. This doesn't work anymore -- it's SEO-optimised listicle blog posts all the way down. My trick now is to use Twitter to discover interesting people, and follow them there. Granted, it's not a search engine, but it's at least given me the ability to discover weird things again.

Many times I have thought about what could happen if Twitter asked you to recommend up to 3-5 people you value, and write a tweet-sized (or shorted) recommendation. Over time you would get a 'pagerank for people' and could do awesome stuff with that, like 'You don't know XYZ, but 3 people you trust trust her, and this is what they tell about her:' ...

Until that gets screwed using social engineering and people optimising for being affiliated with people writing them good reviews.

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#206

Post by Theodores » Sun, May 17, 2020, 2:31 PM UTC

teamspirit wrote: post #84

ganstyles wrote: post #58

Earlier quoted context omitted.

Giving you the benefit of the doubt, and assuming this isn't just pedantry, especially since you're getting downvotes (because I assume everyone thinks this is just pedantic correction) I looked it up. In the context of "trying to do research on coronaviruses" your comment appears to be not only correct but an important distinction, rather than the pedantry it appears to be. From Wikipedia: "...more lethal varieties…

Further, CoViD-19 literally stands for: Co[rona] Vi[rus] D[isease] - [discovered in 20]19 - "D" standing for disease caused by this particular strain. To be honest, I was a bit disappointed when I found out, though I admit now it's a little refreshing to have be so simply named.

Which can be further abbreviated as C19. I have seen this in personal chats and wonder how long it will be before it gets into newspaper headlines where space is at a premium in print editions.

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#207

Post by netsectoday » Sun, May 17, 2020, 2:40 PM UTC

You can boot up your own custom search engine in a few minutes with YaCy (Ya See!) an open-source, P2P, Dockerized crawler and search engine built on top of Solr.

https://yacy.net/

If you're generous; you can make your index available to other P2P instances.

I wanted to run an API search the other week and was blown away with how quickly I could prop-up my own custom search portal (I didn't want to pay for API access to other search engines, and YaCy comes with a JSON and Solr endpoints).

I ran it locally to test my crawl filters, then pushed a private instance out to Digital Ocean to turn up the heat with the crawling. The only issue I had was the crawler would hit the max memory threshold on long crawls and the container would restart, but that was fixed by scaling up the box.

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#208

Post by fedede » Sun, May 17, 2020, 2:57 PM UTC

Hey! I actually liked this idea and I'm considering starting a learning project on it. I've seen a lot of interest and ideas in the comments, and decided to create a very short Google form to start gathering all the interested people so we can organise something interesing. Is anybody in? :)

https://forms.gle/5KuTYVdYaMzRD2n78

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#209

Post by scottlocklin » Sun, May 17, 2020, 3:00 PM UTC

chris_f wrote: post #67

noad wrote: post #4

This is a great question, I also want a way to search the internet but exclude all major media domains as well as any company over a certain size. So I just want to search through old blogs, SO, non-corporate social media, weird forums, etc. There are so many cool things I remember reading on the web like 10-20 years ago that still exist that are so buried now on Google they might as well not exist. Nowadays searchin…

If you are looking for a forum search, check out https://boardreader.com/ . I have a theory that web crawling alone is not the best way forward to find the most relevant results because of the volume of content continually being created, much of which is niche and sometimes dynamic. Instead I believe linking together vertical search sources that have targeted information based on search intent will provide better res…

This is really good!

https://www.runnaroo.com/search?term=J+apply+verb

versus google's completely useless:

https://www.google.com/search?safe=active&q=J+apply+verb&oq=...

Everyone else, get in here: this is top notch stuff.

Re: Ask HN: Is there a search engine which excludes the world's biggest websites?

#210

Post by fedede » Sun, May 17, 2020, 3:03 PM UTC

noad wrote: post #4

This is a great question, I also want a way to search the internet but exclude all major media domains as well as any company over a certain size. So I just want to search through old blogs, SO, non-corporate social media, weird forums, etc. There are so many cool things I remember reading on the web like 10-20 years ago that still exist that are so buried now on Google they might as well not exist. Nowadays searchin…

Hey! I actually liked this idea and I'm considering starting a learning project on it. I've seen a lot of interest and ideas in the comments, and decided to create a very short Google form to start gathering all the interested people so we can organise something interesing. Is anybody in? :)

https://forms.gle/5KuTYVdYaMzRD2n78