Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Imagine a search engine that removed top million sites from its index

millionshort.com

Post reply on HN
Print view View original

191–200 of 204 posts

Re: Show HN: Imagine a search engine that removed top million sites from its index

#191

Post by noahc »

dsrguru wrote: post #179
hello_asdf wrote: post #114

Earlier quoted context omitted.

Ah, I was unaware of the !g bang syntax, that's a great one. I'm also fond of !archwiki, and all the language bangs. DuckDuckGo is the only search engine that I know of that could take over a significant portion of search traffic. By significant I don't mean tens of millions, but I mean a dedicated userbase who use it in leue of Google.

Just out of curiosity, what does DuckDuckGo do that conventional keyword searches in Firefox or Chrome don't do?

I have DDG set as my default search engine, but can you do the following:

! fuji Ames => takes me to Fuji Steak House in Ames Iowa directly

!w Sushi => Takes me Directly to the article on Sushi on Wikipedia

!m 801 grand ave Des Moines => takes me directly to google maps at the address.

I still probably do about 30% of my searches using !g kicking me straight to google for search results.

Re: Show HN: Imagine a search engine that removed top million sites from its index

#192

Post by mswen »

I just tried it with a search for some competitive intelligence. I used the 100K removal option. I found a competitor in another country that had not made the top 2 pages on Google. It confirms that others are launching something similar to what I am building... but also the fact that it doesn't bubble to the top on Google means that the market space is not dominated yet.

Re: Show HN: Imagine a search engine that removed top million sites from its index

#193

Post by sparknlaunch12 »

nostromo wrote: post #102

It's interesting to see popularity used as an inverse corollary with quality. Imagine a TV that skipped the most popular programming (goodbye American Idol), or a radio station that only plays non-hits. Of course, there are great websites out there that are very popular (Wikipedia, NYTimes/WSJ, StackOverflow). I'd love to see a search engine with a better signal for quality than non-popularity (this search engine), o…

I think that quality and page rank are not correlated. You may well have some quality sitting anywhere between page 1 and a million. Page rank is driven by various things like freshness and keyword matching. How do you determine quality?

This website is an interesting experiment however if you're after discovering random semi related pages, why not just use google or stumble upon?

Is there a way to include domains automatically excluded? For example WordPress.com is excluded.

Re: Show HN: Imagine a search engine that removed top million sites from its index

#194

Post by shurane »

taxonomyman wrote: post #47
wildmXranat wrote: post #28

Earlier quoted context omitted.

I think search APIs like Yahoo BOSS allow you pass arguments that contain a black list of domains. I think it's the 'sites' argument that may be used like this: &sites=-google.com

You are right, but they won't allow the list to be 1 million sites long. You are talking 15 megs of data per data in plain text per request. But I like the idea of being able for users to, via a setting perhaps, add their own list of deny/include sites. Thx for the comment.

Isn't this under Blekko's domain of ideas with the slashtags letting you include particular sites?

Re: Show HN: Imagine a search engine that removed top million sites from its index

#195

Post by copper »

taxonomyman wrote: post #110
nostromo wrote: post #102

It's interesting to see popularity used as an inverse corollary with quality. Imagine a TV that skipped the most popular programming (goodbye American Idol), or a radio station that only plays non-hits. Of course, there are great websites out there that are very popular (Wikipedia, NYTimes/WSJ, StackOverflow). I'd love to see a search engine with a better signal for quality than non-popularity (this search engine), o…

Maybe think of Million Short as more of a discovery engine. We're not saying a quality site can't be popular.

I think Fravia+ would have actually liked what you have here - its a really nice search engine.

Re: Show HN: Imagine a search engine that removed top million sites from its index

#196

Post by dsrguru »

noahc wrote: post #191
dsrguru wrote: post #179

Earlier quoted context omitted.

Just out of curiosity, what does DuckDuckGo do that conventional keyword searches in Firefox or Chrome don't do?

I have DDG set as my default search engine, but can you do the following: ! fuji Ames => takes me to Fuji Steak House in Ames Iowa directly !w Sushi => Takes me Directly to the article on Sushi on Wikipedia !m 801 grand ave Des Moines => takes me directly to google maps at the address. I still probably do about 30% of my searches using !g kicking me straight to google for search results.

But how is that different from regular keyword searches? If I want to pull up the Wikipedia article on sushi in Firefox, I enter "w sushi" in the URL bar. If I want to read the Arch Linux wiki article on pacman, I type "arch pacman". If I want to watch Friday by Rebecca Black on Youtube, I type "y Friday", etc. I feel like there's more than a 50% chance that I'm missing something. What does DDG do that browsers don't?

Re: Show HN: Imagine a search engine that removed top million sites from its index

#198

Post by lubujackson »

hsparikh wrote: post #112

As someone learning web development, I'd love to get some insights into how one could build this.

Not knowing anything about what they do, the hack-ish way you could do it is to use Google CSE (custom search engines) to add a list of negative domains. Where to get the list of top 1 million domains? Probably from Quantcast here: http://www.quantcast.com/top-sites-1

Re: Show HN: Imagine a search engine that removed top million sites from its index

#199

Post by tripzilch »

copper wrote: post #195
taxonomyman wrote: post #110

Earlier quoted context omitted.

Maybe think of Million Short as more of a discovery engine. We're not saying a quality site can't be popular.

I think Fravia+ would have actually liked what you have here - its a really nice search engine.

He called it the "Yoyo" technique[1]. It doesn't work very well anymore, because Google's results are all quirky these days (you don't actually get what you literally search for, but what Google guesses what you intend to find).

[1] http://www.searchlores.org/yoyo1.htm (try to read past the preachy anti-commercialism, he could get kind of hot about that--whether you agree or not, there's troves of knowledge to be gained from that site)

Re: Show HN: Imagine a search engine that removed top million sites from its index

#200

Post by tripzilch »

dsrguru wrote: post #179
hello_asdf wrote: post #114

Earlier quoted context omitted.

Ah, I was unaware of the !g bang syntax, that's a great one. I'm also fond of !archwiki, and all the language bangs. DuckDuckGo is the only search engine that I know of that could take over a significant portion of search traffic. By significant I don't mean tens of millions, but I mean a dedicated userbase who use it in leue of Google.

Just out of curiosity, what does DuckDuckGo do that conventional keyword searches in Firefox or Chrome don't do?

Of course you can put all those keyword searches into your browser. The only difference is that DDG has already made a huge selection, so you don't have to, anymore.

So basically, DDG probably has a whole bunch of !bang searches that you simply had not thought of to create your own keywords for, yet. And when you need it, it's already there.

There are also a few !bang queries that are not external searches, such as one for rolling dice (it can do !roll 3d6+3).

Another minor difference is that you can add the !bang keyword anywhere inside the query, also at the end. Being able to add it at the end makes it easier to "ok let's try this on another search engine".

Post reply on HN