Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Right Dao, a new independent search engine that doesn't track users

rightdao.com

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Right Dao, a new independent search engine that doesn't track users

#31

Post by ivyabc »

m-i-l wrote: post #13
ivyabc wrote: post #2

(Disclaimer: Self Promotion) Today, Google accounted for 92% of all Internet searches. Google and Bing together occupy 96% of the search market. Most other search engines, including the popular privacy focused ones, simply get the search results from Google or Bing and reorder them, because developing a search engine is neither easy nor cost effective. Consequently, the few big tech companies control what people see.…

Looks promising. But how do we know you are doing your own indexing and not simply buying in the results from Google or Bing like pretty much all of the other search startups? And if you are doing your own indexing, I see you include some very large sites like wikipedia which is going to cost a lot of money to index on a regular basis, so how are you going to pay for this in a sustainable way?

Our results are different from other search engines. Wikipedia has regular database dumps (https://dumps.wikimedia.org/), which is a relatively low cost to index. Overall, our scale is currently small and the cost is manageable.

Re: Right Dao, a new independent search engine that doesn't track users

#32

Post by permo-w »

ivyabc wrote: post #26
NonEUCitizen wrote: post #4

Re: "independent," how are you funded (so you can pay for infrastructure)? Are you independent of VCs? of governments?

Currently our scale is small and the cost is manageable. We are not funded by venture capitals or governments.

Yes, but how are you funded?

Re: Right Dao, a new independent search engine that doesn't track users

#33

Post by kwhitefoot »

If I search for a person by given name and family name with quotation marks around the whole string then I would expect the hits that include the text to be at the top of the list. Google manages this for my name but Right Dao does not.

If I search for my name like this: "kevin whitefoot" the first 27 hits on Google are directly relevant and my name appears in the link or in the extracted text. Right Dao on the other hand returns a list where the most of the hits do not include my name as quoted just the two words separately which means that the hits are completely irrelevant as they refer to a completely different person.

So how does one search for a person by name?

Re: Right Dao, a new independent search engine that doesn't track users

#34

Post by indymike »

jiehong wrote: post #17
indymike wrote: post #16

Earlier quoted context omitted.

Here's the problem with EU regulations: they really do bar small projects and hobbyists and assume that even the smallest website is backed by a corporation with resources to comply with pretty complex rules.

Actually, that's not really true. If you don't store any user information, you're compliant. It's only if you start storing those that you have some rules to follow. Nowadays, it's the same if you are in California with the recent data protection laws. Also, Right Dao is under New York law, so it has to follow US law I guess.

"If you don't store any user information, you're compliant."

So, how do I do business with people?

Re: Right Dao, a new independent search engine that doesn't track users

#35

Post by detaro »

indymike wrote: post #16
jmcs wrote: post #9

"The Services are offered from the United States of America and, regardless of your place of residence or access location, your use of them is governed by the laws of the United State of America. Right Dao makes no representations that the Sites are appropriate for use in other locations or are legal in all jurisdictions. Those who access the Sites from other locations do so at their own risk and consent to the trans…

Here's the problem with EU regulations: they really do bar small projects and hobbyists and assume that even the smallest website is backed by a corporation with resources to comply with pretty complex rules.

For small businesses that don't have a large mess of legacy stuff to clean up, the requirements aren't that bad. Yes, it is extra effort, but mostly documentation, and lots of it can be minimized by keeping as little data as necessary.

Re: Right Dao, a new independent search engine that doesn't track users

#37

Post by dpezely »

ivyabc wrote: post #27
drannex wrote: post #6

Do you have your own indexer or are you relying on others?

We have our own crawler and indexing.

Have you considered using Common Crawl [1], and if so, what was your assessment when compared to having your own spyders?

Long-term, a combination of theirs and your own could be optimal.

There are strengths and weaknesses with using their dumps: on one hand, benefits include them having crawled and having dealt with being throttled, etc. They offer monthly dumps for general content and daily dumps for news [2].

On the other hand, it's a huge pile of data to wade through, and their index format might not be your preferred method. The archive and index reside officially at AWS, so that may decide where to process it. (Not sure whether other providers maintain a copy as well or not.)

By "huge", specifically:

> October 2020 [...] contains 2.71 billion web pages or 280 TiB of uncompressed content.

From our analysis a few years ago, that was to be the approach for the now-defunct Snagz.net [3] (which never fully launched because co-founders were unable to join due to extenuating circumstances).

[1] https://CommonCrawl.org

[2] https://commoncrawl.org/2016/10/news-dataset-available/ - this one can be hard to find unless you know to look for it

[3] https://web.archive.org/web/20180320001756/http://snagz.net/

Re: Right Dao, a new independent search engine that doesn't track users

#38

Post by ivyabc »

ColinHayhurst wrote: post #25
jhoechtl wrote: post #19

I want this to succeed. I have been searching for a while for a search engine which is independent from Google or Bing. First searches have been relevant and also refreshing different and new from the big ones. Battle-testing will tell.

I maintain this list of search engines: https://twitter.com/SearchEngineMap/lists Disclosure: Mojeek team member Happy to add Right Dao but did not find it on Twitter.

Thanks! We have created our Twitter page. https://twitter.com/RightDao

Re: Right Dao, a new independent search engine that doesn't track users

#39

Post by ivyabc »

rstarast wrote: post #20

Impressed by the speed of the results. Some of my more obscure tests didn't give relevant results, but I guess this is still early on. Do you have any numbers about the size of the index, and where you're aiming to go? My big question: What's behind the name? I find it a bit confusing and not very memorable at first sight, maybe an explanation would help.

From Wikipedia: Dao is a Chinese word signifying the "way", "path", "route", "road"... In most belief systems, the word is used symbolically in its sense of 'way' as the 'right' or 'proper' way of existence...

https://en.wikipedia.org/wiki/Tao

Re: Right Dao, a new independent search engine that doesn't track users

#40

Post by ivyabc »

ralphc wrote: post #24

How often is this indexed? I have a great example for today, December 3rd. Yesterday, Salesforce announced a new product, Hyperforce. It's a new way to deploy their product, basically a big deal from a big company. Searching Salesforce Hyperforce gets no results in rightdao, plenty in duckduckgo.

By the way, thanks for your feedback. We have fixed a bug and now news stories are shown on the page.

https://rightdao.com/search?q=Salesforce+Hyperforce

Post reply on HN