Any suggestions on how could one start an open source search engine business?
Being able to index the whole internet as an MVP and quickly serve queries on top of this data has a major initial and operating cost.
How is search so bad? A case study
231–240 of 416 posts
Re: How is search so bad? A case study
#232I increasingly just use google to search sites I already know have better content than the web at large, since it so often feels like it sucks for any depth of information.
Want to get in touch with someone? name job site:linkedin.com. Want to find how to solve a tech issue? "issue" site:stackoverflow.com, and so on.
Google's search of sites like this is pretty good (although the recency working well would be really good... but perhaps impossible to solve well), and often better than in site search. But that + very basic fact finding "how much is a lb in kg", "where is restaurant X" are pretty much all I feel it's good for. Then again, I guess it's not supposed to be an encyclopaedia (or it can be, site:wikipedia.org!)
Re: How is search so bad? A case study
#233What was the query the author used? Did he try something like this
"https://www.google.com/search?sitesearch=reddit.com&q=best+p...
Re: How is search so bad? A case study
#234What was the query the author used? Did he try something like this " https://www.google.com/search?sitesearch=reddit.com&q=best+p...
Watch out with doing more complicated searches like extending those to a date range - they get people “banned” from using google search.
Re: How is search so bad? A case study
#235Earlier quoted context omitted.
The real reason why search is so bad is that Google is downranking the internet. I should know - I blew the whistle on the whole censorship regime and walked 950 pages to the DOJ and media outlets. --> zachvorhies.com What did I disclose? That Google was using a project called "Machine Learning Faireness" to rerank the entire internet. Part of this beast has to do with a secret Page Rank score that Google's army of w…
> Despite the fact that I've had around 50 interviews and countless articles written about my disclosure, my website zachvorhies.com doesn't show up on Google's search index, even when using the exact url as a query! I am not sure what is happening but I directly searched for your website : zachvorhies.com on Google (in Australia, if that matters), which returned the website as the first result: https://i.imgur.com/Z…
I'm in the US and Google does not display zachvorhies.com when searching for "zachvorhies.com".
Re: How is search so bad? A case study
#236I have been thinking about the same problem since a few weeks. The real problem with search engines is the fact that so many websites have hacked SEO that there is no meritocracy left. Results are not sorted based on relevance or quality but by SEO experts' efforts at making the search results favor themselves. I can possibly not find anything deep enough about any topic by searching on Google anymore. It's just surf…
> It's just surface-level knowledge that I get from competing websites who just want to make money off pageviews. Can you give some examples of queries/topics? Not that I disagree, I often have the same problem, but have found ways to mitigate.
Can you elaborate some of these ways? I just have a big minus list of sites.
Re: How is search so bad? A case study
#237Earlier quoted context omitted.
> What we might need to break this is ... Create a core protocol at the same level as DNS etc., that web servers can use to offer an index of everything they serve/relay. A multitude of user-side apps may then query that protocol, with each app using different algorithms, heuristics and offering different options.
I've been thinking along similar lines for a year or so now. There are several puzzling omissions from Web standards, particularly given that keyword-based search was part of the original CERN WWW discussion: http://info.cern.ch/hypertext/WWW/Addressing/Search.html IF we had a distributable search protocol, index, and infrastructure ... the entire online landscape might look rather different. Note that you'd likely n…
I too have been thinking about these things for a long time, and I also believe a better future is going to include "aggregation of such blocklists would, of course, be of some use, and facilitating this is an interesting challenge."
I decided it is time for us to have a bouncer-bots portal (or multiple) - this would help not only with search results, but also could help people when using twitter or similar - good for the decentralized and centralized web.
My initial thinking was these would be 'pull' bots, but I think they would be just as useful, and more used, if they were perhaps active browser extensions..
This way people can choose which type of censoring they want, rather than relying on a few others to choose.
I believe creating some portals for these, similar to ad-block lists - people can choose to use Pete'sTooManyAds bouncer, and or SamsItsTooSexyfor work bouncer..
ultimately I think the better bots will have switches where you can turn on and off certain aspects of them and re-search.. or pull latest twitter/mastodon things.
I can think of many types of blockers that people would want, and some that people would want part of - so either varying degrees of blocking sexual things, or varying bots for varying types of things.. maybe some have sliders instead of switches..
make them easy to form and comment on and provide that info to the world.
I'd really like to get this project started, not sure what the tooling should be - and what the backup would be if it started out as a browser extension but then got booted from the chrome store or whatever.
Should this / could this be a good browser extension? What language / skills required for making this? It's on my definite to do future list.
Re: How is search so bad? A case study
#238Earlier quoted context omitted.
The real reason why search is so bad is that Google is downranking the internet. I should know - I blew the whistle on the whole censorship regime and walked 950 pages to the DOJ and media outlets. --> zachvorhies.com What did I disclose? That Google was using a project called "Machine Learning Faireness" to rerank the entire internet. Part of this beast has to do with a secret Page Rank score that Google's army of w…
> Despite the fact that I've had around 50 interviews and countless articles written about my disclosure, my website zachvorhies.com doesn't show up on Google's search index, even when using the exact url as a query! I am not sure what is happening but I directly searched for your website : zachvorhies.com on Google (in Australia, if that matters), which returned the website as the first result: https://i.imgur.com/Z…
[removed, for some reason I thought australia was part of the EU. I'm in the US, also did not see the site link through google]
Re: How is search so bad? A case study
#239Earlier quoted context omitted.
That's the problem that web directories solve. It's not that you're wrong, it's just largely orthogonal to the problem that you'd need a large crawl of the internets for, i.e. spotting sites about X niche that you wouldn't find even from other directly-related sites, and that are too obscure, new, etc. to be linked in any web directory.
That's the problem that web directories solve Not really. A web directory is a directory of web sites. I can't search a web directory for content within the web sites, which is what a niche search engine would do.
On the other hand, the niche search engine depends upon having such a web directory (the list of sites to index).
Re: How is search so bad? A case study
#240This should probably be a separate submission but why is search so bad everywhere? - Confluence: Native search is horrible IME - Microsoft Help (Applications): .chm files Need I say more. - Microsoft Task Bar: Native search okay and then horrible beyond a few key words and then ... BING :-( - Microsoft File Search: Even with full disk indexing (I turned it on) it still takes 15-20 minutes to find all jpegs with an SS…
Anything Microsoft other than Office+outlook sucks. I don't know about azure though as I have not endured it yet.
Adobe wants to have you by your balls the moment you install their installer :-) I keep a separate computer for Adobe stuff just for that reason. Actually to run some MS junk too.
>Seriously though with all the interview code tests bubble sort, quick sort, bloom filters, etc. Why can't companies or even websites get this right?
I have see some of the stinkiest stuff created by people who will appear smartest in any test these companies can throw at them. Some people are always gambling/gaming and winging it. They leave a trail...unfortunately.