Well, yes, we can. Just use Google with non-Chrome browser with built-in ad-blocking, like Edge or Brave.
We can do better than DuckDuckGo
201–210 of 383 posts
Re: We can do better than DuckDuckGo
#202Am I the only person who just doesn't have problems with DDG search results? What am I doing wrong (or right), here? I put a thing in and find it. I just don't use Google any more. Genuinely curious why it's working for me and such garbage for everyone else.
You're probably searching for English language articles and are being explicit about what you want.
For example, you might search for `vue js on show` whereas `vue on show` will show you (in the UK) results for what is on at Vue cinemas.
With Google, I expect it would understand that you are probably searching for JS related vue questions and rank those higher.
Re: We can do better than DuckDuckGo
#203Earlier quoted context omitted.
I'm not sure why email should be an example of the correct way to do it. And with email I can check my spam folder and see exactly what has been rejected. So unless the search engine has a list of sites that aren't deemed worthy included with every search (which probably wouldn't happen), I think this solution has some pretty big flaws. It should be noted that the current system also has these flaws, as Google and DD…
I like this idea! Have the usual official results... then have an option to go to level 2, level 3, level 4 etc (lvl 1 is not included in lvl 2) You can have really biased technically terrible filters that for example put a site on level 4 because it is to new, to small and any number of other dumb SEO nonsense arguments. (The topic was not in the url! There was poor choice of text color!) I think wikipedia has a lot…
Per your 2nd para Google used to have some options to tailor the results more, like allinurl or inurl or title or link (IIRC the word had to be in a link pointing to that page) or whatever.
I expected that to evolve to get more specificity but things went completely the otherway and we can't even specify a term is on a page reliably with Google now.
Similarly, I was all in on xhtml and semantics (like microformats) where you'd be able to search for "address: high street AND item:beer with price:<2" to find a cheap drink.
Re: We can do better than DuckDuckGo
#204I tried to build something like this in 2007, together with a small band of nerds and geeks and Linux enthusiasts. It was called Beeseek. [0]
I knew close to nothing about building a company or a project, or how a proper business model would have helped it. I was the leader (SABDFL) of the group, and unfortunately I didn't lead it well enough to succeed. We had some good ideas, but ultimately we failed at building more than the initial prototype.
The idea behind it was simple: WorkerBee nodes (users' computers) would crawl the web, and provide the computational power to run Beeseek. Users could upvote pages (using "trackers" that anonymously "spy" the user in order to find new pages - repeat: anonymously). The entire DB would be hosted across multiple nodes. Auth and other functionalities would be provided by "higher level" nodes (QueenBee nodes). Everything was going to be open source.
Well, it didn't work.
Thankfully, because of Beeseek, I met a few very smart people that I am in touch with to this day.
Life is strange and beautiful in its own way.
Weird, though, that today I still believe that Beeseek could have been the right thing to build. Who knows?
Re: We can do better than DuckDuckGo
#205Earlier quoted context omitted.
> Who chooses the white list, and why should I trust them? Is it democratically chosen? You could have user compiled lists of sites to show in search results. Let the users pick the lists they want to see, and communities can create and distribute lists within themselves.
Great idea, but why build a search engine at all in this case? You can use DDG + your filter and see only the results from your whitelist. Could easily be implemented for any current search engine. To a large extent, this is what you already do when you view a page of search results. Filter them based on your understand of what sites / results hold value.
> why build a search engine at all in this case?
On a public scale, you could make an argument for tighter integration/better privacy with the lists. For example:
Browser -----Request-to-SE-----> Search Engine
^ |
| Unfiltered Results (In YAML/JSON)
| |
| V
|--Desired Results------ Local Filtering/Rendering
Edit: Possibly this could be easier to use to set up distributed search as well, as each node could index a given list, and then distribute that list similarly to DNS. Don't really know how well that would work though, just an idea.
Re: We can do better than DuckDuckGo
#206Earlier quoted context omitted.
Its hard to get readership writing blogs these days. Thats pretty demotivating.
Also difficult to distinguish a blog from a content farm if you are just crawling the web. Any content pattern you select for would likely be quickly adopted by SEOs.
You could use machine learning instead of a hard-coded heuristic.
Re: We can do better than DuckDuckGo
#207> The search results suck
Do they really though, for normal people that is?. Some of my searches today below, can't remember the exact terms I used. Mix of DDG and Google.
1) Walt Whitman, I wanted a basic overview of his work to satisfy some idle curiosity. DDG gave me his wikipedia page. Bingo
2) EAN-13 check digit. First result wikipedia telling me how to calculate it. I see it is simple and I have a long list in Excel to check. I can't be bothered to think so...
3) EAN-13 Excel. First result has an example that I copied and pasted.
4) Timezone [niche cloud system]. Said system didn't do what we expected, seems to be timezone issue. First article is discussing this niche issue and offers solutions
5) Does Shopify support x payments. Yes it does
6) Coronavirus test. Got straight to government site.
7) MacOS version numbers. First hit...
8) How come my Microsoft x platform is showing as being at y level of service when my Buddies is not. Straight in
Am I just a perfect search customer? I don't seem to be getting the problems Drew is?
Re: We can do better than DuckDuckGo
#208Its still pretty new and I'm working on it in my spare time, but my side-project https://searchmysite.net/ seems pretty close to what the author is after:
- "100% of the software would be free software, and third parties would be encouraged to set up their own installations" - I'm planning on open sourcing it under AGPL soon, once I've got documentation, testing etc. ready. Plus it's easy to set up your own installation (git clone; mkdirs for data; docker-compose up -d).
- "I would not have it crawling the entire web from the outset" - That's one of the key features of my approach, only crawling submitted domains. I'm focussing on personal websites and independent websites at the moment, primarily because I don't currently have the money for infra to crawl big but useful sites like wikipedia, but there's nothing to stop people setting up their own instances for other types of site.
- "who’s going to pay for it? Advertisements or paid results are not going to fly" - A tough anti-advert stance is another key differentiating feature to try to keep out spam, e.g. I detect adverts on indexed pages and make sure those pages are heavily downranked. Planning to pay running costs via a listing fee, which gives access to additional features like greater control over indexing (e.g. being able to trigger reindexing on demand).
Re: We can do better than DuckDuckGo
#209I'd love a truly open source world class search engine. Curious how both the crawler and the search index / search is done by the likes of Google/Bing/DDG. Eventually someone will make an oss version of it that can compete. The beauty of such oss solution maybe the custom heuristics that can be created based off the crawled data.
The challenges to OSS developers are numerous. First of all, many popular sites on the internet block crawlers other than Google and Bing, because only those ones seem to matter to their business, and any small upstart would be assumed to be a dodgy bot. Secondly, Google amasses the database it has only with vast data centers, incredible amounts of bandwidth, and power requirements unavailable to a startup.
Just use the string 'googlebot' in your user agent.
After all, Google uses 'Mozilla' in their Google Bot user agent String for similar reasons - because sites might expect it.
Re: We can do better than DuckDuckGo
#210I tried to build something like this in 2007, together with a small band of nerds and geeks and Linux enthusiasts. It was called Beeseek. [0] I knew close to nothing about building a company or a project, or how a proper business model would have helped it. I was the leader (SABDFL) of the group, and unfortunately I didn't lead it well enough to succeed. We had some good ideas, but ultimately we failed at building mo…
In what ways does what OP describes remind you of your project? Just that it was an open source web search?
One difference from what you describe is that the OP is specifically recommending against decentralization/federation, where it seems to have been the core differentiator of your effort. I don't think what OP is describing is quite what you are describing.