91–100 of 383 posts
I think this person actually means "We can imagine doing better than DuckDuckGo".
Well, ideas are much easier than implementations.
They wrote their own search engine.
They closed shop earlier this year.
Yes, we can do better than DDG. But if you are expecting to fund a real search engine with a few hundred thousand dollars you are insane. It will take a ton of development and a ton of hardware to create an index that isn't a pile of garbage. This isn't 2000 anymore. You need to index >100 billion pages and you need it updated and you need great crawling and parsing and you need great algorithms and probably an entir…
> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…
I think Google sort of takes into account "votes", in that they look at the last thing you clicked on from that search, and consider that the "right answer", which they then feed back into their results. As such, they effectively have a list of "tier 1" domains.
Any system that ranks things purely based on votes or view counts can have a feedback loop that can amplify "bad" results that happen to get near the top for whatever reason. For web search, this would encourage results that look right from the results page, even if they're not actually a good result of what the user is looking for.
An example of this would be when you're trying to find an answer to a specific question like "How do I do X when Y?". The best result I'd hope for is a page that answers the question (or a close enough question to be applicable), while the promising-looking-but-actually-bad result is a page where someone asks the exact same question but there are no answers.
> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…
There is a DuckDuckGoBot and I think it was an interview or podcast Gabriel did a while back that he mentioned they use it for filling out gaps in the Bing API data to provide the instant answers, favicons. Their preference for the instant answers were authoritative references such as docs.python.org. This would have been a while back though.
> they’ve demonstrated gross incompetence in privacy Not sure I buy the example that is given here. 1. It's an issue in their browser app, not their search service. 2. It's not completely indefensible: it allows fetching favicons (potentially) much faster, since they're cached, and they promise that the favicon service is 100% anonymous anyway. 3. They responded to user feedback and switched to fetching favicons loca…
I think Google sort of takes into account "votes", in that they look at the last thing you clicked on from that search, and consider that the "right answer", which they then feed back into their results. As such, they effectively have a list of "tier 1" domains.
> Second, we measure engagement of specific events on the page (e.g. when a misspelling message is displayed, and when it is clicked). This allows us to run experiments where we can test different misspelling messages and use CTR (click through rate) to determine the message's efficacy. If you are looking at network requests, these are the ones going to the one-pixel image at improving.duckduckgo.com. These requests are anonymous and the information is used only by us to improve our products.
The Firefox network logger does show requests to this domain when I click on a link in the search results, before the page navigates away. This suggests to me they might by logging this information. To be clear, this is speculation on my part, because I haven't examined the URL parameters in detail.
In any case, I'm not sure how much this manages to improve the results, since usually I can get help with my Python query (for example) using whatever crappy blog post is first in the results, but results from the official docs or StackExchange are still probably better and should be prioritized.
https://github.com/jmqd/folklore.dev
It's not even really at the POC stage yet, but I hope to host it with a simple web frontend sometime soon. Primarily, this is just for myself... I just want a good way to search the sources that I myself trust.
My main problem with DDG is that there's no way to be sure they actually respect their users' privacy as they claim to. Ideally, services like theirs would be continuously audited by respectable, trusted organizations like the EFF.. multiple such organizations even. Then I'd have at least some reason to believe their claims of not collecting data about me. As it stands, I only have their word for it.. which in this d…
Facebook and Google are huge, global companies where their main product is free, and yet they aren't a charity. The only way to be mega-rich and offer something free is to be shady and manipulative with user's data. Exploiting privacy is their business model. They aren't gonna respect it. Being super financially successful off free products and services is not a recipe for an honest, citizen respecting company.
> Instead, it should crawl a whitelist of domains, or “tier 1” domains. These would be the limited mainly to authoritative or high-quality sources for their respective specializations, and would be weighed upwards in search results. Not a big fan of this conclusion. Who chooses the white list, and why should I trust them? Is it democratically chosen? Just because a site is popular very clear does not mean it's trustw…
You could have user compiled lists of sites to show in search results.
Let the users pick the lists they want to see, and communities can create and distribute lists within themselves.
Trust me, there is not a ton of potential "just sitting on the floor" in web search.