Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The world needs a non-profit search engine

daoudclarke.net

Post reply on HN
Print view View original

221–230 of 396 posts

Re: The world needs a non-profit search engine

#221

Post by AussieWog93 »

lotsofpulp wrote: post #202
zo1 wrote: post #149

Earlier quoted context omitted.

I went back to Reddit a week back after a 1 or so year, and what I saw was just disgusting on the main feed. One click, a bit of scroll and I was watching people calling republicans terrorists and being praised/upvoted/gifted. It's not just about bullshit at the top, it's also about echo chambers and the online equivalent of mob-behavior.

> it's also about echo chambers and the online equivalent of mob-behavior. This particular example might be about the Republican party-wide support of echo chambers and offline mob behavior that led to an invasion of the building containing politicians certifying the vote of a newly elected leader and fueling disinformation about elections to weaken the trust and integrity of the system?

Regardless of this particular example, there has always been a pretty strong bias towards one side of US (and Australian, for that matter) partisan politics on the front page of Reddit; with over-the-top accusations towards the Republican Party as well as the LNP both receiving thousands of upvotes despite their poor quality.

Re: The world needs a non-profit search engine

#222

Post by lovemenot »

sen wrote: post #3
mmazing wrote: post #2

> The technology for organizing the world’s knowledge should be owned by everyone. This is what really nails it for me. There's far too much black box in pretty much every major search engine out there. Maybe it's by design "so that people can't game it". Even so, it's not working very well. I'm excited for the next 10 years to see what we (humans) come up with to solve the state of the internet, because something's…

Why though? 99.9% of people don’t care, and the internet (Facebook, instagram, TikTok, YouTube, gmail, the occasional Wikipedia) work fine for them. The overwhelming vast majority of people just don’t care about the internet outside those major few silos, so as far as “humanity” is concerned the internet is working as intended. It pisses me off what’s become of the internet, but I don’t personally see it changing.

You are right about people's current opinions, but seem to be assuming that given a better option, people would continue to make sub-optomal choices.

If so, I don't agree. Your small minority's job is to deliver those alternatives, and to feed the flames while the rest of the world makes the transition. Which they will do because the next thing is clearly so much better.

Have more faith in most of humanity and doubt yourself and similar others for having failed so far to disrupt this industry with better technology.

Re: The world needs a non-profit search engine

#223

Post by fauigerzigerk »

slazaro wrote: post #200
nonameking2026 wrote: post #189

Earlier quoted context omitted.

Allow users to blacklist sites. Share blacklisted sites. Have the option of instead of hiding blacklisted sites entirely - show them in a different column. Have 4 different types of list. Whitelist - highest scoring Not listed - these websites are not ranked. Yellowlist - show but keep in a separate column Blacklist - don't show https://search.brave.com/help/goggles is interesting.

99%+ of people wouldn't use this, they would just try the search engine, see that the results suck, then stop using it. It could be useful for power users, though.

It could work like spam filtering. Once a certain number of users have marked a site as spammy (or put it on their blacklist), it gets downranked for everyone. But jstummbillig is right. It could easily get gamed.

Re: The world needs a non-profit search engine

#224

Post by zaarn »

8organicbits wrote: post #212
zaarn wrote: post #133

Earlier quoted context omitted.

Then I'll just add 3 million bots to the network (or just enough to have about 50%) and I can guarantee to win the A/B test against an honest client most of the time.

It's an arms race, but this is mostly a question of rate limiting account creation, assigning a trustworthiness score to different accounts, some network analysis to detect coordinated accounts, and having some trusted accounts (run by the project) that can help double check results. After an account loads poisoned data, you can detect this after the attack (user reported spam), and then block (or probably shadow ban…

You make it sound easy but companies have been trying to fight this stuff for ages.

You can buy a trustworthy residential IP for low cost, you can buy them in bulk in the thousands. All of them are real residential IPs from any ISP of your choosing in any country. You can rent Chrome browsers running over those IPs, directed via remote desktop and accessibility protocols (good luck banning that without running awful of anti-discrimination laws). You can do all that for under 1k$ a month for like 1 million clients.

My workplace has been at the other end of DDoS attacks directed by such services, best you can do is ban specific Chrome versions they use but that lasts until they update.

It's an uphill battle that you will loose in the long term if you rely on client trust.

Re: The world needs a non-profit search engine

#225

Post by fauigerzigerk »

allendoerfer wrote: post #193
nonameking2026 wrote: post #189

Earlier quoted context omitted.

Allow users to blacklist sites. Share blacklisted sites. Have the option of instead of hiding blacklisted sites entirely - show them in a different column. Have 4 different types of list. Whitelist - highest scoring Not listed - these websites are not ranked. Yellowlist - show but keep in a separate column Blacklist - don't show https://search.brave.com/help/goggles is interesting.

I found the thought of adding a "yellowlist" funny, as some people are starting to find the use of "whitelist" and "blacklist" racist.

It would be more consistent to call it greenlist, yellowlist, redlist then.

Re: The world needs a non-profit search engine

#226

Post by 6510 »

DisjointedHunt wrote: post #209
vineyardmike wrote: post #172

Earlier quoted context omitted.

While I agree that they may need to better consider the economics - both of the engine and the websites that may SEO it- that doesn’t mean we should just assume as supported is the way to go or the only way to support people. The economy of today looks different than 20 years ago or 20 years before that. Doesn’t mean we shouldn’t grow and change.

I respect that. I guess my point is "Grow and change with a decent understanding of what the present state enables" A search engine like Google isn't just a search engine as the author describes it. It is a very integral part of the economy of the internet and just labelling a simplistic interpretation of the present state as "Evil" with an academically poor write up of what a viable alternative is does little good.

I want to write articles and read articles written for me by others. Ideally as few as possible should profit from this process.

As google is now a turd, not just no longer capable of delivering this service but actively destroying the good part of the web by refusing to index it.

It is my attention, it doesn't belong to anyone else.

My access to information and educated opinion is a far more integral part of the greater economy.

Google is like a screaming man at a town meeting making sure no one else can get a word in. The meeting is now pointless.

Re: The world needs a non-profit search engine

#227

Post by hnarn »

jstummbillig wrote: post #218
nonameking2026 wrote: post #189

Earlier quoted context omitted.

Allow users to blacklist sites. Share blacklisted sites. Have the option of instead of hiding blacklisted sites entirely - show them in a different column. Have 4 different types of list. Whitelist - highest scoring Not listed - these websites are not ranked. Yellowlist - show but keep in a separate column Blacklist - don't show https://search.brave.com/help/goggles is interesting.

How do you prevent the gaming of blacklisting?

That’s the same question as “how do you ensure complete trust” which is, of course, not possible. That doesn’t mean that “distributed trust-based blacklisting” still isn’t better than what Google is offering today, which is nothing.

Re: The world needs a non-profit search engine

#228

Post by netheril96 »

closedloop129 wrote: post #139
kolinko wrote: post #93

Earlier quoted context omitted.

Bombing is one thing, but you also have a whole SEO industry now that will exploit any way possible to get to the top of the rankings. The moment you have community rankings on search, and your search gets popular, you land in a war zone with bots trying to mangle those. Reddit is kind of good dealing with that, but it is very resource intensive.

What if you limit accounts to real people and then keep track of their credibility? It's some initial effort but how could the ranking be manipulated when all dishonest people have burned their credibility?

> What if you limit accounts to real people and

In China, all social accounts must be associated with a phone number, and phone numbers are tied to government identities. It doesn't stop any manipulation of scores and rankings.

> then keep track of their credibility?

It is very likely China will do that too soon. I think you can already imagine the ramifications.

Re: The world needs a non-profit search engine

#229

Post by jstummbillig »

kreeben wrote: post #205
blowski wrote: post #190

Earlier quoted context omitted.

I’ve got 10k karma by making pithy criticisms of Node.js and Kubernetes. Do you therefore trust my opinions on US healthcare?

I would, if what you say were to make a whole lot of sense, otherwise, no. You know how it works here. We would strip you of your internet points if you start being nonsensical.

I think you are kind of ignoring the issue that parent brought up.

The hn machine is not all that smart and can easily be gamed, and would, if the stakes were to become high enough. Farming hn karma by making pointed statements on crowd favorites (parent named two, oss licensing or privacy also spring to mind) gets you your 10k, no originality or honesty required, in no time.

What's protecting hn is a lot of moderation + relative irrelevance. If those 10k were to systematically bring you enough eyes (by driving search results), you are in effect printing money. There is no reason to assume the number of people doing it would not scale with the return attached to doing it.

Re: The world needs a non-profit search engine

#230

Post by asah »

kolinko wrote: post #93
TulliusCicero wrote: post #57

> Instead of looking at how long people spend on a site, we would encourage users to give explicit feedback on rankings and use this to improve our ranking system. While they're not wrong about how the way Google determines ranking has its issues, this way has its own set of problems. If you explicitly use user ratings as part of your rankings in some way, people can punish sites they don't like, ala review bombing o…

Bombing is one thing, but you also have a whole SEO industry now that will exploit any way possible to get to the top of the rankings. The moment you have community rankings on search, and your search gets popular, you land in a war zone with bots trying to mangle those. Reddit is kind of good dealing with that, but it is very resource intensive.

if this becomes popular, that same industry will recruit gangs to manipulate this new system.

source: worked in ads and search for decades, incl google.

Post reply on HN