Live data from Hacker News

The world needs a non-profit search engine

daoudclarke.net

391–396 of 396 posts

Re: The world needs a non-profit search engine

#391
post #226

Earlier quoted context omitted.

I want to write articles and read articles written for me by others. Ideally as few as possible should profit from this process. As google is now a turd, not just no longer capable of delivering this service but actively destroying the good part of the web by refusing to index it. It is my attention, it doesn't belong to anyone else. My access to information and educated opinion is a far more integral part of the gre…

Google is a catalogue. There is no physical analogy that comes close. Your point about wanting to read articles written for you by others is certainly possible. The very fact that such a desirable outcome drives you to Google and nowhere else should suggest the complexity of the problem they’re solving and how there isn’t really anything else out there doing so well. >Ideally as few as possible should profit from thi…

> The very fact that such a desirable outcome drives you to Google and nowhere else should suggest the complexity of the problem they’re solving and how there isn’t really anything else out there doing so well.

Yes and I'm not impressed.

> > Ideally as few as possible should profit from this process.

> Why?

> When you accepted a job offer in the software industry, did you stipulate that your mission is to write code for your employer and you will be charging as little as possible for that privilege? Minimum wage should get you by just fine, right?

I'm not a good example as I indeed live wonderfully on minimum wage and write software for free.

> I hate fully grown adults behaving as though anyone except them making a profit is somehow evil.

Don't worry, my philosophy is not that superficial. We have people who make things, people who organize the making of things and people who organize the things made.

It can be true that the meta data is more valuable than the data it self and organizing an effort can be much more intense than any of the tasks involved. But lets not pretend that is always the case.

Before money and before the written word we had the exchange of thoughts, observations and ideas. I believe this to be somewhat like the foundation on which everything else we did is build. I want to see this process benefit from technology.

You wrote your comment perhaps a bit limited by the ropes of the platform but sincerely, free from any agenda, you wrote pretty much what you think.

Now if we [beyond HN] add additional layers of agendas between our exchange, each interested in maximizing their profit from it perhaps not you but many others will resort to self-moderation.

You wont be able to state it simply like: "I hate fully grown adults behaving as though anyone except them making a profit is somehow evil."

It could become something like "I don't understand why some people don't like others making money" stripped from how strong you feel about the subject. You could also chose not to say anything.

At that point we are messing with the very fabric of our collective reality.

If I had to chose between freely communicating and the economy it wouldn't be a hard choice.

Re: The world needs a non-profit search engine

#392
post #386

Earlier quoted context omitted.

This is fascinating, thanks. Have you experimented on allowing the bing ad bot that you have blocked? If they have some kind of retaliatory non-crawling?

Interesting theory. But the IPBL doesn’t even have a robots.txt and a different, larger site from a German celebrity I host does have the same directives and is indexed, although incompletely. My working theory is that Bing’s selection algorithm is biased towards large and already popular sites. In the server logs, I don’t see Bing even attempting to crawl the sites I mentioned, except requesting robots.txt and the r…

Just received a final non-answer from Microsoft support after weeks of escalating:

—————

Thank you for your patience!

After further review, it appears that your site http://herrbischoff.com https://ipbl.herrbischoff.com/> did not meet the standards set by Bing the last time it was crawled.

Bing constantly prioritizes the content to be indexed that will drive highest users satisfaction. Please follow Bing Webmaster Guidelines to better understand criteria for most valuable content.

Re: The world needs a non-profit search engine

#393

Earlier quoted context omitted.

> the free ad supported model supports millions of peoples livelihoods That doesn't necessarily make it right. Online scamming also supports a lot of people's livelihoods.

One is a business model, one is a crime and fraud. I’d expect you to know the difference.

Lying in a privacy policy and breaching privacy-related regulations can also be fraud and illegal. Think about why there's so much pushback against the GDPR despite it only primarily mandating transparency with regards to data usage (if they were doing things above-board why would they be afraid?).

Re: The world needs a non-profit search engine

#394

Earlier quoted context omitted.

Honestly that is probably safer. Having a typo in the url could Easley give you a phishing link. However, I also have gotten fishing adds when looking websites up so it’s not cut and dry at all.

Easley, I have to ask, was that a typo or highlighting your point?

When I see something like that, I tend to assume that some people use speech recognition to write comments. (No idea if that was the case here.)

Re: The world needs a non-profit search engine

#395
post #231
post #90

Perhaps a better approach would be building an open source www index or even a full current cache - as an enabler for people to build their own search engines? Right now it is extremely difficult to build your own web crawler that would compete with Google. And that is not because of the technology, but because multiple sites will prevent your bot from accessing them if you're not Google or Bing - either through robo…

> Right now it is extremely difficult to build your own web crawler that would compete with Google. Nah, not a small task but you can break it down into well understood problems that have known solution. The hard part is ranking everything.

Are you speaking from theory or experience?

If from experience - how did you get around multiple sites disallowing crawlers other than google or bing?

Re: The world needs a non-profit search engine

#396

Earlier quoted context omitted.

For clarification, “red queen” means a conflict between two or more entities where the cost of engagement grows, but the relative advantage does not change. Simply put, search engines have been at war with SEO for over 30-years, which has significantly raised the bar not only being a search engine, but producing content; not to mention knowing how to search for information. With the introduction of machine generated…

To your last point, I don’t quite agree. Google’s incentives are misaligned such that keeping you on Google.com just a little longer is better than not because you are more likely to click on an ad. But also yes the users and the UI both fail. When I used to search for something I would type in something like “gutter clog clean” but slowly started noticing that Google likes longer sentences like “how do I clean a clo…

Google search, from very early on, considered it a success metric when users went quickly to a result. I have no idea how that factors into the current surely hideously complex ranking algorithms, though.

As for the parsing of queries, that's probably based on how most users use search. Not everyone is familiar with keyword -based search. I expect they've done tons of A/B tests to determine what kind of query interpretation makes most users get better results. We're just not "most users".

Post reply on HN