Live data from Hacker News

The world needs more search engines

0x65.dev

401–410 of 549 posts

Re: The world needs more search engines

#401
post #395

Earlier quoted context omitted.

Source? It’s in their best interests to also use it for ad targeting (in a plausibly deniable way so they don’t get in trouble). We’ve seen them using dark patterns to coerce users into opting into more data collection, and another advertising company got caught using phone numbers for ad purposes even if they originally promised to only use them for 2FA, so why should we trust them this time ?

Source: I work on ads at Google, and if we were doing this I'm pretty sure I'd know. If it was being used for targeting it would be practical to run an external study demonstrating that.

Apologies but I can’t really trust someone who works at a company whose best interests are to violate people’s privacy, confirmed by all the dark patterns (both on the web and in Android) and their lack of GDPR compliance.

I would be very curious as to how you’d prove this is or isn’t happening with a reasonable degree of accuracy considering all the factors involved in ad targeting. Unless you’re willing to give us access to all your source code and SSH access to the systems running it, it’s reasonable people have their doubts.

Re: The world needs more search engines

#402
post #378
post #73

Google's dominance is almost entirely due to the fact that its by far the best at search. I use ddg, but I use google search via !g about 25-50% of the time after a failed attempts. And 9 times out of 10 Google gives me exactly what I'm looking for. For instance: "the actor that plays the news guy in spiderman" ddg: Tom Holland (side bar) Spider-Man Homecoming (imdb) Tom Holland (wiki) Spider-Man: Into the Spider-Ver…

"... so Google wasn't able to cheat with privileged information about me as a user." The way they "cheated" was to look at the tens, hundreds, thousands, or tens of thousands of previous queries that matched yours in substance. If the competing websites for "web search" had the same volume of submitted queries that are substantially similar to yours, then they too would be able to give you "exactly" what you are look…

Just because google has seen my search before, how does that translate to being able to give me good search results? Search is an incredibly hard and nuanced problem. I doubt ddg's main disadvantage is training set of search queries.

Plus, you're underestimating the percentage of queries that google has never seen before. Google processes trillions of searches every year, and still, 15% of those queries have never been seen by Google before. [0]

[0] https://searchengineland.com/google-reaffirms-15-searches-ne...

Re: The world needs more search engines

#403
Because you can inject your ad network (https://myoffrz.com/en/fuer-nutzer/) more easily if you have your own search engine, people like Google don't let you use your own ads on their results generally.

I don't disagree with the premise, after all I was part of a team that tried to do the same thing (well, without the ad network) and discovered that the adtech business has become so corrupt (not that it is all corrupt just that enough of it is) that the only way to cover the costs of running the thousand or so machines you need to hold a decent index and cache is hard to achieve.

Re: The world needs more search engines

#404
post #385

Earlier quoted context omitted.

The claim: "Google's dominance is almost entirely due to the fact that its by far the best at search." is, and I'm sorry to say, a make-believe Google's quality is better than anyone else, that is a fact. Let's go to the point. [[I work on Cliqz, and in search]] Do you know how much Google pays apple to be the default search engine? According to you, nothing, because people will go to Google because it's the best. We…

This is business. Apple and Google bring something to the table and they meet in the middle. For zero dollars Apple could just let the user choose from a randomised list which is arguable better and more impartial.

Of course it's business, but it invalidates the premise that people chose "mostly" because of the quality. If that was the main driver, Google could just pay zero, 9 Billion for few hundred million users is quite an hefty sum.

The other option is that Google does not pay to get the Apple users, which might come otherwise, but to prevent Apple to try something funny, either directly or indirectly.

In any case, the "build a better product and people will come" mantra is flawed when companies in the space pay each other billions for distribution.

Re: The world needs more search engines

#405
post #402
post #378

Earlier quoted context omitted.

"... so Google wasn't able to cheat with privileged information about me as a user." The way they "cheated" was to look at the tens, hundreds, thousands, or tens of thousands of previous queries that matched yours in substance. If the competing websites for "web search" had the same volume of submitted queries that are substantially similar to yours, then they too would be able to give you "exactly" what you are look…

Just because google has seen my search before, how does that translate to being able to give me good search results? Search is an incredibly hard and nuanced problem. I doubt ddg's main disadvantage is training set of search queries. Plus, you're underestimating the percentage of queries that google has never seen before. Google processes trillions of searches every year, and still, 15% of those queries have never be…

"Plus, you're underestimating the percentage of queries that google has never seen before."

No. I am highlighting that 75% are queries that they have seen before. Note also that the 15% is a figure that is declining, based on what it was in 2007.

Re: The world needs more search engines

#406
post #216
post #90

Earlier quoted context omitted.

It is really good when you search for vague terms, but it seems to come at the cost of ruining precision technical search. Google also censors some results in controversial categories, where Bing/Yahoo return what I am looking for. I don't trust a search engine that censors entire categories of results. I am not an adwords user but anecdotes in another HN thread seems to affirm that Google is due to be disrupted: htt…

to give a fun example for those who didnt run into this silly agenda. "free energy" use to produce countless results that now require extra keywords. Under "free energy suppression" you find professionally crafted hatemongering. The actual list of claimants is huge, non of it is here. https://peswiki.com/directory:suppression#Wiki_2796702 "free energy device" also produces really crappy results compared to what it wa…

I just searched for “free energy” with both google and bing and with both engines got a combination of the use in actual thermodynamics plus “omg free energy device exist coverup!” type results.

Re: The world needs more search engines

#407
post #333

Earlier quoted context omitted.

The MAC address is not sent to the server, or accessible to web pages client-side, so that wouldn't matter.

In theory. But it's not impossible for a nefarious browser to send it along as an HTTP header. Or a nefarious AP: https://news.ycombinator.com/item?id=21463266

Your link describes the Milan airport captive portal putting the MAC in the URL (don't do this). The referrer is automatically attached to any requests the page makes. This is a comically broken configuration, not something at all common.

Re: The world needs more search engines

#408
post #397
post #378

Earlier quoted context omitted.

"... so Google wasn't able to cheat with privileged information about me as a user." The way they "cheated" was to look at the tens, hundreds, thousands, or tens of thousands of previous queries that matched yours in substance. If the competing websites for "web search" had the same volume of submitted queries that are substantially similar to yours, then they too would be able to give you "exactly" what you are look…

Knowing what the most common questions are doesn't help you find the corresponding answers. At best you can find the result that users click on most frequently, but if all of the results in the result page suck then that's not going to help. Google's advantage is simply billions of dollars and 20 years of R&D into NLP tech.

> Knowing what the most common questions are doesn't help you find the corresponding answers. At best you can find the result that users click on most frequently

I doubt that users have a strict expectation of finding the correct (interpretation of "corresponding") answer. Even if the most clicked answer[s] are still not correct, they may still be the ones who suck less, which can still be acceptable/desirable.

Keep in mind that in this perspective, the concept is similar to Google Translate - Google built a translator that, at least originally, doesn't understand language, instead, it applies (applied) a static model on a large amount of documents. While they certainly poured a large amount of money on it, it's success can't be centered purely on the economical factor.

Re: The world needs more search engines

#409
I've built a few different types of search engines over the years.

The first attempt was in 1999, i built a meta search engine using coldfusion.. I didn't have any server to run it on so it was on my dialup running locally. Worked pretty well, i was a big fan of metacrawler at the time.

The next time was back in around 2003.. I had a server running on my adsl connection at the time. I wrote a crawler of sorts that indexed word positions to search by proximity. At the time, Google didn't do this, it was a feature i was looking for. Eventually, the project died. If i had a 200k startup fund like google had, it could have probably turned into something.

I did build it on mysql and perl but it needed more servers.. Something i didn't really understand back then with only about 4 years web experience.

Then, i built another a meta search engine, ran it on a hosted server, gained some limited traction around 2011. Died as the sources for the api dried up or became too costly. It did make some money from advertising at the time, which is a pretty big deal. The coolest thing was the fact that i kept the searches open, each search had an rss feed that you could use to keep track of certain keyword sets.

Something i need to say about Google is this: it seems simple, but there is a lot going on in the background. Its doing a lot of things like checking for malware, spam, link farms, etc. but also its running the page in a real browser. Its pulling in data from google analytics and clickthrough/bounce rates, its always split-testing result sets, showing results based upon geographic location, language recognition, date recognition, etc. They use machine learning with real humans training those models, that's not to mention their internal systems for managing servers.

I think you shouldn't try and copy what they are doing, in the end it doesn't matter if you do.. Look at bing, its kind of like a joke compared to Google.

Facebook did the right thing, that was to bring the community and data onto the site itself. Identity is the future. Reddit is pretty good as well, you can talk about any link on the internet.

User-moderated content is the future, as AI/ML can always fool another AI/ML but not a human.

You only have to look at this site hacker news to tell that.

Re: The world needs more search engines

#410
post #73

Google's dominance is almost entirely due to the fact that its by far the best at search. I use ddg, but I use google search via !g about 25-50% of the time after a failed attempts. And 9 times out of 10 Google gives me exactly what I'm looking for. For instance: "the actor that plays the news guy in spiderman" ddg: Tom Holland (side bar) Spider-Man Homecoming (imdb) Tom Holland (wiki) Spider-Man: Into the Spider-Ver…

The google result is exactly what I want.

Yes, when what you want happens to be the lowest common denominator - in this case, the most recent Spiderman movie. When the next movie in the franchise comes out in a few years and you don't like it so much because the news guy is now played by someone else, you'll be complaining that your preferences have stayed the same but Google no longer handles them as well.

Post reply on HN