191–200 of 630 posts
Earlier quoted context omitted.
> Yes, Google has a huge index, but most queries aren't in the long tail. I'm not quite sure about that. 15% of Google searches per day are unique, as in, Google has never seen them before. [1]. That's quite an insane number. [1] https://searchengineland.com/google-reaffirms-15-searches-ne...
Yes, sorry - that's me. Copy and pasting Sharepoint error messages
Earlier quoted context omitted.
Google's claim that the algorithm is generic is demonstrably false. Type in "hillary clinton e" and there is no suggestion for "email", type "donald trump e" and email is the first suggestion. Given the news content that we know is out there, that can only be the result of adjusting the results for clinton specifically (if anything, we would not expect "email" to be autocompleted for trump). This is not research that…
This is not the most scientific test, since previous searches are generally taken into account. Was this test conducted from a system that mostly searches for / clicks on pro-trump or anti-trump content?
I searched from a Firefox private window over a VPN from the Netherlands. But since the results are the same (regarding presence/absence of "email" as an autocomplete term) I don't think it matters much.
Earlier quoted context omitted.
The difference in this case is that the usual defenders of private property rights and libertarianism feel as though Google is suppressing them removing the usual roadblock to this kind reform. This is a rare instance where anti-corporate leftists and dejected right wingers could actually do something substantive together. In most cases I would agree with you the defenders of private industry are pretty fierce in the…
Some people are arguing that catering search results or what content is allowed on a platform to a specific set of political views makes those platforms publishers rather than mere platforms. Apparently this also has some implications in some political campaign laws I don't really understand. I think we're heading towards political ideologies having the same protection as religious institutions (which IMO, are exactl…
The government has never had the right to meddle in what publishers decide to publish (with exceptions for regulations on pornography and classified information). I can't imagine the government forcing Mother Jones or The Nation to print conservative viewpoints. If the platforms become "publishers" their power to select what information is available increases. They become liable for more of the content on their sites, but they also gain full control over content.
The only way I see to protect freedom of access to information would be to declare certain spaces (Facebook, Reddit, YouTube, etc.) as "privately-owned public forums" where suppression based on ideology would be heavily restricted.
Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…
> Indexing the top billion pages or so won't take as long as people think. This is what makes me wonder why we don't have a LOT of competing search engines. Perhaps i'm vastly under-estimating the technology and difficulty (I could well be - it's not my domain) but it surely it can't be THAT hard to spawn Google-like weighted crawl-based search results? It's a long-since solved problem - heck, pageRank's first iterat…
It's easy to gather the necessary data, but it's hard to know which parts of that data are the most relevant for finding good content and avoiding bad content. Is it more relevant if key words show up in links or titles than in the body of the text? If so, SEO spam sites will include a bunch of keywords in links and titles. Is it more relevant if keywords show up in the first 200 visible words of the page? If so, spam pages will make tons of pages with relevant keywords at the top.
The hard part about building a search engine isn't indexing the internet, it's adapting to spam. Spammers are continually adapting to changes in the algorithm, so the algorithm needs to adapt as well. And the more popular your search engine is, the more money you make and the more able you are too adapt to spam (and the more spammers focus on your engine).
So, the problem isn't that Google has a better index (though I'm sure it does), the problem is that nobody else has the will to spend the money necessary to tune the search algorithm to stay on top of spammers. When Google started, companies didn't care as much about improving their index and instead focused on building their other content (Yahoo, MSN, etc). Google saw the value of search and got a lead on everyone else in terms of curating results, and now they have the momentum to stay in front and have shifted to building content to improve monetization. Nobody else has the monetization network for search that Google has, so they'll continue having the problem that other companies had (Microsoft wants to point you to their other services, DuckDuckGo is limited by their commitment to privacy, etc).
In short, Google wins because:
- it was better when it mattered - it makes money directly from search - its other services improve their ability to understand what users want, which improves search quality and ad relevance
You can't make a better algorithm by being clever, you make a better algorithm by having better data, and that's hard to come by these days. The only way I can think of a competitor stepping in is if they target an underserved demographic and focus data collection and monetization there, and DuckDuckGo is close by targeting privacy conscious power users.
Earlier quoted context omitted.
Especially with voice searches. People are searching entire sentences rather than specific keywords which are much more likely to be unique.
Do people do this? Or do you mean queries forwarded by home assistants trying to parse inputs?
As a developer, I search using keywords; for example, if I was looking for property for sale in Inverness, I might search for "property Inverness", whereas I've seen and heard "typical" users use something like "find me a 2 bedroom house with a garden for sale in the North of Inverness" - much more verbose, and containing stop words and phrases unlikely to help (I think!).
Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…
I'd wager any startup that tries to crawl a few sites like Amazon, Yelp, Linkedin, etc will be blocked. Google, however gets a pass because they're Google. So yes, I believe their huge index, and ability to crawl any site at will is a huge, huge advantage for them.
Earlier quoted context omitted.
Yes, sorry - that's me. Copy and pasting Sharepoint error messages
Those searches are unlikely to be unique.
is not atypical.
Fun timing to read this, this last weekend I was playing around with making my own search engine to understand better how ElasticSearch and Lucene work. It occurred to me that the two most powerful things Google has to work with are records of clicks, and the time users spent on the webpages Google returned. I've argued against Google monopoly before because I can throw together a web crawler and search engine in a w…
> but with Google Analytics... No: "Search does not use Google Analytics for ranking" https://www.youtube.com/watch?v=LLmO1GE4GvI
Okay, this is a relatively serious proposal to require Google to allow API access to its search index, with the premise that it would democratize the search engine ecosystem. There are some issues with the regulations he proposes (you have to allow throttling to prevent DDoS attacks, and you can't let anyone with API access add content to prevent garbage results), but it's roughly feasible. The main problem is, I thi…
> Indexing the top billion pages or so won't take as long as people think. This is what makes me wonder why we don't have a LOT of competing search engines. Perhaps i'm vastly under-estimating the technology and difficulty (I could well be - it's not my domain) but it surely it can't be THAT hard to spawn Google-like weighted crawl-based search results? It's a long-since solved problem - heck, pageRank's first iterat…
Consider DuckDuckGo, which sells itself on privacy, but after more than a decade has only 0.18% market share. Without the power to make it the default in an OS or browser, you'd have to have a really strong value proposition to convince people to switch.