Live data from Hacker News

Google search only has 60% of my content from 2006

tablix.org

91–100 of 168 posts

Re: Google search only has 60% of my content from 2006

#91
To play devils advocate for a second, remember how much noise Google has to sift through. Every possible search term exists in every possible combination, often written in lovingly crafted content farm articles by actual humans.

If Google offered you those, it might be 1000 pages of empty nonsense before your actual desired content.

Re: Google search only has 60% of my content from 2006

#92
post #85

Earlier quoted context omitted.

This of course depends on what you mean by 'information'. Lets say we have data points ABCDEFGHIJKLMNOP But depending on the URL you follow to get there you can get a page containing only some of the elements. index.html?ACD or index.html?AP or index.html?GI All different combinations return a page that could be weighted differently by an algormith and represent valid informational return data. To a person looking fo…

True. A crawler need to differentiate generated content from "real" content somehow. I.e. a service: www.thenumberinsanskrit.com/?q=1 that returns the queried number in Sanskrit, need to not be indexed (except the entry page) while: www.news.com/?article=major-jones-in-scandal-20190103 needs to be indexed. Usually interesting pages are indexed on the site or linked somewhere on it, though.

>A crawler need to differentiate generated content from "real" content somehow.

"Somehow", aka using computing power and storing results, but that still turns into an explosion of computing time and data storage. I mean, what is the difference between the example I listed and Facebook's front page? They are both 'real' content in a generated format.

And a converse argument for your Sanskrit example is, what if I have the sanskrit number and don't know what it is? I put it in google and the site returns it as the number one.

> linked somewhere on it

And those links can all be generated by algorithms.

Anyway, back to your original statement. There is no 'real' content. Only data exists. Most content systems used on the internet allow this data to be combined and displayed in a multitude of different ways depending on the call method and attributes of the viewee. Many times these combinations of data can present novel value to the user. And with the future only presenting us more automated data collection and presentation methods, search engines have lost this battle.

Re: Google search only has 60% of my content from 2006

#93
post #75
post #23

Google will also happily surface a stackoverflow article from 2010 about how to solve a js problem... frustrating the top 3 answers will be with jquery, when its not the approach someone would take in the last 5 years. Definitely frustrating, but also showing some need to retire specific pieces of the past away from the top recommendations.

What's wrong with using jQuery in 2019?

It’s a JavaScript tool that existed 5 years ago.

Re: Google search only has 60% of my content from 2006

#94
post #13

Why does Google deeply index those useless telephone directory sites? Try searching for the impossible U.S. phone number "307-139-2345" and you'll see a bunch of "who called me?" or "reverse phone number lookup" sites. Virtually all of those sites are complete garbage. They make no attempt to collect numbers from telephone directories or from the web. They won't identify a number as being the main phone number for Di…

https://www.google.com/search?hl=en&q=307%2D139%2D2345 You comment now comes up first, but the rest of the results all try to contact googlesyndication.com, so ads? Google will not exclude sites that literally give them money.

I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not in the search index.

If Google was so evil, why would we purposefully send traffic to sites where we only get a percentage cut of the revenue?

EDIT: I realize I needed to explain this statement a little bit more. If we show ads on google.com we get 100% of the revenue. If show ads on reversephonelookup.it they get majority of the revenue. There is a limited amount of advertiser demand. Instead of manipulating the organic search results, it would be more profitable for google to just show more ads on the search page or inflate the ad price or something.

Re: Google search only has 60% of my content from 2006

#95
post #94

Earlier quoted context omitted.

https://www.google.com/search?hl=en&q=307%2D139%2D2345 You comment now comes up first, but the rest of the results all try to contact googlesyndication.com, so ads? Google will not exclude sites that literally give them money.

I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…

> If Google was so evil, why would we purposefully send traffic to sites where we only get a percentage cut of the revenue?

Can you explain this? Not sure if it's your phrasing or what, but I'm not getting what you mean.

Re: Google search only has 60% of my content from 2006

#96
post #94

Earlier quoted context omitted.

https://www.google.com/search?hl=en&q=307%2D139%2D2345 You comment now comes up first, but the rest of the results all try to contact googlesyndication.com, so ads? Google will not exclude sites that literally give them money.

I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…

Because if you show artificially irrelevant results too often (by biasing search results to prioritise AdSense pages), then users would stop using Google Search at some point. Eventually you would loose the revenues coming from the SERP, which are 10 times higher than AdSense :)

Would you risk loosing 10, to get 1 ?

Re: Google search only has 60% of my content from 2006

#97
post #13

Why does Google deeply index those useless telephone directory sites? Try searching for the impossible U.S. phone number "307-139-2345" and you'll see a bunch of "who called me?" or "reverse phone number lookup" sites. Virtually all of those sites are complete garbage. They make no attempt to collect numbers from telephone directories or from the web. They won't identify a number as being the main phone number for Di…

I have long believed that the proliferation of phone-number-lookup sites was antisocial media: spammers and other bad actors creating these sites to prevent people from having a central place to talk about bad phone experiences, associated with specific numbers. Which one is the good one? You can't tell.

Re: Google search only has 60% of my content from 2006

#98
post #94

Earlier quoted context omitted.

I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…

> If Google was so evil, why would we purposefully send traffic to sites where we only get a percentage cut of the revenue? Can you explain this? Not sure if it's your phrasing or what, but I'm not getting what you mean.

[deleted]

Re: Google search only has 60% of my content from 2006

#99
post #94

Earlier quoted context omitted.

https://www.google.com/search?hl=en&q=307%2D139%2D2345 You comment now comes up first, but the rest of the results all try to contact googlesyndication.com, so ads? Google will not exclude sites that literally give them money.

I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…

If Google was so evil, why would we purposefully send traffic to sites where we only get a percentage cut of the revenue?

I'm pretty sure you didn't mean to open this can of worms, did it have something else printed on its label?

Phone number lookup sites are almost certainly a) easily detectable; and b) low traffic. If Google only gets a percentage cut of this, why index them for fractions of pennies per year?

Re: Google search only has 60% of my content from 2006

#100
post #99
post #94

Earlier quoted context omitted.

I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…

If Google was so evil, why would we purposefully send traffic to sites where we only get a percentage cut of the revenue? I'm pretty sure you didn't mean to open this can of worms, did it have something else printed on its label? Phone number lookup sites are almost certainly a) easily detectable; and b) low traffic. If Google only gets a percentage cut of this, why index them for fractions of pennies per year?

There may be cases where we drop low traffic sites from our index, which is separate from the search index. I don't how much of that is public information, so can't go into detail.
Post reply on HN