You need many high quality incoming links to have all content indexed quickly.
Google search only has 60% of my content from 2006
101–110 of 168 posts
Re: Google search only has 60% of my content from 2006
#102Earlier quoted context omitted.
I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…
If Google was so evil, why would we purposefully send traffic to sites where we only get a percentage cut of the revenue? I'm pretty sure you didn't mean to open this can of worms, did it have something else printed on its label? Phone number lookup sites are almost certainly a) easily detectable; and b) low traffic. If Google only gets a percentage cut of this, why index them for fractions of pennies per year?
It's because Google's constraint is number of engineer-attention-hours. This kind of thing probably just isn't a priority, when they have a billion other things they could work on.
Keep in mind that Google is not a product company, they are a data company. They work on the biggest most impactful items, and don't have a lot of time for one-offs.
Finally, it might not be clear at all that these sites should be removed from the index.
Re: Google search only has 60% of my content from 2006
#103To play devils advocate for a second, remember how much noise Google has to sift through. Every possible search term exists in every possible combination, often written in lovingly crafted content farm articles by actual humans. If Google offered you those, it might be 1000 pages of empty nonsense before your actual desired content.
Yes, but that is STILL more useful than 50 pages that entirely miss the target.
You are describing the harder 20% of the usual 80/20 effort scale.
Yes, to be truly useful, Google needs to solve also that last, and harder, 20% (and the 10% 0f the 90/10 equation and the 1% of the 99/1 version).
Shortcuts are fine for an initial MVP, but they need to buckle down and solve the problems. It isn't like they don't have the funds.
Re: Google search only has 60% of my content from 2006
#104Earlier quoted context omitted.
I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…
Because if you show artificially irrelevant results too often (by biasing search results to prioritise AdSense pages), then users would stop using Google Search at some point. Eventually you would loose the revenues coming from the SERP, which are 10 times higher than AdSense :) Would you risk loosing 10, to get 1 ?
Let's say the search index is size N and the AdSense index is size M. If we were to join them, we would save the storage space for pages that are in both indices.
Also, the search index would gain all the sites that are in the AdSense index so search results would potentially improve. However, it would give an unfair advantage to google publishers.
Re: Google search only has 60% of my content from 2006
#105Google will also happily surface a stackoverflow article from 2010 about how to solve a js problem... frustrating the top 3 answers will be with jquery, when its not the approach someone would take in the last 5 years. Definitely frustrating, but also showing some need to retire specific pieces of the past away from the top recommendations.
You have hit on a major problem there. Stack Overflow was once the fount of all useful genius grade knowledge, but times change and some of the top answers are plain wrong. Take for example the 'how do I centre a div' type of question. You will get to find an answer with thousands of up-votes that will be some horrendous margin hack type of thing where you set the width of the content and have some counter intuitive…
There are plenty of developers, myself included, that would prefer to approach tasks like this without having to ever lay our fingers on CSS.
Re: Google search only has 60% of my content from 2006
#106Google will also happily surface a stackoverflow article from 2010 about how to solve a js problem... frustrating the top 3 answers will be with jquery, when its not the approach someone would take in the last 5 years. Definitely frustrating, but also showing some need to retire specific pieces of the past away from the top recommendations.
i have no problem with the jquery part, the problem is that the same question has a more recent answer so i constantly have to use date filters
Re: Google search only has 60% of my content from 2006
#107Why does Google deeply index those useless telephone directory sites? Try searching for the impossible U.S. phone number "307-139-2345" and you'll see a bunch of "who called me?" or "reverse phone number lookup" sites. Virtually all of those sites are complete garbage. They make no attempt to collect numbers from telephone directories or from the web. They won't identify a number as being the main phone number for Di…
I always assumed they collect phone numbers this way. People google a phone number and click on a search result. The site can - by looking at the referrer - extract the phone number and infer that it belongs to someone (and use it for any purpose).
It's a similar mechanism that some forums use to highlight the terms that were part of the google query which lead to this site.
Re: Google search only has 60% of my content from 2006
#108To play devils advocate for a second, remember how much noise Google has to sift through. Every possible search term exists in every possible combination, often written in lovingly crafted content farm articles by actual humans. If Google offered you those, it might be 1000 pages of empty nonsense before your actual desired content.
[deleted]
Re: Google search only has 60% of my content from 2006
#109Why does Google deeply index those useless telephone directory sites? Try searching for the impossible U.S. phone number "307-139-2345" and you'll see a bunch of "who called me?" or "reverse phone number lookup" sites. Virtually all of those sites are complete garbage. They make no attempt to collect numbers from telephone directories or from the web. They won't identify a number as being the main phone number for Di…
>for someone else to do better.
It's upsetting to me that doing better than Google in search seems to be very close to an impossible feat of magic at this point.
I know that will change some day, but I can't see how.
Even if somebody gave you hundreds of millions of dollars to spend on infrastructure and employees, it would still be an insane risk.
Writing that out, it almost sounds like internet search engines should be as big and as important of an operation like the TLD registrars. Funded by big governments in collaboration with each other.
Re: Google search only has 60% of my content from 2006
#110Earlier quoted context omitted.
https://www.google.com/search?hl=en&q=307%2D139%2D2345 You comment now comes up first, but the rest of the results all try to contact googlesyndication.com, so ads? Google will not exclude sites that literally give them money.
I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…
because it's better to have 10% of all visits than 100% of only some visits. Google is not the only way people get to those pages. Yes, maybe phone registries, but since the policy cannot be split, we are talking about the whole internet.