Earlier quoted context omitted.
The theory I've heard is related to 'crawl budget'. Google is only going to devote a finite amount of time to indexing your site. If the number of articles on your site exceeds that time, some portion of your site won't be indexed. So by 'pruning' undesirable pages, you might boost attention on the articles you want indexed. No clue how this ends up working in practice. Google's suggestion isn't to delete pages, but…
Even if that rule were true, why wouldn’t everything in the say, top NNN internet sites get an exemption? It is the Internet’s most hit content, why would it not be exhaustively indexed? Alternatively, other than ads, what is changing on a CNN article from 10 years ago? Why would that still be getting daily scans?
CNET is deleting old articles to try to improve its Google Search ranking
81–90 of 592 posts
Re: CNET is deleting old articles to try to improve its Google Search ranking
#82Earlier quoted context omitted.
The theory I've heard is related to 'crawl budget'. Google is only going to devote a finite amount of time to indexing your site. If the number of articles on your site exceeds that time, some portion of your site won't be indexed. So by 'pruning' undesirable pages, you might boost attention on the articles you want indexed. No clue how this ends up working in practice. Google's suggestion isn't to delete pages, but…
But as that linked guide explains, that's only relevant for sites with e.g. over a million pages changing once a week . That's for stuff like large e-commerce sites with constantly changing product info. Google is clear that if your content doesn't change often (in the way that news articles don't), then crawl budget is irrelevant.
Re: CNET is deleting old articles to try to improve its Google Search ranking
#83Earlier quoted context omitted.
In fairness to them. If one of their top ways of getting traffic is being marked as less relevant because of older articles, what do you want them to do? Just continue to lose money because Google can't rank them appropriately?
Or mayhap Google et al could realize and understand that having old articles in archive shouldn't penalize your ranking.
Re: CNET is deleting old articles to try to improve its Google Search ranking
#84Earlier quoted context omitted.
The theory I've heard is related to 'crawl budget'. Google is only going to devote a finite amount of time to indexing your site. If the number of articles on your site exceeds that time, some portion of your site won't be indexed. So by 'pruning' undesirable pages, you might boost attention on the articles you want indexed. No clue how this ends up working in practice. Google's suggestion isn't to delete pages, but…
They could specify in the sitemap how often do old articles change. Or set a indefinite caching header.
Re: CNET is deleting old articles to try to improve its Google Search ranking
#85Is there any evidence this would even work? Surely Google determines "fresh, relevant" content according to whatever has recently been published, which this doesn't change. If anything, doesn't Google consider sites with a long history of content with tons of inbound links as more authoritative and therefore higher-ranked? This baffles me. It baffles me why this would be successful SEO -- and assuming that it actuall…
The theory I've heard is related to 'crawl budget'. Google is only going to devote a finite amount of time to indexing your site. If the number of articles on your site exceeds that time, some portion of your site won't be indexed. So by 'pruning' undesirable pages, you might boost attention on the articles you want indexed. No clue how this ends up working in practice. Google's suggestion isn't to delete pages, but…
Once a site has been indexed once, should it really be crawled again? Perhaps Google should search for RSS/Atom feeds on sites and poll those regularly for updates: that way they don't waste time doing to a site scrape multiple times.
Old(er) articles, once crawled, don't really have to be babysat. If Google wants to double-check that an already-crawled site hasn't changed too much, they can do a statistical sampling of random links on it using ETag / If-Modified-Since / whatever.
Re: CNET is deleting old articles to try to improve its Google Search ranking
#86Re: CNET is deleting old articles to try to improve its Google Search ranking
#87Is there any evidence this would even work? Surely Google determines "fresh, relevant" content according to whatever has recently been published, which this doesn't change. If anything, doesn't Google consider sites with a long history of content with tons of inbound links as more authoritative and therefore higher-ranked? This baffles me. It baffles me why this would be successful SEO -- and assuming that it actuall…
I have noticed some articles (and not just "Best XXX of 202Y" articles) that seem to always update their "Updated on" date which Google unhelpfully picks up and shows in search results leading me to think the page is much more recent than it is.
Re: CNET is deleting old articles to try to improve its Google Search ranking
#88So google's shitty search now economically incentivizes sites to destroy information. Can there be any doubt that Google destroyed the old internet by becoming a bad search engine? Could their exclusion of most of the web be considered punishment for being sites being so old and stable that they don't rely on Google for ad revenue?
Google is destroying the internet is a good way to put it. AD dollars are their only priority. I hope Google dies because of it.
Re: CNET is deleting old articles to try to improve its Google Search ranking
#89Earlier quoted context omitted.
But as that linked guide explains, that's only relevant for sites with e.g. over a million pages changing once a week . That's for stuff like large e-commerce sites with constantly changing product info. Google is clear that if your content doesn't change often (in the way that news articles don't), then crawl budget is irrelevant.
To determine whether content changes Google has to spend budget as well, hasn't it? So it has to fetch that 20-years old article.
It doesn't have to fetch every article (statical sampling can give confidence intervals), and it doesn't have to fetch the full article: doing a "HEAD /" instead of a "GET /" will save on bandwidth, and throwing in ETag / If-Modified-Since / whatever headers can get the status of an article (200 versus 304 response) without bother with the full fetch.
Re: CNET is deleting old articles to try to improve its Google Search ranking
#90So google's shitty search now economically incentivizes sites to destroy information. Can there be any doubt that Google destroyed the old internet by becoming a bad search engine? Could their exclusion of most of the web be considered punishment for being sites being so old and stable that they don't rely on Google for ad revenue?
Google is destroying the internet is a good way to put it. AD dollars are their only priority. I hope Google dies because of it.