Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CNET is deleting old articles to try to improve its Google Search ranking

theverge.com

Post reply on HN
Print view View original

101–110 of 592 posts

Re: CNET is deleting old articles to try to improve its Google Search ranking

#101

Post by ahmedfromtunis »

pessimizer wrote: post #68

So google's shitty search now economically incentivizes sites to destroy information. Can there be any doubt that Google destroyed the old internet by becoming a bad search engine? Could their exclusion of most of the web be considered punishment for being sites being so old and stable that they don't rely on Google for ad revenue?

Even if we pretend for a moment that your statement, that google's search is "shitty", is universally accepted as truth, you can't blame this one on Google.

People have been committing horrifying atrocities in the name of SEO for years. I've seen it firsthand. And it spectacularly backfired each time.

This can very probably be yet another one of such cases.

Re: CNET is deleting old articles to try to improve its Google Search ranking

#102

Post by npunt »

robertkeizer wrote: post #86

This is why I work at archive.org. Is it perfect? No. Does it have value to society? Absolutely.

I adore archive.org. I'm worried though it is becoming somewhat of a load bearing element of civilization, given the importance of shared and accurate history. We need redundancy.

~~I'm also worried about the deletion of old pages on archive because new owners of a domain update the robots.txt file to disallow it, which I've heard wipes the entire archive.org history of that domain. I hope that gets addressed.~~

Edit: this is no longer the case

Re: CNET is deleting old articles to try to improve its Google Search ranking

#103

Post by codedokode »

strken wrote: post #97
codedokode wrote: post #82

Earlier quoted context omitted.

To determine whether content changes Google has to spend budget as well, hasn't it? So it has to fetch that 20-years old article.

If the content is literally the same, the crawler should be able to use If-Modified-Since, right? It still has to make a HTTP request, but not parse or index anything.

If the content is dynamic (e.g. a list of popular articles in a sidebar has changed), then the page will be considered "updated".

Re: CNET is deleting old articles to try to improve its Google Search ranking

#104

Post by voramok »

throw0101a wrote: post #89
codedokode wrote: post #82

Earlier quoted context omitted.

To determine whether content changes Google has to spend budget as well, hasn't it? So it has to fetch that 20-years old article.

> So it has to fetch that 20-years old article. It doesn't have to fetch every article (statical sampling can give confidence intervals), and it doesn't have to fetch the full article: doing a "HEAD /" instead of a "GET /" will save on bandwidth, and throwing in ETag / If-Modified-Since / whatever headers can get the status of an article (200 versus 304 response) without bother with the full fetch.

There’s an obvious way this can be exploited. Bait and switch.

Re: CNET is deleting old articles to try to improve its Google Search ranking

#105

Post by yjftsjthsd-h »

skissane wrote: post #54
flangola7 wrote: post #37

Earlier quoted context omitted.

What reasons would there to be to delete old new articles? My inner historian is screaming.

One reason is if you keep all your old versions of product docs up, Google will randomly send people to the old version instead of the current one, and then customers will get confused by the outdated info I’m sure there must be some way to fix this with META tags/etc, but it is often easier just to delete the old stuff, than change the META tags on 100s or 1000s of legacy doc pages

I'm decently fond of how ansible docs do it (for example) - consider ex. https://docs.ansible.com/ansible/2.9/modules/list_of_all_mod... which has a nice banner at the top saying this is an old version.

Re: CNET is deleting old articles to try to improve its Google Search ranking

#106

Post by cout »

ghaff wrote: post #55
kenjackson wrote: post #46

Earlier quoted context omitted.

That's what the article says they're doing.

The article isn't super-clear about what's happening and, for most purposes, just dumping stuff on the Wayback Machine is probably not that different from deleting it even if the bits are still "somewhere." A few years back I copied any of my CNET stuff I cared about to my own site and, in general, that's a strategy I've followed with a number of sites as I don't expect anything to continue to be hosted or at least b…

I kinda wish there were some way to store every page I ever visit automatically and index it locally for easy search. Then when I want to look up e.g. the guy who had the popular liquid oxygen fire website back in the 90s it would be easy. But I also fear it would be used against me somehow too.

Re: CNET is deleting old articles to try to improve its Google Search ranking

#107

Post by influx »

throw0101a wrote: post #85
burnhamup wrote: post #15

Earlier quoted context omitted.

The theory I've heard is related to 'crawl budget'. Google is only going to devote a finite amount of time to indexing your site. If the number of articles on your site exceeds that time, some portion of your site won't be indexed. So by 'pruning' undesirable pages, you might boost attention on the articles you want indexed. No clue how this ends up working in practice. Google's suggestion isn't to delete pages, but…

> The theory I've heard is related to 'crawl budget'. Google is only going to devote a finite amount of time to indexing your site. Once a site has been indexed once, should it really be crawled again? Perhaps Google should search for RSS/Atom feeds on sites and poll those regularly for updates: that way they don't waste time doing to a site scrape multiple times. Old(er) articles, once crawled, don't really have to…

This should be what sitemap.xml provides already.

Re: CNET is deleting old articles to try to improve its Google Search ranking

#108

Post by kccqzy »

bagacrap wrote: post #98
pessimizer wrote: post #68

So google's shitty search now economically incentivizes sites to destroy information. Can there be any doubt that Google destroyed the old internet by becoming a bad search engine? Could their exclusion of most of the web be considered punishment for being sites being so old and stable that they don't rely on Google for ad revenue?

I'll just assume you neglected to read TFA, because if you had, you would have discovered that it links to an official Google source that states CNET shouldn't be doing this.[1] [1] https://twitter.com/searchliaison/status/1689018769782476800

This is another problem of the entire SEO industry. Websites trust these SEO consultants and growth hackers more than they trust information from Google itself. Somehow, it becomes widely accepted that the best information on Google ranking is from those third parties but not Google.

Re: CNET is deleting old articles to try to improve its Google Search ranking

#109

Post by svachalek »

bagacrap wrote: post #98
pessimizer wrote: post #68

So google's shitty search now economically incentivizes sites to destroy information. Can there be any doubt that Google destroyed the old internet by becoming a bad search engine? Could their exclusion of most of the web be considered punishment for being sites being so old and stable that they don't rely on Google for ad revenue?

I'll just assume you neglected to read TFA, because if you had, you would have discovered that it links to an official Google source that states CNET shouldn't be doing this.[1] [1] https://twitter.com/searchliaison/status/1689018769782476800

If Google says one thing and rewards a different thing, guess which one will happen.
Post reply on HN