Google de-indexed Bear Blog and I don't know why • Hacker News Board

Re: Google de-indexed Bear Blog and I don't know why

#191

Post by baobun » Sun, Dec 14, 2025, 10:08 AM UTC

FuturisticLover wrote: post #119

nubinetwork wrote: post #97

Earlier quoted context omitted.

Check that you're not routing unnamed SNI requests to your web content. If someone sets up a reverse proxy with a different domain name, google will index both domains and freak out when it sees duplicate content. Also make sure you're setting canonical tags properly. Edit: I'd also consider using full URLs in links rather than relative paths.

Canonical Tags are done perfectly. Never changed them, and the blog is quite old too. I found a pattern where Google considers a page a duplicate because of the URL structure. For example: www.xyz.com/blog/keyword-term/ www.xyz.com/services/keyword-term/ www.xyz.com/tag/keyword-term/ So, for a topic, if I have two of the above pages, Google will pick one of them canonically despite different keyword focus and intent.…

I guess a workaround could be to tell googlebot to keep off /tag/?

Re: Google de-indexed Bear Blog and I don't know why

#192

Post by MobiusHorizons » Sun, Dec 14, 2025, 4:05 PM UTC

weird-eye-issue wrote: post #134

MobiusHorizons wrote: post #128

Earlier quoted context omitted.

I think the question is “how are the behavior of random spammers on your search page getting picked up by the crawler”? The assumption with cache is that searches of one user were being cached so that the crawler saw them. Other alternatives I can imagine are that your search page is powered by google, so it gets the search terms and indexes the results, or that you show popular queries somewhere. But you have to adm…

You just link to that page from a page that Google crawls. Cache isn't involved unless you call links caching

Ah that makes sense, thanks for clarifying.

Re: Google de-indexed Bear Blog and I don't know why

#193

Post by jfil » Sun, Dec 14, 2025, 4:09 PM UTC

paganel wrote: post #39

leoedin wrote: post #21

Earlier quoted context omitted.

I don’t think I’ve met anyone in the UK who routinely checked tripadvisor for anything! I just checked a few local restaurants to me in London that opened in the last few years, and the ratio of reviews is about 16:1 for google maps. It looks like stuff that’s been around longer has a much better ratio towards trip advisor though. Almost certainly Instagram/tiktok are though. I know a few places which have been ruine…

Not in the UK, but from Romania, I last checked Tripadvisor back in 2012, and that was for a holiday stay in the Greek islands. Google Maps has eaten the lunch of almost all of the entrants in this space, and I say that having worked for a local/Romanian "Google places"-type of company, back in 2010-2012 (after which Google Places came in, ~~stole~~ scrapped some of our data and some of our direct competitor's data a…

I'd be curious to read that whole story of Google putting the conpany out of business. Would you consider writing about it in a post?

Re: Google de-indexed Bear Blog and I don't know why

#194

Post by [deleted] » Sun, Dec 14, 2025, 5:26 PM UTC

ImPostingOnHN wrote: post #112

xnx wrote: post #60

The author doesn't know the cause but states "The whole affair is Google’s fault"?

You're absolutely right, we should give google the opportunity to defend itself. What's the phone number google provides so their victims can speak directly with an informed googler to discuss it? What's that? Google doesn't publish a phone number for their victims to do that? They just victimize and hide? Okay then google, here's your chance: please reply to this-here post of mine, with a plausible explanation that…

[deleted]

Re: Google de-indexed Bear Blog and I don't know why

#195

Post by FuturisticLover » Sun, Dec 14, 2025, 5:36 PM UTC

baobun wrote: post #191

FuturisticLover wrote: post #119

Earlier quoted context omitted.

Canonical Tags are done perfectly. Never changed them, and the blog is quite old too. I found a pattern where Google considers a page a duplicate because of the URL structure. For example: www.xyz.com/blog/keyword-term/ www.xyz.com/services/keyword-term/ www.xyz.com/tag/keyword-term/ So, for a topic, if I have two of the above pages, Google will pick one of them canonically despite different keyword focus and intent.…

I guess a workaround could be to tell googlebot to keep off /tag/?

I did that. But I am not sure what to do with blog and service pages. Can't tell Google to off one of them. For now, I have priortized service page as canonical.

Re: Google de-indexed Bear Blog and I don't know why

#196

Post by marbu » Sun, Dec 14, 2025, 5:56 PM UTC

Run into a similar problem with my blog this year. After spending some time trying to resolve it, I just gave up.

I can understand that every now and then Google changes it's rules and validation procedures, so that what used to work now gets removed from the index out of sudden, given their fight with spam and slop. But what I'm struggling to understand is how could Google crawler and Google Search Console be so bad so that:

* google crawler stops fetching sitemap out of sudden, even though Google claims it's an important signal for the search engine * requesting sitemap refresh via GSC fails on "unknown" error, which is puzzling considering according to my web logs, nobody tried to load it between my request and the error * after fixing an error, validation job gets stuck for weeks, only to fail for unclear error * random deindexing events as explained in the post

And I don't buy the argument that this is necessary for Google to deal with spam, because Bing Webmaster Tools just works flawlessly, and they have to deal with it as well.

I don't understand how a small business deal with this kind of issues.

Re: Google de-indexed Bear Blog and I don't know why

#197

Post by account42 » Mon, Dec 15, 2025, 9:49 AM UTC

mohas wrote: post #151

donatj wrote: post #81

About six months ago Ahrefs recommended I remove some Unicode from the pathing on a personal project. Easy enough. Change the routing, set up permanent redirects for the old paths to the new paths. I used to work for an SEO firm, I have a decent idea of best practices for this sort of thing. BAM, I went from thousands of indexed pages to about 100 See screenshot: https://x.com/donatj/status/1937600287826460852 It's b…

exactly my experience, suddenly thousands of non indexed pages, never figured out why, had to disband the business as it was content website selling ads.

Good riddance.

Re: Google de-indexed Bear Blog and I don't know why

#198

Post by _heimdall » Tue, Dec 16, 2025, 12:00 AM UTC

ethbr1 wrote: post #186

_heimdall wrote: post #180

Earlier quoted context omitted.

I wouldn't personally want companies to be punished without evidence of collusion. A company isn't doing anything wrong by earning market share, and companies aren't doing anything wrong if they happen to move in a similar direction based on market incentives. If we think free markets are generally going to move in the right direction, we should just want companies to want to fill market gap and outcompete. I don't a…

So there's no collusion or misbehavior, and the market ends up as a duopoly: one participant has 70% and another has 25%. You don't think that alone distorts the market enough to merit intervention to encourage more competitors? If you tie intervention to proven malfeasance, you allow abusers to skirt the rules for decades, entrench their positions with obscene profits, and then maybe eventually face consequences if…

Yes, when crimes are committed it is often hard to prove and you won't catch them all. That's by design and a fundamental part of how our legal system was designed in the first place.

Having a duopoly as you described isn't in itself a crime, nor should it be. If they are skirting the rules such that they are breaking the rules, enforcement should step in as there is actually something to enforce. If the only "crime" is winning market share, what's the problem?

Re: Google de-indexed Bear Blog and I don't know why

#199

Post by mohas » Wed, Dec 24, 2025, 8:53 AM UTC

account42 wrote: post #197

mohas wrote: post #151

Earlier quoted context omitted.

exactly my experience, suddenly thousands of non indexed pages, never figured out why, had to disband the business as it was content website selling ads.

Good riddance.

now I don't have clients and I'm poor :(