Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kogan.com mysteriously disappears from Microsoft Search Engines

kogan.com

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#31

Post by blauwbilgorgel »

They serve their robots.txt as a "application/octet-stream" not "text/plain". Might be confusing some search engines?

Also according to Internet Archive their robots.txt has changed in the last months.

This was a version that was archived:

http://web.archive.org/web/20080104044407/http://www.kogan.c...

At a glance, they are whitelisting some bots, including Googlebot, but not including Bingbot, and disallowing the rest.

  User-agent: Googlebot
  Disallow: 

  User-agent: *
  Disallow: /
Another marketing stunt?

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#32

Post by michaelt »

chrislomax wrote: post #27

Anyone else thinking they added a robots.txt entry for bing to disallow, waited to get de-listed then removed the offending robots.txt entry, posted this news article knowing bing would pick them back up in a couple of days?

The HTTP last modified header for www.kogan.com/robots.txt has Last-Modified: Wed, 06 Jun 2012 06:14:35 GMT

It's possible they did this and they were cunning enough to turn back the last-modified clock.

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#33

Post by brudgers »

Just a thought, but I seem to recall that Google was giving higher rankings to sites that advocated upgrading to Chrome, and created a bit of embarrassment.

Given that Kogan.com advocates upgrading and all the linking to their website which occurred recently, perhaps it seems possible that a Microsoft algorithm designed to avoid similar SEO manipulation was triggered.

In other words, the way in which the Kogan.com story unfolded was sufficiently similar to the PR move by Google, that it triggered a Microsoft safeguard.

http://www.pcworld.com/article/247257/google_disciplines_its...

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#35

Post by UnoriginalGuy »

Their robots.txt file is being delivered as the wrong content type for me. It is being served as a "application/octet-stream" instead of a "text/html"

I have no idea what consequences this might have since I've never seen another site which does it. Some quick Googling turned up nothing.

Maybe Bing choked on the wrong content type and decided to blackball their entire domain in order to avoid indexing the unindexable?

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#36

Post by chrislomax »

brudgers wrote: post #33

Just a thought, but I seem to recall that Google was giving higher rankings to sites that advocated upgrading to Chrome, and created a bit of embarrassment. Given that Kogan.com advocates upgrading and all the linking to their website which occurred recently, perhaps it seems possible that a Microsoft algorithm designed to avoid similar SEO manipulation was triggered. In other words, the way in which the Kogan.com st…

They did this to themselves as it's against their guidelines to pay for page ranked incoming links. It was simply because in the end they were paying for the link to their site

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#37

Post by rmc »

jbigelow76 wrote: post #29
chrislomax wrote: post #27

Anyone else thinking they added a robots.txt entry for bing to disallow, waited to get de-listed then removed the offending robots.txt entry, posted this news article knowing bing would pick them back up in a couple of days?

If that's what they did (no telling really) that would be pretty clever PR wise.

Clever… and dishonest.

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#38

Post by UnoriginalGuy »

blauwbilgorgel wrote: post #31

They serve their robots.txt as a "application/octet-stream" not "text/plain". Might be confusing some search engines? Also according to Internet Archive their robots.txt has changed in the last months. This was a version that was archived: http://web.archive.org/web/20080104044407/http://www.kogan.c... At a glance, they are whitelisting some bots, including Googlebot, but not including Bingbot , and disallowing the r…

Your link is pointing at 2008. That is four years ago. Hardly proof of anything.

You are right about the content type however.

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#39

Post by ot »

"we noticed that the www.kogan.com website stopped appearing in organic search results on Microsoft backed search engines"

They don't say explicitly, let alone prove, that the result ever appeared on Bing, while "stopped" would imply that.

If the comments about the wrong robots.txt got it right, it is well possible that the website was never indexed by Bing, and they just noticed it (or decided to talk about it) now.

Re: Kogan.com mysteriously disappears from Microsoft Search Engines

#40

Post by jawr »

Maxious wrote: post #30
citricsquid wrote: post #8

Microsoft themselves want their users to upgrade to latest browsers (see: http://www.ie6countdown.com/ ) so the idea that they would take issue with someone else doing this (so much so that they'd remove them from bing) seems a bit.. unlikely.

For those who are not familiar with Mr. Kogan: http://en.wikipedia.org/wiki/Kogan_Technologies#Controversie... As another commenter suggested, I would not put it past him that he blocked bing somehow just for a publicity stunt. The wikipedia article hasn't even caught up with last week's stunt (or there are just so many that it cannot list them all): http://www.gizmodo.com.au/2012/07/kogan-launches-campaign-to...

I like these guys, particularly for the 'Website glitch', I can't imagine many retailers would take a similar stance.

Seems to me that these guys are great marketers.

Post reply on HN