Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Removing my site from Google search

btao.org

Post reply on HN
Print view View original

21–30 of 119 posts

Re: Removing my site from Google search

#21

Post by jefftk »

onion2k wrote: post #16
avipars wrote: post #12

seems like it would hurt your traffic

That depends on where your traffic originates from. Back when I tracked people on my site, I found I got very little from search results. Most of it (> 95%) came from links from social media and Github. On a blog that's heavily about privacy I wouldn't expect much to come from Google. Also, so what if the numbers go down? If your reason from writing a blog is to see a number on a screen then what does that actually g…

> so what if the numbers go down? If your reason from writing a blog is to see a number on a screen then what does that actually give you?

Traffic numbers are not an end in themselves, but are a decent proxy for "are other people getting value out of what I write?"

Re: Removing my site from Google search

#22

Post by lucb1e »

avipars wrote: post #12

seems like it would hurt your traffic

If that's your goal. Personally I host content that people can use or not. I'll link friends if I want them to see it. Visitors don't cost me anything, it doesn't really bring me anything (other than ego?) to have visitors either. Hence I saw fit to also block google (two years ago already apparently, I thought it was much more recent) and it didn't negatively impact my site in any way.

Re: Removing my site from Google search

#23

Post by thih9 »

Animats wrote: post #13
tao_oat wrote: post #4

Earlier quoted context omitted.

Hi, author here. Google stopped supporting robots.txt [edit: as a way to fully remove your site] a few years ago, so these meta tags are now the recommended way of keeping their crawler at bay: https://developers.google.com/search/blog/2019/07/a-note-on-...

Did you think that mighty Google would pay attention to your puny "noindex" tag? Ha!

According to google's own docs, this should work.

> You can prevent a page from appearing in Google Search by including a noindex meta tag in the page's HTML code, or by returning a noindex header in the HTTP response.

Source: https://developers.google.com/search/docs/advanced/crawling/...

Re: Removing my site from Google search

#24

Post by lucb1e »

judge2020 wrote: post #20
TedDoesntTalk wrote: post #11

Earlier quoted context omitted.

Doesn’t that mean you have to know every IP Address used by Google bot now and in the future?

The way to check googlebot (in a way that will be resistant to expansion of Googlebot's IP ranges in the future) is to perform hostname lookup, with dns lookup as well to verify that the rDNS isn't a lie: https://developers.google.com/search/docs/advanced/crawling/...

Indeed, this was one of the things I considered (note I'm not OP), but then I didn't really want to rely on DNS. https://duckduckgo.com/?q=it's+always+DNS

Re: Removing my site from Google search

#26

Post by bryanrasmussen »

thih9 wrote: post #23
Animats wrote: post #13

Earlier quoted context omitted.

Did you think that mighty Google would pay attention to your puny "noindex" tag? Ha!

According to google's own docs, this should work. > You can prevent a page from appearing in Google Search by including a noindex meta tag in the page's HTML code, or by returning a noindex header in the HTTP response. Source: https://developers.google.com/search/docs/advanced/crawling/...

I mean technically that says that your site won't appear in search results, not that your site won't be used to profile people, determine other site ratings based on your site's content etc.

they won't show your site's content, but that doesn't mean they won't use your site's content.

Re: Removing my site from Google search

#27

Post by freediver »

avipars wrote: post #12

seems like it would hurt your traffic

Only if there is relevant traffic from Google to begin with, which is highly unlikely for a site like this. A high percentage of results in almost every Google search comes from the closed circle of the same top 10,000 sites or so.

This is the beauty of a protest like this, because this site does have valuable content, and if enough sites like this joined the protest it could actually hurt the relevancy of the Google index, that by the time Google figures out is valuable, would not be allowed to index anymore.

Re: Removing my site from Google search

#28

Post by mro_name »

Animats wrote: post #13
tao_oat wrote: post #4

Earlier quoted context omitted.

Hi, author here. Google stopped supporting robots.txt [edit: as a way to fully remove your site] a few years ago, so these meta tags are now the recommended way of keeping their crawler at bay: https://developers.google.com/search/blog/2019/07/a-note-on-...

Did you think that mighty Google would pay attention to your puny "noindex" tag? Ha!

yes, I do think that
Post reply on HN