Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google search only has 60% of my content from 2006

tablix.org

Post reply on HN
Print view View original

151–160 of 168 posts

Re: Google search only has 60% of my content from 2006

#151

Post by dcbadacd »

andrewprock wrote: post #146
Theodores wrote: post #58

Earlier quoted context omitted.

Have you been reading SO answers?!? It works fine in Edge. Why don't you try? IE latest - there is no such thing. The latest steam engines, the latest CRT screens and the latest buggy whips probably don't do CSS Grid either. IE exists for compatibility with enterprise apps but isn't 'latest'. It is legacy. And the content will still work even if CSS Grid is not supported, just the layout won't be perfect.

Checking here: https://netmarketshare.com/browser-market-share.aspx It looks like IE is still the #3 browser, ahead of both Edge and Safari.

Heavily depends on your target market, neither does it just hurt to say "This browser is unsupported for being too outdated" we do it for why not :/

Re: Google search only has 60% of my content from 2006

#152

Post by tempestn »

YawningAngel wrote: post #61
tempestn wrote: post #7

Earlier quoted context omitted.

I'm not sure I buy that they have no choice. For websites that literally have an infinite number of (dynamically generated) pages, sure, they could detect that and exclude them. But we're talking about unique, static pages here. And they don't even have to store the whole page, just the indexed info. I read this as, they could, but it's cheaper not to, and most people won't notice anyway.

I'm not sure that's true. How can one automatically determine whether a page is unique or static? As a trivial example, a URL path that accepts arbitrary strings and hashes them generates unique, immutable pages, but obviously cannot be crawled in its totality.

So, this issue isn't about sites that Google can't crawl in totality, it's about sites where they discard pages that they have crawled. If a site has less than [large number] of pages, there would be no need to worry about it; they could just index them all. But it's not like their indexing algorithm is operating naively either—for sites with a lot of pages, there's plenty of analysis they can do to determine things like whether the pages contain coherent text and other such things, to determine whether the information is worth indexing.

In the case described here though, these pages were actually indexed at one point; Google just decided that once they reached a given age, they were no longer necessary to remember. They could have simply decided to keep them instead.

Re: Google search only has 60% of my content from 2006

#153

Post by alister »

samstave wrote: post #90
alister wrote: post #13

Why does Google deeply index those useless telephone directory sites? Try searching for the impossible U.S. phone number "307-139-2345" and you'll see a bunch of "who called me?" or "reverse phone number lookup" sites. Virtually all of those sites are complete garbage. They make no attempt to collect numbers from telephone directories or from the web. They won't identify a number as being the main phone number for Di…

Aside from what others have said ITT, i have a personal hate for the general fact that we cannot lookup a phone number on the internet with accuracy and ease. FFS, 411 was amazing before the web. Also, in about 1989 my friend and i used to have a contest between us; to call 411 and see who could keep the 411 operator on the phone the longest. This was a fun social engineering exercise for 14 year old nerds who like t…

> we cannot lookup a phone number on the internet with accuracy and ease

I wondered the same thing and just to speculate, here's my list of reasons on why phone number search is so awful:

- As far as I know, not a single cell phone carrier publishes a telephone directory (whether opt-in or opt-out). So there's no (public) data to index.

- Some landline carriers still publish telephone directories, but of course landlines are dying out. And I remember reading that 30-50% of landline subscribers choose to be unpublished or unlisted anyway. So that source of phone number data is drying up.

- Because international phone calls have become so cheap and caller ID is now easily spoofable, spam and scam calls have become huge. So no one wants their phone number to publicly accessible these days.

- In the early web years, there were legitimate phone directory websites who appeared to have collected their data from landline telephone directories and "city directories" (if anyone still remembers those things). But I guess they didn't find a good way to monetize the service, so the honest phone lookup sites died off.

Re: Google search only has 60% of my content from 2006

#154

Post by 3xblah »

GarrisonPrime wrote: post #73
3xblah wrote: post #57

Earlier quoted context omitted.

What makes no sense to me about this blocking scenario is that the pages being searched for are presumably non-commercial ones that no one else is searching for. In other words, they are in low demand. It follows that a monopoly search engine would have little reason to block "robots" from copying these pages, maybe to appear on some mythical competing search engine; almost no one is searching for them. The results p…

In other words, true democracy kills any chance for individuality.

Can you elaborate?

Re: Google search only has 60% of my content from 2006

#155

Post by h0p3 »

eddieh wrote: post #145
rigorman wrote: post #143

Earlier quoted context omitted.

Thoughts after thinking about this comment and thread for a day: Has the time come for wiki directory of non-commercial (possibly: advertising-free, cookie-free) sites with robust, actually valuable information, and other sites that are doorways to them (think: topical forums, even revived webrings, etc)? Could this feasibly get enough action to be useful?

Yes. I was looking for a modern _human_ curated directory of web content just the other day and found nothing usable. I don't think that excluding commercial entries would be necessary, but perhaps there could be some way to filter commercial entries out. Ad-free, JS-free, and cookie-free would be ideal.

I recommend: https://href.cool/ as a sick (though highly particular) example.

Re: Google search only has 60% of my content from 2006

#156

Post by h0p3 »

rigorman wrote: post #143
alister wrote: post #13

Why does Google deeply index those useless telephone directory sites? Try searching for the impossible U.S. phone number "307-139-2345" and you'll see a bunch of "who called me?" or "reverse phone number lookup" sites. Virtually all of those sites are complete garbage. They make no attempt to collect numbers from telephone directories or from the web. They won't identify a number as being the main phone number for Di…

Thoughts after thinking about this comment and thread for a day: Has the time come for wiki directory of non-commercial (possibly: advertising-free, cookie-free) sites with robust, actually valuable information, and other sites that are doorways to them (think: topical forums, even revived webrings, etc)? Could this feasibly get enough action to be useful?

I think you should talk to kicks @ https://kickscondor.com/.

Re: Google search only has 60% of my content from 2006

#157

Post by mirimir »

mxd3 wrote: post #139
mirimir wrote: post #41

Earlier quoted context omitted.

See https://uk.pcmag.com/news-analysis/120580/how-a-vpn-review-s... for discussion of major trickage.

The only “trick” here is that the author of the website has a fake name. Otherwise, having good content that people link to is not a trick.

No, there was more to it than that. I spent some time, last year, looking into those pay-for-rating VPN "review" sites. And Google displayed some very odd behavior in searches involving TheBestVPN.com and VPN services that had paid it for top rankings.

I'm no SEO guru, but I suspect that some of those VPN services created numerous clones, which all linked to TheBestVPN.com, and so improved its ranking. For example, ExpressVPN had at least 128 clones. Such as expressvpn..., buy-express-vpn-..., get-xpress-vpn...., and xpress-vpn.... I used myip.ms to get hosting information, and they were linked. Also, I bought subscriptions from a few of them, and they all provided working ExpressVPN apps, with the right certificates. And I found no evidence of affiliate codes in the traffic.

Re: Google search only has 60% of my content from 2006

#158

Post by JetSpiegel »

ma2rten wrote: post #94
JetSpiegel wrote: post #67

Earlier quoted context omitted.

https://www.google.com/search?hl=en&q=307%2D139%2D2345 You comment now comes up first, but the rest of the results all try to contact googlesyndication.com, so ads? Google will not exclude sites that literally give them money.

I work on AdSense at Google. We need to crawl all the pages that serve our ads so that we can show ads that are relevant. We actually store those crawl results in a different place than search to prevent exactly that problem. We could probably save a lot of money if we consolidated those indices, but we don't do that prevent biasing search results. As a result a large percentage of pages that show Google Ads are not…

> As a result a large percentage of pages that show Google Ads are not in the search index.

Those results are organic, at least they were not marked as ads. That's even worse, it means the regular crawler is preferring generated phone number sites over blogs. That's the real money waster right there.

> We need to crawl all the pages that serve our ads so that we can show ads that are relevant.

Why? I though Google's business model was tracking users to show relevant ads to them. You say it's more like DuckDuckGo, or the old magazine ads: film magazines get blockbuster trailers, gardening magazines get compost ads.

Either the ads are personalized, so the surrounding content doesn't matter, or the ads are "static", so why track everyone all the time, then?

> If we show ads on google.com we get 100% of the revenue. If show ads on reversephonelookup.it they get majority of the revenue. There is a limited amount of advertiser demand. Instead of manipulating the organic search results, it would be more profitable for google to just show more ads on the search page or inflate the ad price or something.

As your sibling comments imply, this works like a protection racket. The profit is closing the market to webmasters that don't allow ads at all, or that use non-Google networks.

Re: Google search only has 60% of my content from 2006

#159

Post by mathnmusic »

eddieh wrote: post #145
rigorman wrote: post #143

Earlier quoted context omitted.

Thoughts after thinking about this comment and thread for a day: Has the time come for wiki directory of non-commercial (possibly: advertising-free, cookie-free) sites with robust, actually valuable information, and other sites that are doorways to them (think: topical forums, even revived webrings, etc)? Could this feasibly get enough action to be useful?

Yes. I was looking for a modern _human_ curated directory of web content just the other day and found nothing usable. I don't think that excluding commercial entries would be necessary, but perhaps there could be some way to filter commercial entries out. Ad-free, JS-free, and cookie-free would be ideal.

Although it's early days for this project but do check out: https://github.com/learn-awesome/learn-awesome

Re: Google search only has 60% of my content from 2006

#160

Post by mxd3 »

nikeee wrote: post #107
alister wrote: post #13

Why does Google deeply index those useless telephone directory sites? Try searching for the impossible U.S. phone number "307-139-2345" and you'll see a bunch of "who called me?" or "reverse phone number lookup" sites. Virtually all of those sites are complete garbage. They make no attempt to collect numbers from telephone directories or from the web. They won't identify a number as being the main phone number for Di…

I always assumed they collect phone numbers this way. People google a phone number and click on a search result. The site can - by looking at the referrer - extract the phone number and infer that it belongs to someone (and use it for any purpose). It's a similar mechanism that some forums use to highlight the terms that were part of the google query which lead to this site.

You can’t get extract search query, not from referrer or with analytics software. Google changed that starting around 2011 for logged in users.
Post reply on HN