Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Memory Loss

tbray.org

Post reply on HN
Print view View original

91–100 of 552 posts

Re: Google Memory Loss

#91

Post by lolc »

jsnell wrote: post #65

> I think Google has stopped in­dex­ing the old­er parts of the We­b. I think I can prove it. Google’s com­pe­ti­tion is do­ing bet­ter. The first sentence is just common sense, and no particular proof is needed. The last sentence might or might not be true, but the anecdotes in this article say nothing about whether or not its true. The problem is that we don't know how Tim selected these two particular pages as exa…

> The first sentence is just common sense, and no particular proof is needed.

How is this common sense?

> The problem is that we don't know how Tim selected these two particular pages as examples.

Yes we do know. He was using Google to find his own old stuff over the years. Some content he was referring regularly disappeared from Google's results. These pages had previously been included in the results.

> Another reasonable method would be to look at aggregate referer trends; is traffic from Google to old pages decreasing faster than traffic from Bing to those pages.

Yes that would be interesting.

I've been wondering whether Google would actually purge the URL too. The Googlebot used to be very persistent in retrying "404 not found" results.

Re: Google Memory Loss

#92

Post by anfilt »

brianpan wrote: post #78
anfilt wrote: post #7

I would not be surprised if google still has the data. Not sure how google handles things internally. However, google needs to pull up the results fast. So they might have 4 billion results with the word "water" in it. They only make tiny portion of that available. So if I type the words "Hot water" google it looks at the subset of pages with words "Hot" and the word "Water" So google must pull the pages that have bo…

Google does not need to pull up ALL results fast. It only needs to return 10 results quickly. That's not relevant to the article, which says that the results are not available AT ALL. (Although as of my posting the two articles seem to be available again.)

True, but google is not searching there entire index for those. A simple linear search takes N time. So for a word that occurs billions of times. Google is not going to go through that entire list. They might use some clever hashing to jump around, and sorting. However, when trying to intersect two keywords they either have to pre-generate the intersection or make the data set they are intersecting small enough to get those 10 results quickly.

Re: Google Memory Loss

#93

Post by userbinator »

I've noticed this many times too, particularly recently, and I call it "Google Alzheimer's" --- what was once a very powerful search engine that could give you thousands (yes, I've tried exhausting its result pages many times, and used to have much success finding the perfect site many dozens of pages deep in the results) of pages containing nothing but the exact words and phrase you search for has seemingly degraded into an approximation of a search engine that has knowledge of only very superficial information, will try to rewrite your queries and omit words (including the very word that makes all the difference --- I didn't put it in the search query for nothing!), and in general is becoming increasingly useless for finding the sort of detailed, specific information that search engines were once ideal for.

To add insult to injury, if you do try to make complex and slightly varying queries and exhaust its result pages in an effort to find something you know exists, very often it will think you're a robot and present you with a CAPTCHA, or just ban you completely (solving the CAPTCHA just gives you another, and no matter how many you solve it keeps refusing to search; but they probably benefit from all the AI help you just gave them, what bastards...) for a few hours.

Google had the biggest most comprehensive index for many years, which is why it was my sole search engine. Now I'm often finding better results with Bing, DuckDuckGo, Yahoo, and even Yandex, but part of me is very worried that large and extremely valuable parts of the Web are, despite still being accessible, simply "falling off the radar".

Re: Google Memory Loss

#94

Post by mark-r »

mark-r wrote: post #31
pbw wrote: post #2

It seems possible these particular articles fell out of Google's index for some other reason than they are not indexing "old" stuff. That's kind of a big leap to make without many more examples.

Q: What's the difference between amnesia and Alzheimer's? A: I don't remember.

And by that I really mean it doesn't matter why they forgot. The fact that they forgot means there's a sickness.

Re: Google Memory Loss

#95

Post by scoot »

ohazi wrote: post #17

I've convinced myself that this happens in gmail / hangouts history search too. It'll very confidently tell you that here are the only six results for your search term going back to the beginning of time, but if you go and manually dig up something that you know is there from ten years ago, then all of a sudden there are seven results the next time you search for the same term. I haven't done this methodically, and I…

Can confirm. Google, the search company, has lst the ability to accurately search even the data that it holds, never mind any other.

Re: Google Memory Loss

#96

Post by jxramos »

imgabe wrote: post #81

> They’ve nev­er claimed to in­dex ev­ery word on ev­ery page. Not in those words, but they do claim to aspire to “Organize the world’s information and make it universally accessible and useful.”[1] which ought to include old web pages. They've gone to the effort of finding out of print books and digitizing them to make those searchable so it doesn't seem like a ten year old web page should be such a stretch. [1] htt…

you'd think it would at least come up in the internet archive if not anywhere else.

Re: Google Memory Loss

#97

Post by Puer »

aglionby wrote: post #69
ohazi wrote: post #17

I've convinced myself that this happens in gmail / hangouts history search too. It'll very confidently tell you that here are the only six results for your search term going back to the beginning of time, but if you go and manually dig up something that you know is there from ten years ago, then all of a sudden there are seven results the next time you search for the same term. I haven't done this methodically, and I…

I've had this for Chrome history as well. There have been multiple times where I'm sure I've browsed a site with some keyword in the title and it just doesn't show up in search. I don't tend to have a clue about the time window it would be in either so I can't go looking for it, so I can't prove it.

Chrome history may as well be useless and/or developed by the same people who created Reddit search.

Re: Google Memory Loss

#98

Post by taneq »

harshreality wrote: post #72

Are Google using a neural nets as an integral part of search indexing yet? It's well known that there are a bunch of metrics that go into which results to return — metrics including things like pagerank, (probably) historical value (# of clicks when the page appears in results), and social media popularity. I wouldn't be surprised if Google has experimented with training models to predict most of those metrics, given…

From bits and pieces they've posted, it sounds like they use some all-encompassing glob of statistical inference that they call RankBrain, which almost certainly includes some deep-learning components. They've said that the old PageRank algorithm is now one input into RankBrain.

Re: Google Memory Loss

#99

Post by jxramos »

smoyer wrote: post #89

I've done essentially the opposite ... I use DuckDuckGo as my everyday search engine and only use Google when there's not an acceptable result there. Over the last couple of years, the number of times per week that I switch to Google has probably halved.

that's been my experience too, sometimes I need the power of Google to infer the tenuous relationships my mind is able to regurgitate on a subject I can't recall precise terminology for. I cast a loose net of terms fumbling for something feeble I can hook on to dredge up from my mind to unveil the whole glorious thing I'm chasing.

Re: Google Memory Loss

#100

Post by monort »

incompatible wrote: post #68
yesenadam wrote: post #63

Earlier quoted context omitted.

Yes, verbatim is distinctly broken sometimes. edit: It's not tedious for me on my browser, just click Verbatim on the LHS of page. (Can select that or All results )

I don't see that on the LHS. It would be nice if there's a link to it, something like https://www.google.com.au/search?verbatim=true that I could bookmark. Edit: or somehow set as the browser's default search engine.

add tbs=li:1 to the url:

http://google.com/search?q=hacker+news&tbs=li:1

Post reply on HN