Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Memory Loss

tbray.org

Post reply on HN
Print view View original

71–80 of 552 posts

Re: Google Memory Loss

#71

Post by malchow »

tomc1985 wrote: post #23

I would bet this is one of those more subtle long-term effects that nobody really saw coming... when Google refocused search with an eye towards commercial results, I imagine it deprioritized a lot of the older, more innocent informational content lying around

This has been my experience. With Google I am constantly asking myself: Wait, what about all that glorious, smart, noncommercial web content that I know exists? Like the Stanford Philosophy Encyclopedia[2] or that economics professor's dataset that I remember being referenced in a podcast a year ago?

Google seems to have decided that Wikipedia is the only blessed noncommercial source of intelligence.

I guess, if I were to put it strongly, I'd say: using Google is not like using the Internet any longer.

FWIW, HNers may wish to check out Yewno[1], a knowledge search engine based in Redwood City that I've had the pleasure of being (tangentially) involved in.

[1] http://yewno.com

[2] Yes, I know this is indexed. It just frequently gets buried in my searches.

Re: Google Memory Loss

#72

Post by harshreality »

Are Google using a neural nets as an integral part of search indexing yet?

It's well known that there are a bunch of metrics that go into which results to return — metrics including things like pagerank, (probably) historical value (# of clicks when the page appears in results), and social media popularity.

I wouldn't be surprised if Google has experimented with training models to predict most of those metrics, given only content from the site itself, and tried using those models as a filter for what to index in the first place. If the NN is accurate enough, they can use it as a filter at indexing stage ("should I index this?") rather than at the results ranking stage (where real data, rather than NN model output, answers the question "should I show this page close enough to the top of results that someone will see it?").

Re: Google Memory Loss

#73

Post by flukus »

viraptor wrote: post #33
flukus wrote: post #16

De-indexing old stuff might not be a good idea, but I'm increasingly running into the problem of google (and DDG) returning old and outdated results, I wish they would put more weight on recent articles, or at least add the option too. The time filtering options just aren't enough.

Why is the drop down in tools not enough?

Because I often don't know the range I'm looking for, it could have been yesterday or it could have been 4 years ago. If I select last 12 months I might miss something 13 months ago. There's a lot of ambiguity in what I'm searching for (otherwise it wouldn't be called a lookup) and when I'm looking for current information then weighting by age is a lot more natural.

Another issue is that I don't know what the filter is selecting either, a 6 year old article might be better if it's been updated, but I can't tell from the interface what property is being filtered.

Re: Google Memory Loss

#75

Post by BjoernKW »

mherdeg wrote: post #67

I have some questions about information retrieval and SLOs: * Is there a metric of search quality which is appropriate here -- specifically, "when I search for [site:tbray.org rock roll], and receive a set of results, that set includes Tim's article"? What do we call this metric? The metric would be lower when the result set is empty (no relevant results returned) and higher when the result set contains the desired a…

This won’t answer all of questions but the measures you’re looking for are called ‘recall’ and ‘precision‘:

- recall: number of relevant documents retrieved / number of relevant documents

- precision: number of relevant documents in result set / number of documents in result set

Re: Google Memory Loss

#77

Post by graedus »

gonzo41 wrote: post #66
ohazi wrote: post #17

I've convinced myself that this happens in gmail / hangouts history search too. It'll very confidently tell you that here are the only six results for your search term going back to the beginning of time, but if you go and manually dig up something that you know is there from ten years ago, then all of a sudden there are seven results the next time you search for the same term. I haven't done this methodically, and I…

Hate to side with the big guys, but its a free service. Beggars can't be choosers. They probably dump indexes after a while for content older than x. Seems fairly reasonable actually.

You're paying with your data, same as with Facebook and many others. These extremely successful businesses are obviously able to make plenty of money using that data.

Re: Google Memory Loss

#78

Post by brianpan »

anfilt wrote: post #7

I would not be surprised if google still has the data. Not sure how google handles things internally. However, google needs to pull up the results fast. So they might have 4 billion results with the word "water" in it. They only make tiny portion of that available. So if I type the words "Hot water" google it looks at the subset of pages with words "Hot" and the word "Water" So google must pull the pages that have bo…

Google does not need to pull up ALL results fast. It only needs to return 10 results quickly.

That's not relevant to the article, which says that the results are not available AT ALL. (Although as of my posting the two articles seem to be available again.)

Re: Google Memory Loss

#80

Post by lelandbatey »

1024core wrote: post #61
StavrosK wrote: post #47

DuckDuckGo seems to have no problem returning the result, even with general terms: https://duckduckgo.com/?q=tim+bray+rock+roll+animal

DDG uses Bing as the backend.

I don't believe that's true, at least not in the way you seem to imply. It is true that Yahoo and Bing are literally the same search engine on the backend, but that is not true of DuckDuckGo and Bing.

To quote the first line of the DDG wikipedia article on how it works[0]:

> DuckDuckGo's results are a compilation of "over 400" sources, including Yahoo! Search BOSS; Wikipedia; Wolfram Alpha; Bing; its own Web crawler (the DuckDuckBot); and others.

[0] - https://en.wikipedia.org/wiki/DuckDuckGo#Overview

Post reply on HN