Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Internet search tips

gwern.net

Post reply on HN
Print view View original

11–20 of 86 posts

Re: Internet search tips

#11

Post by winwang »

Yemeth wrote: post #9
imchillyb wrote: post #5

Soon, there will be an LLM that can utilize indexes and large scale distributed databases, and that will replace every search engine on earth. Google should be quaking, and pissing themselves. It won't be their bardum, and they won't have the financial power to purchase said LLM. Poof. Goodbye useless SEO spam, advertisement hell. We hated your guts.

I'm not sure how LLMs are going to be immune to SEO spam and advertising. As if human nature would magically transform and people would stop buying stupid stuff. LLMs are already being used to make the web even less useful, by shitting out vast ammounts of meaningless and even outright wrong text for SEO purposes. And don't forget the systems are being trained on the web in the first place… using a LLM that is able t…

I think the idea is that, if we (some of us) can figure out when something is SEO spam or rather, generally low quality, an LLM should be able to but faster and more quantitatively why.

Re: Internet search tips

#12

Post by 8bitsrule »

Many good scarce tips in there (once past Google Scholar, unless you're monopoly-oriented). 'Dealing with paywalls' for example.

He did mention IA searches, but I didn't spot a mention of their https://scholar.archive.org/ with "over 25 million research articles and other scholarly documents preserved in the Internet Archive."

For book metadata, I find https://openlibrary.org/ search has a lot of 'MARC' type data. Esp useful for books with many editions.

Some services are getting more restrictive. e.g. WorldCat recently got harder to use, rejecting many searches. But if you can find a book's OCLC ###, then https://www.worldcat.org/oclc/### works every time. Useful for finding local hardcopy if you've got your location turned on. (With an ISBN , WPedia will also do this for you at: https://en.wikipedia.org/wiki/Special:BookSources/ )

Re: Internet search tips

#13

Post by aardshark »

Daub wrote: post #3

If I had my way, I would introduce internet search techniques as a core module in all university programs. Too often I see in my students work evidence of lazy searching. It is as if they expect Google to be able to read their minds or even foresee their future intentions. Lack of variety of search terms is a key shortcoming. Also, lack of exploration of terms which are tangentially related to their search topic. An…

But search techniques change over time. You can see that in the responses in this thread. What used to work doesn't anymore.

Re: Internet search tips

#14

Post by chkal »

38 wrote: post #2

my problem is some operators just dont work any more, either on purpose or because of crappy quality control. for example, you used to be able to do: allintitle:Neil Diamond If You Go Away on YouTube, and get exactly what you would think, results with all those words in the title. but now, you dont: https://youtube.com/results?search_query=allintitle:Neil+Dia... now, I get crap like this: Neil Diamond & Shirley Basse…

> did I just have a stroke? those two searches are exactly the same.

I just had the exact same though while reading the page.

Re: Internet search tips

#15

Post by Jakob »

+1 for his tip of "Search The Internet Archive".

The other day I searched for the Tom & Jerry full episodes on the web to no avail (streaming platforms and video platforms like youtube only have rubbish cuts).

The internet archive has every episode starting from the first one in 1940, in an easily accessible player without any ads or recommendations: https://archive.org/details/tom-and-jerry-all-114-episodes

Re: Internet search tips

#16

Post by Thoeu388 »

I found it useful to screenrecord all my computer activity. It is OCR searchable. This way if I vaguely remember some bits, I can reconstruct my sources. It also preserves original text, in case sources get edited.

Re: Internet search tips

#18

Post by Thoeu388 »

For complement I would recommend Russian yandex, and vk.com (russian facebook). It is like snapshot of internet from 2018, before everything was hit by massive censorship. It has some very niche communities that are no longer on internet. For example Ancient Egypt history, western net is filled with alien theories, Russians are doing experimental archaeology. Also repair, electronics...

Also Telegram is like second internet, nobody talks about!

Re: Internet search tips

#20

Post by jcmeyrignac »

8bitsrule wrote: post #12

Many good scarce tips in there (once past Google Scholar, unless you're monopoly-oriented). 'Dealing with paywalls' for example. He did mention IA searches, but I didn't spot a mention of their https://scholar.archive.org/ with "over 25 million research articles and other scholarly documents preserved in the Internet Archive." For book metadata, I find https://openlibrary.org/ search has a lot of 'MARC' type data. Es…

I also recommend https://www.semanticscholar.org/ because you can easily retrieve some of the PDFs.
Post reply on HN