Live data from Hacker News

Can you search “Common Crawl” on Google with keywords only?

1–2 of 2 posts

Post by chauchakching » Sun, Oct 03, 2021, 3:18 PM UTC

I didn't expect how hard it was to search it on google. All I got were "top x crawler"/"scrape with 1 click"/"scrapy". Is it bad SEO or something?

Re: Can you search “Common Crawl” on Google with keywords only?

Post by Someone » Sun, Oct 03, 2021, 4:52 PM UTC

If you mean https://commoncrawl.org/: of course you can. Grab a few specific words from the page and away you go.

Sample searches:

  an open repository of web crawl data ⇒ first hit
  open web crawl data ⇒ first hit
  free web crawl data ⇒ third hit

it seems “data” is essential here.