Scraping made easy with jQuery and SelectorGadget (and Node.js!)

Post by DTrejo » Sat, Jan 22, 2011, 3:41 AM UTC

Post by mmaunder » Sat, Jan 22, 2011, 5:00 AM UTC

  perl -MLWP::UserAgent -e 'map { $_ =~ s/([^([^]*>(\d+ points)/print "$1 $2 $3 $4\n" if($i++ new->get("http://news.ycombinator.com/")->content;'

Post by [deleted] » Sat, Jan 22, 2011, 5:08 AM UTC

Post by DTrejo » Sat, Jan 22, 2011, 5:19 AM UTC

JoshCole wrote: post #6

People using clojure can get selector based scraping using Enlive instead of JQuery [1]. It also ends up doubling as a templating library. I use it for templating on my website and I use it to scrape Hacker News, though that project the scraping is for isn't ready for launch [2]. 1: https://github.com/cgrand/enlive 2: https://github.com/jColeChanged/mysite

Post by xtacy » Sat, Jan 22, 2011, 6:32 AM UTC

mmaunder wrote: post #3

perl -MLWP::UserAgent -e 'map { $_ =~ s/ ([^ ([^ ]*>(\d+ points)/print "$1 $2 $3 $4\n" if($i++ new->get("http://news.ycombinator.com/")->content;'