Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scrape: A simple, higher level interface for Go web scraping

github.com

Post reply on HN
Print view View original

1–10 of 16 posts

Re: Scrape: A simple, higher level interface for Go web scraping

#6

Post by ericchiang »

chrissnell wrote: post #5

This is very cool. I'm not much of a front-end guy so I'm struggling with the examples. Would you mind posting up a simple example that will scrape--say--the first TD tag of every row of a table? Thanks.

  rows := scrape.FindAll(table, scrape.ByTag(atom.Tr))
  cols := []*html.Node{}
  for _, row := range rows {
      // Find returns the first result
      col, ok := scrape.Find(row, scrape.ByTag(atom.Td))
      if ok {
          cols = append(cols, col)
      }
  }

Re: Scrape: A simple, higher level interface for Go web scraping

#7

Post by rdudekul »

To me goquery seems more intuitive than scrape, may be because I am more familiar with jquery selectors syntax.

Any reason why yhat guys (ericchiang) created Scrape (and not use say goquery)?

Can you make the matcher function in main.go go away with a simpler (more intuitive) interface/api/dsl?

Re: Scrape: A simple, higher level interface for Go web scraping

#9

Post by peteretep »

Go has some weird syntactic sugar including where a method invocation is rewritten by the compiler to pass in a value or a pointer depending on what the callee wants(!?!). And yet Go code is still littered with:

    if err != nil {
... rather than some simple, compile-time validated sugar to pass the error value up the call chain. Yes, I've read the justification documents. No, they still don't make a very convincing argument.
Post reply on HN