XPath tips from the web scraping trenches

#1

Post by predius » Thu, Jul 17, 2014, 3:59 PM UTC

XPath tips from the web scraping trenches
blog.scrapinghub.com

Re: XPath tips from the web scraping trenches

#2

Post by patrickg » Thu, Jul 17, 2014, 4:20 PM UTC

There are some things in XPath that I really like. For example `x = ( a, b, c )' means: `(x == a or x == b or x == c)' which is quite handy sometimes. Also `(a,default)[1]' makes it easy to provide a default if a is null.

Re: XPath tips from the web scraping trenches

#3

Post by krapp » Thu, Jul 17, 2014, 4:22 PM UTC

I've been working on a web scraper that uses xpath for a while as part of several projects - resources like this are very much appreciated.

Re: XPath tips from the web scraping trenches

#4

Post by eliasdorneles » Thu, Jul 17, 2014, 4:37 PM UTC

patrickg wrote: post #2

There are some things in XPath that I really like. For example `x = ( a, b, c )' means: `(x == a or x == b or x == c)' which is quite handy sometimes. Also `(a,default)[1]' makes it easy to provide a default if a is null.

Thanks, that is novelty to me! I don't seem to get an example working, though. Is that XPath 2.0?

Re: XPath tips from the web scraping trenches

#5

Post by patrickg » Thu, Jul 17, 2014, 5:07 PM UTC

eliasdorneles wrote: post #4

patrickg wrote: post #2

There are some things in XPath that I really like. For example `x = ( a, b, c )' means: `(x == a or x == b or x == c)' which is quite handy sometimes. Also `(a,default)[1]' makes it easy to provide a default if a is null.

Thanks, that is novelty to me! I don't seem to get an example working, though. Is that XPath 2.0?

Yes, this is XPath 2.0. I don't bother with 1.0 if I can, because it is soooo far behind 2.0.

data:

xsl:

gives: false2

Re: XPath tips from the web scraping trenches

#6

Post by eliasdorneles » Thu, Jul 17, 2014, 5:20 PM UTC

patrickg wrote: post #5

eliasdorneles wrote: post #4

Earlier quoted context omitted.

Thanks, that is novelty to me! I don't seem to get an example working, though. Is that XPath 2.0?

Yes, this is XPath 2.0. I don't bother with 1.0 if I can, because it is soooo far behind 2.0. data: xsl: gives: false 2

That is very cool, indeed! =) I'm stuck with XPath 1.0 still, because XPath 2.0 hasn't found its way to lxml yet -- it seems to be quite a lot of work to implement it.