XPath is actually pretty useful once it stops being confusing • Hacker News Board

Re: XPath is actually pretty useful once it stops being confusing

#41

Post by masklinn » Mon, Nov 04, 2013, 8:22 PM UTC

yeahbutbut wrote: post #28

masklinn wrote: post #4

Earlier quoted context omitted.

> In every single task I do that involves munching on XML with xpath And more generally that's true of every single task involving muching on namespaced XML. Namespaces are a good idea implemented absolutely terribly. XPath is a good idea well-implemented (no, XPath 2 does not exist, there is only one XPath). One of the few I've found in XML-land. I still hate that we have to use CSS selectors rather than XPath (alth…

> Also, that might have finally gotten us a non-eye-stabbing standard function for "match any item of a space-separated list in an attribute" CSS handles this rather nicely: [class~=foo] https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_s...

Whereas xpath... does not. Which is a severe understatement considering the equivalent to the CSS selector you wrote up (or to `.foo`) in xpath 1 is something along the lines of:

    [contains(concat(' ', normalize-space(@class), ' '), ' foo ')]

the normalize-space can be dropped IIF you're certain all spaces are normalized, the spaces around the needle not.

xpath 2 does quite a bit better through `tokenize`:

    //*[tokenize(@class, '\s+')='foo']

but still not great. And god forbid you need to match multiple classes in the same selector.

[0] or xpath 1 + exslt if your xpath implementation provides it. exslt actually does slightly better as the pattern is optional and defaults to whitespace characters

Re: XPath is actually pretty useful once it stops being confusing

#42

Post by narrator » Mon, Nov 04, 2013, 8:24 PM UTC

One problem with XPath is that it can be a lot slower than native or JIT'd code depending on the implementation. Interestingly enough you can do xpath like things in Scala with native code using pattern matching:

http://ofps.oreilly.com/titles/9780596155957/HerdingXMLInSca...

Re: XPath is actually pretty useful once it stops being confusing

#43

Post by aliakbarkhan » Mon, Nov 04, 2013, 8:44 PM UTC

slig wrote: post #36

> But it gets more interesting if the lyrics are stored as an HTML fragment. Is there any reason to store the HTML version with s and s instead of a plain text and converting it to HTML with simple rules à la markdown? (single line break = , double line break = )

Well, at a minimum, it saves the processing time required to format the text, which lessens the server cost of each page hit. It's a small optimization, but when the vast majority of the users are just coming to the site to read text, I'd imagine it would save a lot of CPU time.

Re: XPath is actually pretty useful once it stops being confusing

#44

Post by monksy » Mon, Nov 04, 2013, 8:48 PM UTC

kjhughes wrote: post #18

XPaths are extremely useful. I actually enjoy writing them, much like I enjoy writing regular expressions. In fact, I consider both to return manifold the modest investment they require to learn well. XPath : XML :: regex : text

I feel the same way. I frequetly use them, and find them a lot easier to use than Regexes. My site even uses regexes a lot. [ theexceptioncatcher.com ]

Re: XPath is actually pretty useful once it stops being confusing

#45

Post by toyg » Mon, Nov 04, 2013, 8:54 PM UTC

callmeed wrote: post #40

I just started getting into xpaths pretty hardcore with my trivia generator for http://playhattrick.com ... I use it for identifying tables of data to scrape. It's not as fun as regex IMO but it is powerful. Pro Tip: the chrome inspector lets you right-click on an element and get its xpath. Pro Warning: sometimes the xpath generated by chrome doesn't work when scraping with Nokogiri. I'm not sure why yet, I've just l…

A problem with XPath is that many tools don't actually support the latest spec, or even a spec, and just give you a bastardised syntax that looks like XPath and walks like XPath but will never swim like XPath. This is frustrating because you're never sure which features you can actually rely on.

Re: XPath is actually pretty useful once it stops being confusing

#46

Post by garethadams » Mon, Nov 04, 2013, 8:59 PM UTC

callmeed wrote: post #40

I just started getting into xpaths pretty hardcore with my trivia generator for http://playhattrick.com ... I use it for identifying tables of data to scrape. It's not as fun as regex IMO but it is powerful. Pro Tip: the chrome inspector lets you right-click on an element and get its xpath. Pro Warning: sometimes the xpath generated by chrome doesn't work when scraping with Nokogiri. I'm not sure why yet, I've just l…

There's not "an" XPath for an element, as XPath describes the route you take from the root of a document to the element in question. The correct route for your situation depends on your use case.

Describing an element as "the first child of the fifth child of the second child of the first child of the eighth child of the second child of HTML" is as much the right path to an element as if you described the way to your house as "Walk past the park then walk past the bus stop then walk past the hardware store then walk past the butchers then turn left then walk past the pizza shop then walk past the library"

Re: XPath is actually pretty useful once it stops being confusing

#47

Post by callmeed » Mon, Nov 04, 2013, 9:17 PM UTC

garethadams wrote: post #46

callmeed wrote: post #40

I just started getting into xpaths pretty hardcore with my trivia generator for http://playhattrick.com ... I use it for identifying tables of data to scrape. It's not as fun as regex IMO but it is powerful. Pro Tip: the chrome inspector lets you right-click on an element and get its xpath. Pro Warning: sometimes the xpath generated by chrome doesn't work when scraping with Nokogiri. I'm not sure why yet, I've just l…

There's not "an" XPath for an element, as XPath describes the route you take from the root of a document to the element in question. The correct route for your situation depends on your use case. Describing an element as "the first child of the fifth child of the second child of the first child of the eighth child of the second child of HTML" is as much the right path to an element as if you described the way to your…

I'm aware what XPath is/does.

And I'm not disagreeing with you–I'm only saying Chrome has this feature. I don't know what route they choose for you but I know they don't always work in tools that parse (HT/X)ML.

Here's a screenshot in case you don't believe me: http://imgur.com/9FZSMSt

The XPath Chrome returns for this page is: //*[@id="details"]/article/table[1]

Re: XPath is actually pretty useful once it stops being confusing

#48

Post by maaaats » Mon, Nov 04, 2013, 9:28 PM UTC

slig wrote: post #36

> But it gets more interesting if the lyrics are stored as an HTML fragment. Is there any reason to store the HTML version with s and s instead of a plain text and converting it to HTML with simple rules à la markdown? (single line break = , double line break = )

At least for me it was interesting when considering the aspect of finding/modifying stuff in data you cannot control yourself. For instance crawling, indexing or retrieving data in an unspecified format.

Re: XPath is actually pretty useful once it stops being confusing

#49

Post by vdaniuk » Mon, Nov 04, 2013, 9:33 PM UTC

aliakbarkhan wrote: post #43

slig wrote: post #36

> But it gets more interesting if the lyrics are stored as an HTML fragment. Is there any reason to store the HTML version with s and s instead of a plain text and converting it to HTML with simple rules à la markdown? (single line break = , double line break = )

Well, at a minimum, it saves the processing time required to format the text, which lessens the server cost of each page hit. It's a small optimization, but when the vast majority of the users are just coming to the site to read text, I'd imagine it would save a lot of CPU time.

Not really, the underlying text is just an HTML page that rarely changes and the requests rarely hit the database because caching.

Re: XPath is actually pretty useful once it stops being confusing

#50

Post by slig » Mon, Nov 04, 2013, 9:41 PM UTC

maaaats wrote: post #48

slig wrote: post #36

> But it gets more interesting if the lyrics are stored as an HTML fragment. Is there any reason to store the HTML version with s and s instead of a plain text and converting it to HTML with simple rules à la markdown? (single line break = , double line break = )

At least for me it was interesting when considering the aspect of finding/modifying stuff in data you cannot control yourself. For instance crawling, indexing or retrieving data in an unspecified format.

Yes, I enjoyed the article as well. My question was why they stored the HTML in the first place.