XPath is actually pretty useful once it stops being confusing • Hacker News Board

Re: XPath is actually pretty useful once it stops being confusing

#21

Post by lobster_johnson » Mon, Nov 04, 2013, 6:11 PM UTC

sixbrx wrote: post #2

I also like XPath for some purposes, but I think it really suffers from (IIRC) having been designed before the xml namespaces, which it only integrates very awkwardly IMO and which ruins the simplicity of XPath. Or maybe XML namespaces spoil everything they affect to some degree :)

I wonder why libraries make it harder to combine XPath and namespaces. For example, in Nokogiri (libxml2 bindings for Ruby, and probably the de facto XML library in Ruby), you have to do:

    doc = Nokogiri::XML(stream)
    doc.xpath("//x:foo", {x: "http://somenamespace/"}).each ...

I always end up declaring the namespaces in a constant and passing it in, because there is no way to specify your mappings globally. It should have been something like:

    doc = Nokogiri::XML(stream)
    doc.namespaces = {x: "http://somenamespace/"}
    doc.xpath("//x:foo").each ...

or at least something functional like:

    doc = Nokogiri::XML(stream)
    doc.using_namespaces(x: "http://somenamespace/").
      xpath("//x:foo").each ...

I have seen this way of integrating namespaces and XPath in several libraries, it's not just Nokogiri.

Re: XPath is actually pretty useful once it stops being confusing

#22

Post by d0m » Mon, Nov 04, 2013, 6:16 PM UTC

I prefer to use html parsers for such problems, such as beautifulsoup in python. I used xpath in the past but the ending code wasn't that much shorter then a more verbose version based on beautifulsoup. And, for someone looking at the code, the beautiful version makes so much more sense.. Xpath also feels like a big regex expression that magically works.

I'm not saying it's not useful. Actually, I believe that if you only have one use-case, then using xpath might be overkill because of all the added-complexity of maintaining a new library/technology/ideology. But if it's the sort of domain that xpath would be useful more than once, then sure use it.

Re: XPath is actually pretty useful once it stops being confusing

#23

Post by yogo » Mon, Nov 04, 2013, 6:18 PM UTC

fein wrote: post #3

sixbrx wrote: post #2

I also like XPath for some purposes, but I think it really suffers from (IIRC) having been designed before the xml namespaces, which it only integrates very awkwardly IMO and which ruins the simplicity of XPath. Or maybe XML namespaces spoil everything they affect to some degree :)

It's the latter (XML namespaces spoil everything). In every single task I do that involves munching on XML with xpath, the largest timesink is figuring out how the ns needs to be set up. (Looking you you, PHP simpleXML).

Yep very true. Depending on what you are working on alternative approach is to strip out the namespaces before querying anything. Obviously, this can get you in trouble, and while namespaces are there for a reason, they can be a PITA when working with XML.

Re: XPath is actually pretty useful once it stops being confusing

#24

Post by TylerE » Mon, Nov 04, 2013, 6:19 PM UTC

d0m wrote: post #22

I prefer to use html parsers for such problems, such as beautifulsoup in python. I used xpath in the past but the ending code wasn't that much shorter then a more verbose version based on beautifulsoup. And, for someone looking at the code, the beautiful version makes so much more sense.. Xpath also feels like a big regex expression that magically works. I'm not saying it's not useful. Actually, I believe that if you…

The thing about xpath is that it runs in highly efficient native code. When iterating over a 100MB+ file it makes an _immense_ difference.

It's conceptually the difference between:

    count = 0 
    for row in db.query("select id from "):
      count += 1

and

    db.query("select count(*) from ")

I also don't find it at all confusing, you just have to understand the tree nature of XML.

Re: XPath is actually pretty useful once it stops being confusing

#25

Post by radicalbyte » Mon, Nov 04, 2013, 6:21 PM UTC

If you have great tools which keep the feedback loop short, then XPath, like Regex, SQL and CSS is extremely powerful and productive.

Just make sure that you document your test cases (i.e. what you should match) or your colleges will hate you.

Re: XPath is actually pretty useful once it stops being confusing

#26

Post by thangalin » Mon, Nov 04, 2013, 6:26 PM UTC

d0m wrote: post #22

I prefer to use html parsers for such problems, such as beautifulsoup in python. I used xpath in the past but the ending code wasn't that much shorter then a more verbose version based on beautifulsoup. And, for someone looking at the code, the beautiful version makes so much more sense.. Xpath also feels like a big regex expression that magically works. I'm not saying it's not useful. Actually, I believe that if you…

http://www.w3.org/Tools/HTML-XML-utils/man1/hxselect.html

"hxselect - extract elements that match a (CSS) selector"

CSS is often a simpler way to extract data.

http://www.w3.org/Tools/HTML-XML-utils/

Re: XPath is actually pretty useful once it stops being confusing

#27

Post by moron4hire » Mon, Nov 04, 2013, 6:28 PM UTC

Years ago, I wrote a tool for wrapping .NET XmlDocuments and making them far easier to work with via XPath: https://github.com/capnmidnight/xml-stuff

On its own, .NET's XML libraries are really only good for consuming XML documents, but even that is a rather painful experience, especially as it forces a namespace on all documents, complicating the XPath expressions necessary to query it. Actually authoring documents is a nightmare. My XmlEdit project makes it almost as simple as key-value-pair config files.

Re: XPath is actually pretty useful once it stops being confusing

#28

Post by yeahbutbut » Mon, Nov 04, 2013, 6:32 PM UTC

masklinn wrote: post #4

fein wrote: post #3

Earlier quoted context omitted.

It's the latter (XML namespaces spoil everything). In every single task I do that involves munching on XML with xpath, the largest timesink is figuring out how the ns needs to be set up. (Looking you you, PHP simpleXML).

> In every single task I do that involves munching on XML with xpath And more generally that's true of every single task involving muching on namespaced XML. Namespaces are a good idea implemented absolutely terribly. XPath is a good idea well-implemented (no, XPath 2 does not exist, there is only one XPath). One of the few I've found in XML-land. I still hate that we have to use CSS selectors rather than XPath (alth…

> Also, that might have finally gotten us a non-eye-stabbing standard function for "match any item of a space-separated list in an attribute"

CSS handles this rather nicely:

    [class~=foo]

https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_s...

Re: XPath is actually pretty useful once it stops being confusing

#29

Post by pyre » Mon, Nov 04, 2013, 6:42 PM UTC

d0m wrote: post #22

I prefer to use html parsers for such problems, such as beautifulsoup in python. I used xpath in the past but the ending code wasn't that much shorter then a more verbose version based on beautifulsoup. And, for someone looking at the code, the beautiful version makes so much more sense.. Xpath also feels like a big regex expression that magically works. I'm not saying it's not useful. Actually, I believe that if you…

Or you could use lxml in Python, which allows you to mix XPath and CSS Selectors. E.g.:

  from lxml import html

  doc = html.fromstring('')

  doc.xpath('//p[@class = "text"]') == doc.cssselect('p.text')

  # or

  doc = html.fromstring('1
2
3')

  doc.xpath('//p[2]')[0] == doc.cssselect('p')[1]

  # Note: My only annoyance is that .xpath() always returns a list, even
  #       when you know that it will return only a single item.

Re: XPath is actually pretty useful once it stops being confusing

#30

Post by gpsarakis » Mon, Nov 04, 2013, 7:02 PM UTC

CSS selectors are much easier to remember than XPath. Python's BeautifulSoup allows you to select elements with selectors and is very convenient. XPath is a bit more verbose and most people already are familiar with CSS syntax.