Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

XPath is actually pretty useful once it stops being confusing

news.rapgenius.com

Post reply on HN
Print view View original

31–40 of 77 posts

Re: XPath is actually pretty useful once it stops being confusing

#31

Post by tieTYT »

gpsarakis wrote: post #30

CSS selectors are much easier to remember than XPath. Python's BeautifulSoup allows you to select elements with selectors and is very convenient. XPath is a bit more verbose and most people already are familiar with CSS syntax.

I'd say more than a bit. When you have multiple namespaces in your xml it can become so verbose that it's hard to see the signal through the noise. But then, maybe there's a way to reduce that noise in a way that I don't understand.

Long comment short, I agree. CSS selectors are easier to understand and read.

Re: XPath is actually pretty useful once it stops being confusing

#32

Post by d0m »

TylerE wrote: post #24
d0m wrote: post #22

I prefer to use html parsers for such problems, such as beautifulsoup in python. I used xpath in the past but the ending code wasn't that much shorter then a more verbose version based on beautifulsoup. And, for someone looking at the code, the beautiful version makes so much more sense.. Xpath also feels like a big regex expression that magically works. I'm not saying it's not useful. Actually, I believe that if you…

The thing about xpath is that it runs in highly efficient native code. When iterating over a 100MB+ file it makes an _immense_ difference. It's conceptually the difference between: count = 0 for row in db.query("select id from "): count += 1 and db.query("select count(*) from ") I also don't find it at all confusing, you just have to understand the tree nature of XML.

Oh no, you've used the performance argument against me ;-) Obviously, when performance issues are on the line, you often need to trade simplicity and maintainability.

Re: XPath is actually pretty useful once it stops being confusing

#33

Post by gsnedders »

gpsarakis wrote: post #30

CSS selectors are much easier to remember than XPath. Python's BeautifulSoup allows you to select elements with selectors and is very convenient. XPath is a bit more verbose and most people already are familiar with CSS syntax.

And indeed, any CSS selector can be converted to an equivalent XPath query, at least for selectors on XML and HTML. http://pythonhosted.org/cssselect/ is a Python implementation of such a conversion. (Note that there is no XPath to CSS selector converter, as XPath can express certain things CSS selectors cannot, as CSS selectors are designed such that they can be matched using a streaming parser as soon as the first child of the element appears.)

Re: XPath is actually pretty useful once it stops being confusing

#34

Post by djur »

dionidium wrote: post #13
m_st wrote: post #10

XPath is alright but as sixbrx noted it suffers from problems with namespaces. I keep using this xslt transformation to remove the ns info. Then it works just fine: http://stackoverflow.com/a/413088/34022

This is fine, I guess, and it's clearly something people want to do based on how often it gets asked on Stackoverflow, but there's a reason XML zealots get snarky when people ask how to do this. Some questions you might want to ask yourself: - Why are the namespaces there in the first place? - Do I really not care if the element is found in a namespace other than the one expected? - Does my host environment have a wa…

The question is usually "is the author of this XML actually using namespaces in a reasonable manner?"

And the answer is usually "no".

I have seen SOAP responses with 20+ namespaces, all of them being essentially implementation details -- every different section of their internal API getting its own namespace. Inevitably, the elements are also prefixed in a way that makes them distinct, or wrapped in a distinguishing element (i.e. Contact/NameInfo/FirstName rather than FirstName xmlns="contact-name").

In situations like that, your best case scenario is that you do the grunt work of setting up aliases for all the namespaces, putting them into your XPaths, and you're done. The worst case scenario (which I've encountered) is when a version update of the API changes the URIs for half the namespaces, even though the structure of the data hasn't changed. In a case like that, you're actually penalized for doing the 'right' thing and not just stripping the damn things off.

Re: XPath is actually pretty useful once it stops being confusing

#35

Post by TylerE »

d0m wrote: post #32
TylerE wrote: post #24

Earlier quoted context omitted.

The thing about xpath is that it runs in highly efficient native code. When iterating over a 100MB+ file it makes an _immense_ difference. It's conceptually the difference between: count = 0 for row in db.query("select id from "): count += 1 and db.query("select count(*) from ") I also don't find it at all confusing, you just have to understand the tree nature of XML.

Oh no, you've used the performance argument against me ;-) Obviously, when performance issues are on the line, you often need to trade simplicity and maintainability.

I'm not sure how using a platform specific library is more maintainable than an open standard with myriad implementations.

Re: XPath is actually pretty useful once it stops being confusing

#36

Post by slig »

> But it gets more interesting if the lyrics are stored as an HTML fragment.

Is there any reason to store the HTML version with

s and
s instead of a plain text and converting it to HTML with simple rules à la markdown? (single line break =
, double line break =

)

Re: XPath is actually pretty useful once it stops being confusing

#37

Post by masklinn »

mmastrac wrote: post #9
masklinn wrote: post #5

> the / in an XPath expression plays the same role as the > in a CSS selector: The `/` in an XPath expression is probably a better match for the space in CSS selectors.

I think you are thinking of "//", which acts like the space (ie: any number of children).

No, that's just a difference in default axis. I mean that `/` is a separator between traversal expressions, much like whitespace in CSS selectors.

Re: XPath is actually pretty useful once it stops being confusing

#39

Post by jefflinwood »

Interesting, (and this could just be a made up problem to illustrate the blog post), but wouldn't it have been much easier to just store the lyrics in another format (not HTML)?

For example, you could use TEI XML (http://www.tei-c.org/index.xml), and then use stanzas and lines. Then when you go to render your lyrics, you can capitalize the first letters in your presentation code.

Re: XPath is actually pretty useful once it stops being confusing

#40

Post by callmeed »

I just started getting into xpaths pretty hardcore with my trivia generator for http://playhattrick.com ... I use it for identifying tables of data to scrape. It's not as fun as regex IMO but it is powerful.

Pro Tip: the chrome inspector lets you right-click on an element and get its xpath.

Pro Warning: sometimes the xpath generated by chrome doesn't work when scraping with Nokogiri. I'm not sure why yet, I've just learned not to rely on it.

Post reply on HN