XPath 1.0 was released in the late 90’s. I remember using it in some server-side XML processing code (Java 1.2?) It did the job where the alternative was writing a ton of procedural code to get at a specific node, etc.
What Happened to XPath?
21–30 of 97 posts
Re: What Happened to XPath?
#22Xpath is so powerful for web scraping I just realized recently. I'd been using css selectors for my occasional scraping needs and never bothered to learn xpath until on day on a whim decided to learn at least the basics.
Man I can now write scrapers in 2 minutes that used to take me quite some time thanks to the power of xpath. Thing like ancestors, contains, the ability to chain, etc is so so powerful. I used to write so many hacks just to do the same with css before.
Re: What Happened to XPath?
#23Anyone who does scraping or automated browser work eventually comes across XPath. In some ways, XPath is like regex. It's got insane power, but comes with a relatively steep learning curve. Remember reading regex for the first time? What? But unlike regex, the number of people using it are few in comparison. I avoided XPath until I couldn't anymore. I could do a lot with CSS selectors, but eventually the DOM traversa…
I agree. I think the original author completely missed the point and conflates lack of mainstream usage with dead tech. If you never run into problems that xpath addresses, of course you’ll never use xpath. It’s not for everyday use. And certainly shouldn’t be billed as a CSS selector replacement.
Re: What Happened to XPath?
#24Anyone who does scraping or automated browser work eventually comes across XPath. In some ways, XPath is like regex. It's got insane power, but comes with a relatively steep learning curve. Remember reading regex for the first time? What? But unlike regex, the number of people using it are few in comparison. I avoided XPath until I couldn't anymore. I could do a lot with CSS selectors, but eventually the DOM traversa…
> In some ways, XPath is like regex. It's got insane power, but comes with a relatively steep learning curve. Remember reading regex for the first time? What? But unlike regex, the number of people using it are few in comparison.
IMO the learning curve of XPath is not that high though, it has a somewhat alien syntax but the only thing I remember giving me trouble is axis, because most tutorials just go on with the "shortcut" syntax so the first time you encounter axis everything goes pear-shaped.
> There are functions in XPath 2.0 that I would love to have, but Nokogiri for Rails is stuck in 1.0 world with no plan to go to 2.0. Sad, but I'll live.
Nokogiri should support function extensions[0] and most of the XPath 2.0 functions were originally extensions to 1.0[1], so even if these functions are not distributed with nokogiri you should be able to add them yourself.
Incidentally, Nokogiri seems to optionally depend on libexslt, which is the exslt implementation in C for libxml2/libxslt, so exslt should be available either as an option or by building it yourself.
[0] https://github.com/sparklemotion/nokogiri/commit/eb56525fbcc...
[1] http://exslt.org
Re: What Happened to XPath?
#25Earlier quoted context omitted.
I think you're referring to XSL. The heavy lifting is done by the transformation language (XSLT), but XPath is definitely an underlying tool.
I may be wrong, as it's been some time since I worked with them, but I think XPath is both its own standard, and a part of XSL at the same time. A lot of XSLT deals with selecting nodes from the source and it happens with XPath expressions.
W3C standards usually depend on and leverage other standards, so XPath is its own standard, which is used by XSLT (and XQuery, and possibly a few other things).
You can't use XSLT without XPath, but you can use XPath on its own.
Re: What Happened to XPath?
#26XPath post 1.0 got ridiculous, like many things do. What started with a simple, elegant language morphed into one with a http client, filesystem methods, json support, functions, loops, extensions and the ability to read environment variables. I wrote a post about it a while back[1] (I regret some of the wording used there) and maintain a tool[2] that can exploit XPath injection issues. I'd recommend sticking with 1…
Holy crap. What is this atrocity that is XPath 3.0!? What was wrong with sprinkling some XPath 1.0 queries into a Python script?
Re: What Happened to XPath?
#27It's hard not to read this as satire, because XPath is so inelegant. Not that CSS selectors are a model of elegance, but it gets the job done (most of the time) and is easy enough for rookie devs and designers to pick up.
> XPath is so inelegant. Not that CSS is a model of elegance
XPath is infinitely more elegant than CSS selectors.
It might be less pretty, especially with the sort of selectors you'd use in CSS, but elegance could hardly be less of an issue.
Re: What Happened to XPath?
#28XPath and XSLT was the first time (despite doing Haskell at university) that I started to really understand functional programming. The first time was working on a tech stack that was basically Microsoft SHAPE queries transformed into HTML. The second was multiple projects customising Google custom search engine results. It was weird realising that these very limited primitive were actually infinitely powerful if you were willing to warp your brain the right way.
That said, I scrape a fair few webpages now and have never once revisited XPath. I suppose people have mostly written off anything that feels too much like XML as enterprisey and deprecated.
Re: What Happened to XPath?
#29Xpath is so powerful for web scraping I just realized recently. I'd been using css selectors for my occasional scraping needs and never bothered to learn xpath until on day on a whim decided to learn at least the basics. Man I can now write scrapers in 2 minutes that used to take me quite some time thanks to the power of xpath. Thing like ancestors, contains, the ability to chain, etc is so so powerful. I used to wri…
Can you point out the resources you used for learning ?
I wrote a lot of scrappers and am knee-deep in css hacks.
Re: What Happened to XPath?
#30The biggest problem with the new XPath versions is that the W3C made the standards, but almost no one implemented them, so you cannot actually use them
I was doing web scraping, and needed regular expressions to get the text, so I have implemented XPath 2. And currently I am updating it to XPath 3.1: http://www.videlibri.de/xidel.html