Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Happened to XPath?

webreflection.medium.com

Post reply on HN
Print view View original

81–90 of 97 posts

Re: What Happened to XPath?

#81

Post by lenkite »

ping_pong wrote: post #50

XPath and XML in general is a great example of "Death by Committee". They tried too hard to be too smart and try to solve everything, and overcomplicated it to death. This is why people largely abandoned it. This is what is happening to C++ and they are steering themselves by committee into a dead end.

Yeah the committee's decision to avoid ABI breakage is a serious deathblow against the language. Especially when a formal ABI was never defined in the first place. So, C++ is stuck with poor implementations for std::regex and std::unordered_map for ever. Where even interpreted languages can beat it.

Re: What Happened to XPath?

#82

Post by benibela »

masklinn wrote: post #31
benibela wrote: post #30

The biggest problem with the new XPath versions is that the W3C made the standards, but almost no one implemented them, so you cannot actually use them I was doing web scraping, and needed regular expressions to get the text, so I have implemented XPath 2. And currently I am updating it to XPath 3.1: http://www.videlibri.de/xidel.html

> I was doing web scraping, and needed regular expressions to get the text, so I have implemented XPath 2. Most XPath implementations have no issue with adding extension functions (in fact many support exslt[0] out of the box), you really do not need to use (let alone implement) XPath 2.0 to use regex functions. [0] http://exslt.org/regexp/index.html

I did not plan to implement it all, only the parts I needed for the webpages in my city. At first I did not even have backward axes. But people care much more about XPath than they care about my city

I also was doing too much competitive programming back then, where you have to discover and implement a highly complex algorithm in a few hours

If such a complex implementation takes a few hours, I could not imagine implementing anything else taking much longer (especially when the spec already says what needs to be implemented and it does not need to be discovered). A few days at most...

But now I am still working on it 14 years later

Re: What Happened to XPath?

#83

Post by benibela »

masklinn wrote: post #19
orf wrote: post #2

XPath post 1.0 got ridiculous, like many things do. What started with a simple, elegant language morphed into one with a http client, filesystem methods, json support, functions, loops, extensions and the ability to read environment variables. I wrote a post about it a while back[1] (I regret some of the wording used there) and maintain a tool[2] that can exploit XPath injection issues. I'd recommend sticking with 1…

I largely agree. XPath 2.0 started the downwards trajectory and XPath 3 made it worse. The things XPath 2.0 and later do improve on XPath 1.0 is the "standard library", most of exslt got standardised in 2.0, and new useful functions got added in later revisions (e.g. contains-token from 3.1 is XPath finally adding the ~= operator from CSS). Here's the deal though: it should be possible to add most functions without u…

There are other useful things besides functions

Sequences for example. In XPath 1 the query returns a set, so the output is always in document order. When the document reorders things, the query output changes, and you can never get the original output. In a sequence, the query can output anything in any order

Re: What Happened to XPath?

#85

Post by ssdspoimdsjvv »

orf wrote: post #2

XPath post 1.0 got ridiculous, like many things do. What started with a simple, elegant language morphed into one with a http client, filesystem methods, json support, functions, loops, extensions and the ability to read environment variables. I wrote a post about it a while back[1] (I regret some of the wording used there) and maintain a tool[2] that can exploit XPath injection issues. I'd recommend sticking with 1…

XPath 3 was conceived with support for XSLT and XQuery in mind - where reading environment variables and text files are most definitely very useful features. This is indeed not something you want in a browser, but that ship had already sailed by then.

Re: What Happened to XPath?

#86

Post by barumi »

marcosdumay wrote: post #69
tingol wrote: post #47

Earlier quoted context omitted.

That's a community based plugin, Vim is still focused on text editing and not much else.

Most of the functionality on the editors like vim and emacs comes from community based plugins. People would mostly not use them if there were no such expansions.

This baseless assertion is simply wrong. Plugins are nice to have, but the bulk of their use is to customize default installs.

Re: What Happened to XPath?

#87

Post by charlesdaniels »

Crazyontap wrote: post #22

Xpath is so powerful for web scraping I just realized recently. I'd been using css selectors for my occasional scraping needs and never bothered to learn xpath until on day on a whim decided to learn at least the basics. Man I can now write scrapers in 2 minutes that used to take me quite some time thanks to the power of xpath. Thing like ancestors, contains, the ability to chain, etc is so so powerful. I used to wri…

Indeed, I wrote a tool[0] to make it easy to grab a page and run xpath queries on it. It’s really surprising how much mileage I’ve gotten out of it. Probably 95% of my web scraping needs can be solved withal xpath query or two. And if you realize you need selenium later, xpath is well supported there, so porting your existing query is usually quite straightforward.

0 - https://git.sr.ht/~charles/charles-util/tree/dev/bin/query-w...

Re: What Happened to XPath?

#88

Post by Mikhail_Edoshin »

oefrha wrote: post #43
Mikhail_Edoshin wrote: post #38

Earlier quoted context omitted.

XPath always was extensible, at least at the implementation level. E.g. in 'lxml' it's trivial to add XPath functions with Python. Homegrown, of course, but still possible. In addition to extension elements this is about the only way to hook XSLT into the rest of the system. How else one is supposed to read environment variables from XSLT? The only other way is to pass everything via command line as parameters. It's…

> How else one is supposed to read environment variables from XSLT? Setting aside whether it’s even a good idea to allow XSLT to do that, XPath is only a subset of XSLT, so you’re just changing the subject. The “path” in XPath should be a hint at what it’s supposed to be: a query language to select nodes by path in XML documents. As opposed to an alternative of Awk, or Perl.

I'd say XPath a way to get a nodeset or another XPath type out of something. E.g. the current date is not selected from a document. There always will be a need to get yet another thing as a nodeset, e.g. list a directory. Or, for boolean expressions, there will always be a need to test yet another thing, such as an environment variable.

These things, of course, should come as extension functions rather than special syntax, but then there will be a need to provide a small standard library of such functions :)

So yes, I believe it's useful if we're going to use XPath in a trusted environment, e.g. as a typical command-line tool. You won't deny Bash or Python this and other powerful abilities, will you? But of course it would be very unwise to run an untrusted Bash script.

Re: What Happened to XPath?

#89

Post by oefrha »

the-dude wrote: post #20
layoutIfNeeded wrote: post #10

Earlier quoted context omitted.

I've read your article... Holy shit. They took a simple, sed-like tool and turned it into an abomination.

It ain't done before it can receive e-mail.

It can receive email. See my follow-up here with a working implementation:

https://github.com/clopen/xpath-receive-email

https://news.ycombinator.com/item?id=24960548

Re: What Happened to XPath?

#90

Post by 0df8dkdf »

oefrha wrote: post #89
the-dude wrote: post #20

Earlier quoted context omitted.

It ain't done before it can receive e-mail.

It can receive email. See my follow-up here with a working implementation: https://github.com/clopen/xpath-receive-email https://news.ycombinator.com/item?id=24960548

Hahah, that just really got carried away. Ok, any idea if there is replacement? I found version 1 useful for scrapping web sites. Or should I just stick with 1.0 ?
Post reply on HN