Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Htmlq: like jq, but for html

github.com

Post reply on HN
Print view View original

141–150 of 172 posts

Re: Htmlq: like jq, but for html

#142

Post by hyperpallium2 »

dotancohen wrote: post #139
hyperpallium2 wrote: post #136

Earlier quoted context omitted.

xmlstarlet is really nothing like jq, as a language. But yes, I use it because it is the best commandline xml processor I'd found. That's the only similarity to jq. Is this the yq? https://kislyuk.github.io/yq/ It does contain an 'xq', as a literal wrapper for jq, piping output into it after transcoding XML to JSON using xmltodict https://github.com/martinblech/xmltodict (which explodes xml into separate JSON data st…

Thank you for the comments. I've only recently discovered both tools, and literally used them once each. Of the two `xq` was easier for my particular work case (parsing a Magento config) but I keep both tools in my virtual toolbox. If you have any other suggestions for parsing XML for exploratory purposes I'm very happy to hear them.

Thanks! Not actually a reccommendation, but I have used xsltproc (command line xslt), but it is horrible to use because xslt syntax is horrible (though xslt's concepts are pretty cool). One thing is it enables you to use XPath in all its glory.

Just installed xq. It's nice just seeing the pretty-printed json output, so thanks for the pointer. Probably better than xmlstarlet for my usage, which just queries and outputs text, not xml. hmmm, that's probably true for most commandline uses...

Re: Htmlq: like jq, but for html

#144

Post by heavyset_go »

gizdan wrote: post #21

Once upon a time I was using pup[0] for such thing as well as later I changed to cascadia[1] which seemed much more advanced. Comparing the two repos, it seems pup is dead, but cascadia may not be. These tools, including htmlq, seem to sell themselves as "jq for html", which is far from the truth. Jq is closer to the awk where you can do just about everything with json. Cascadia, htmlq, and pup seem closer to grep fo…

I've used pup for a few projects, but was unaware of cascadia. Thanks for pointing it out.

Re: Htmlq: like jq, but for html

#145

Post by rendall »

mgdm wrote: post #135
theIV wrote: post #126

Earlier quoted context omitted.

My hunch is that this is a typo and it should read "extract bits OF content."

Exactly this! I’ll fix it after work.

Maybe have the line about "jq" be 2nd. Have the first line be a brief description of what it actually does.

Re: Htmlq: like jq, but for html

#146

Post by benibela »

alpha_squared wrote: post #105
berkes wrote: post #45

Earlier quoted context omitted.

I'd like to add my support here too, but with a note. When scraping and parsing (or writing integration test DSL), I always start out with CSS selectors. But always hit cases where they lack or require hoop-jumping and then fall back on Xpath. I then have a codebase with both CSS-Sel and Xpath, which is arguably worse then having only one method. I suspect here, one uses this tool untill CSS selector limitations are…

Do you mind giving an example? I'm having trouble following where CSS is limited for selection.

XPath does general data processing not just selection

E.g. when you have a list of numbers on the website, XPath can calculate the sum or the maximum of the numbers

Or you have a list of names "Last name, First name", then you can remove the last name and sort the first names alphabetically. Or count how often each name occurs and return the most popular name.

Then it goes back to selection, e.g. select all numbers that are smaller than the average. Or calculate the most popular name, then select all elements containing that name

Re: Htmlq: like jq, but for html

#147

Post by rendall »

philipswood wrote: post #131
digitalsushi wrote: post #93

Earlier quoted context omitted.

"htmlq is like jq but for html" is a very specific 'dog whistle' for people who use jq. I agree that people who don't know what jq is will get no value and pay no attention. But for people who use jq, the claim is, like a dog whistle, clear, concise, and means exactly what it says. In two seconds, everyone using both jq and html will instantly know what is available and log it away. So for general purposes, it's a te…

I can't speak for people who don't know jq, but knowing jq, this is a great tagline: it gives me an immediate understanding of what it does, how I could expect to use it and what value and ease of use I can expect. I'll be trying it out next time I'm on a PC.

> I can't speak for people who don't know jq,

I can, and it's not illuminating at all.

Re: Htmlq: like jq, but for html

#149

Post by androceium »

vlunkr wrote: post #122
alpha_squared wrote: post #105

Earlier quoted context omitted.

Do you mind giving an example? I'm having trouble following where CSS is limited for selection.

Well, the big one is selecting a parent from the child.

You could do this with the :has() CSS psuedo-class[0], though inverted (select a parent that _has_ the child matching a selector).

Looks like that psuedo-class has not been implemented in the kuchiki library that htmlq uses though.

[0]: https://developer.mozilla.org/en-US/docs/Web/CSS/:has

Re: Htmlq: like jq, but for html

#150

Post by unspecified »

alpha_squared wrote: post #105
berkes wrote: post #45

Earlier quoted context omitted.

I'd like to add my support here too, but with a note. When scraping and parsing (or writing integration test DSL), I always start out with CSS selectors. But always hit cases where they lack or require hoop-jumping and then fall back on Xpath. I then have a codebase with both CSS-Sel and Xpath, which is arguably worse then having only one method. I suspect here, one uses this tool untill CSS selector limitations are…

Do you mind giving an example? I'm having trouble following where CSS is limited for selection.

Searching text content is my main remaining use of XPath.
Post reply on HN