Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Htmlq: like jq, but for html

github.com

Post reply on HN
Print view View original

21–30 of 172 posts

Re: Htmlq: like jq, but for html

#21

Post by gizdan »

Once upon a time I was using pup[0] for such thing as well as later I changed to cascadia[1] which seemed much more advanced.

Comparing the two repos, it seems pup is dead, but cascadia may not be.

These tools, including htmlq, seem to sell themselves as "jq for html", which is far from the truth. Jq is closer to the awk where you can do just about everything with json. Cascadia, htmlq, and pup seem closer to grep for html. They can essentially only select data from a html source.

[0] https://github.com/EricChiang/pup [1] https://github.com/suntong/cascadia

Re: Htmlq: like jq, but for html

#22

Post by croon »

rsync wrote: post #18
dmos62 wrote: post #10

Earlier quoted context omitted.

> grok A good opportunity to introduce `gron` to those unfamiliar! ▶ gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | fgrep "commit.author" json[0].commit.author = {}; json[0].commit.author.date = "2016-07-02T10:51:21Z"; json[0].commit.author.email = "mail@tomnomnom.com"; json[0].commit.author.name = "Tom Hudson"; https://github.com/tomnomnom/gron

"A good opportunity to introduce `gron` to those unfamiliar!" Thank you - appreciated. I haven't done much work with json but have had reasons recently to do so - and I immediately saw how difficult it was to pipeline to grep ... But what I still don't understand is that some json outputs I see have multiple values with the exact same name (!) and that still seems "un-grep-able" to me ... What am I missing ?

The json output likely contains multiple objects. Can you request more specifically the object(s) you need and grep on that?

Re: Htmlq: like jq, but for html

#23

Post by mjburgess »

Deukhoofd wrote: post #19
zatkin wrote: post #15

Why not incorporate this into jq itself, like perhaps adding some command line arguments to switch to HTML mode?

What would the benefits of fitting a HTML parser into a JSON parser tool be?

Well once there's an HTML parser, then a pdf viewer, and then everything needed for PDFs (ie., programming, emailer, video support, etc.) we'll finally have that ideal operating system we've been waiting for.

Re: Htmlq: like jq, but for html

#24

Post by croon »

gizdan wrote: post #21

Once upon a time I was using pup[0] for such thing as well as later I changed to cascadia[1] which seemed much more advanced. Comparing the two repos, it seems pup is dead, but cascadia may not be. These tools, including htmlq, seem to sell themselves as "jq for html", which is far from the truth. Jq is closer to the awk where you can do just about everything with json. Cascadia, htmlq, and pup seem closer to grep fo…

Well, jq is grep as well as sed and awk, but yeah, htmlq seems to be just grep, for sake of comparison.

But I don't think html has any need for a sed/awk tool, or at least not as much. Json output could very well be piped forward to the next CLI tool after you've changed it slightly with jq. I don't see this scenario as likely with html.

Re: Htmlq: like jq, but for html

#26

Post by andybak »

gostsamo wrote: post #14
purplecats wrote: post #5

brilliant. does this spin up a heavy DOM implementation in the background or do something lighter such as regexp?

You can't parse html with regular expressions :) https://stackoverflow.com/questions/1732348/regex-match-open...

And the obligitory caveat from the comments:

> While arbitrary HTML with only a regex is impossible, it's sometimes appropriate to use them for parsing a limited, known set of HTML.

Re: Htmlq: like jq, but for html

#28

Post by carnitine »

mdzn wrote: post #7
purplecats wrote: post #5

brilliant. does this spin up a heavy DOM implementation in the background or do something lighter such as regexp?

You can’t parse HTML with regexps. It’s not a regular language.

What language implements regexps that actually correspond to regular languages though?

Re: Htmlq: like jq, but for html

#29

Post by hnbad »

andybak wrote: post #26
gostsamo wrote: post #14

Earlier quoted context omitted.

You can't parse html with regular expressions :) https://stackoverflow.com/questions/1732348/regex-match-open...

And the obligitory caveat from the comments: > While arbitrary HTML with only a regex is impossible, it's sometimes appropriate to use them for parsing a limited, known set of HTML.

The emphasis here is on "known". The tool is general purpose (i.e. handling unknown HTML) so using regexes would be ill-advised.

Re: Htmlq: like jq, but for html

#30

Post by chrismorgan »

Deukhoofd wrote: post #8
purplecats wrote: post #5

brilliant. does this spin up a heavy DOM implementation in the background or do something lighter such as regexp?

Looks like it uses servos html5ever (through kuchiki), so no DOM representation.

Kuchiki materialises what they call a “DOM-like tree”. I’d consider it a DOM tree, myself, despite the differences in precise API.

But it’s not using a full browser to back it, which I suspect is what’s really being asked.

Post reply on HN