Live data from Hacker News

Parsing HTML with Regex

stackoverflow.com

11–20 of 34 posts

Re: Parsing HTML with Regex

#11
post #10

There are plenty of good, real html parsers, so there's no need to try regex. Xpath is nice for scraping HTML, though it always turns out the stuff you need is in the middle of a bunch of other text.

I would imagine that valid HTML could be parsed as XML, e.g. with the Python ElementTree XML API

Re: Parsing HTML with Regex

#12

I reckon that technically regex is a tool that can be used to parse HTML. It's just that you could only use it in a very trivial way that would be better suited to other tools.

No. Technically, you cannot parse HTML with regular expressions. You can find certain strings in HTML which is a different thing.

Re: Parsing HTML with Regex

#13
post #10

There are plenty of good, real html parsers, so there's no need to try regex. Xpath is nice for scraping HTML, though it always turns out the stuff you need is in the middle of a bunch of other text.

I would imagine that valid HTML could be parsed as XML, e.g. with the Python ElementTree XML API

That would be true for XHTML but not for HTML as tags like paragraph do not always require a end tag, which would break XML parsing.

Re: Parsing HTML with Regex

#14
post #2

I had one class where we had to build a multi threaded search engine in java and parsing HTML with regex was a requirement. Regex was the downfall of about 50% of the class and the majority of the students who did well still had slight issues with their HTML parsing. Moral of the story is that regex is a poor solution for HTML. Not to mention, hours debugging regex is one of the least meaningful or rewarding experien…

The obvious workaround for that broken requirement is to use regular expressions to handle the tokenization, and then write a simple recursive descent parser on top of that. Even if this is playing fast and loose with the requirements, it would work out fine, and you would almost certainly get a good grade if you explain why you did it.

Re: Parsing HTML with Regex

#15

While I agree with the answer. What most people take from it is don't use Regex to scrape data from HTML. Which isn't exactly the point of it. Parsing HTML and scraping are two different things. If you know the exact HTML you are working with, using regex to extract the data is in my opinion a superior way of doing it. Less lines and generally less complexity. (Such as taking the name and id of an amazon product from…

With today's widely available DOM manipulation tools (jsdom, phantom, zombie) and proper HTML parsers in javascript there is absolutely no reason to use RegExps.

Re: Parsing HTML with Regex

#16
post #2

I had one class where we had to build a multi threaded search engine in java and parsing HTML with regex was a requirement. Regex was the downfall of about 50% of the class and the majority of the students who did well still had slight issues with their HTML parsing. Moral of the story is that regex is a poor solution for HTML. Not to mention, hours debugging regex is one of the least meaningful or rewarding experien…

Did the other 50% succeed by telling the prof they got that he was joking?

Re: Parsing HTML with Regex

#17

I reckon that technically regex is a tool that can be used to parse HTML. It's just that you could only use it in a very trivial way that would be better suited to other tools.

Ruby regular expressions have a \g operator which lets you call a sub expression - so technically they could be used to parse HTML if you're a masochist.

Propriety requires that someone point out that any "regular expression" permitting recursion is not, in fact, a regular expression in the formal sense. Luckily for anybody wanting to do it anyway, parsing expression grammars can handle that sort of thing with theoretical aplomb:

http://en.wikipedia.org/wiki/Parsing_expression_grammar

Re: Parsing HTML with Regex

#18

While I agree with the answer. What most people take from it is don't use Regex to scrape data from HTML. Which isn't exactly the point of it. Parsing HTML and scraping are two different things. If you know the exact HTML you are working with, using regex to extract the data is in my opinion a superior way of doing it. Less lines and generally less complexity. (Such as taking the name and id of an amazon product from…

With today's widely available DOM manipulation tools (jsdom, phantom, zombie) and proper HTML parsers in javascript there is absolutely no reason to use RegExps.

> (jsdom, phantom, zombie) and proper HTML parsers in javascript

Well, for starters, if you're not using Javascript. All four tools you mentioned are Javascript-based.

Re: Parsing HTML with Regex

#19
post #2

I had one class where we had to build a multi threaded search engine in java and parsing HTML with regex was a requirement. Regex was the downfall of about 50% of the class and the majority of the students who did well still had slight issues with their HTML parsing. Moral of the story is that regex is a poor solution for HTML. Not to mention, hours debugging regex is one of the least meaningful or rewarding experien…

> the majority of the students who did well still had slight issues with their HTML parsing.

I'd like to hire the minority students, since it seems that they quite literally accomplished the impossible!

Re: Parsing HTML with Regex

#20
The answer is pretty entertaining, but in context it's pedantic to the extreme. The poster's question was about matching opening tags that don't contain a closing slash, which is a tiny (regular) subset of HTML. You don't need pushdown automata to recognize these.

English, as any other natural language, is (at least mostly) a context free language too, but you wouldn't go around telling people that you shouldn't ever use regexen to match certain constructions in English text, right?

Post reply on HN