Live data from Hacker News

Oh Yes You Can Use Regexes to Parse HTML

stackoverflow.com

11–20 of 22 posts

Re: Oh Yes You Can Use Regexes to Parse HTML

#11
As everyone has pointed out, this does not count. Note that the idea that regex can't parse html is specific and proven. What it means is that you can't write an expression that matches both the opening and matching closing tags. There's no way to handle nested tags within a single regex. It's only possible to write a regex that matches up to a finite nesting limit.

Re: Oh Yes You Can Use Regexes to Parse HTML

#12

So he is using a full blown parser, but some part of the tokenisation is done with regexes. I call BS. Also I'm pretty sure it will miss some nesting of " I used regexes to parse HTML, it works fine for quick and dirty scripts that need a small chunk of data for a limited sample of pages. Which I believe is the message he is trying to convey. But I'd rather keep the legend of the infamous SO post against parsing HTML…

I have fun story about this. Once I was trying to get data out of this one API that served XML. First I wrote a solution using regexes. Because of confusion elsewhere in the thread, I want to really clarify that I didn't parse the whole thing with one big regex. But neither were they use merely for tokenization. Somewhere in between. It had stuff like this (from memory may not actually be valid regex)

  
It worked perfectly. Then I heard that parsing with regex was a bad thing and you should use a proper parser. It worked for a short time until I got an error about invalid xml. See one of the attributes contained a heart "<3" - this is actually not allowed in xml! It has to be escaped even in attributes. I went back to the regex solution, and it kept chugging along for years on their invalid xml.

Re: Oh Yes You Can Use Regexes to Parse HTML

#13
HTML is not regular, so it can't be recognised by a "theoretical" regular expression, such as introduced in a theoretical CS class. Modern regex engines however, are more powerful and can recognise non-regular languages too.

Then there's a distinction to be made between recognising a language and parsing it.

This article goes into more detail: https://www.npopov.com/2012/06/15/The-true-power-of-regular-...

Re: Oh Yes You Can Use Regexes to Parse HTML

#14
post #11

As everyone has pointed out, this does not count. Note that the idea that regex can't parse html is specific and proven. What it means is that you can't write an expression that matches both the opening and matching closing tags. There's no way to handle nested tags within a single regex. It's only possible to write a regex that matches up to a finite nesting limit.

That is true of "theoretical" regexes, not of the ones actually used by modern languages.

Re: Oh Yes You Can Use Regexes to Parse HTML

#15
post #11

As everyone has pointed out, this does not count. Note that the idea that regex can't parse html is specific and proven. What it means is that you can't write an expression that matches both the opening and matching closing tags. There's no way to handle nested tags within a single regex. It's only possible to write a regex that matches up to a finite nesting limit.

I think this is the difference between the theoretician and the practitioner. You see your interpretation is the obvious one for the former. But as any practitioner can tell a regular expression can't even parse a regular language!

See, normally the whole point of parsing something is to get data out right. And the way a regex gets data out is through capture groups. But herein lies the issue, a capture group can only capture one piece of information!

Consider a simple regular language: a non empty sequence of comma separated positive integers. We would like to get the integers out. An attempt

  (\d+)(,(\d+))*
The first group captures the first number, the second group is just something we introduced for the purpose of writing the regex, we don't care about the value. The third (inner) group should ideally capture all the subsequent numbers separately. But it doesn't! If you try to run that regex on 1,2,3,4,5,6,7,8,9 you will find that group 1 matches 1. And group 3 matches 9. Where did all the other numbers go?!

So really, you have to give the regex some outside help, maybe an outside loop, maybe splitting on a regex rather than parsing with one. Even for this simple language!

And when you are already doing that, why the step to giving it a bit more help, perhaps a stack, is quite small.

Re: Oh Yes You Can Use Regexes to Parse HTML

#16
post #8

This "uses" regexes to parse HTML in the same way that Sunny D is "made with" 100% orange juice.

[flagged]

OP moves the goalposts, not the comments:

"Here’s my HTML lexer. It doesn’t try to do a validating parse; it just identifies the lexical elements. You might think of it more as an HTML chunker than an HTML parser. It isn’t very forgiving of broken HTML, although it makes some very small allowances in that direction."

Re: Oh Yes You Can Use Regexes to Parse HTML

#18
post #8

This "uses" regexes to parse HTML in the same way that Sunny D is "made with" 100% orange juice.

[flagged]

To have shifted goalposts, I would have had to have said one thing, and then say something different. But I only made a single comment.

Re: Oh Yes You Can Use Regexes to Parse HTML

#19
post #17

"You cannot make an alcoholic drink with water." OP: "Oh Yes You Can Use Water to Make A Hard Drink. AH! But if I freeze water and pour in whiskey, I've used water to make an alcoholic drink." -.-

What is hard seltzers?

Hard seltzer is a popular alcoholic drink that combines alcohol with flavored carbonated water. Compared to many other alcoholic drinks, hard seltzer is lower in alcohol content, calories, and sugar.

Re: Oh Yes You Can Use Regexes to Parse HTML

#20
post #19
post #17

"You cannot make an alcoholic drink with water." OP: "Oh Yes You Can Use Water to Make A Hard Drink. AH! But if I freeze water and pour in whiskey, I've used water to make an alcoholic drink." -.-

What is hard seltzers? Hard seltzer is a popular alcoholic drink that combines alcohol with flavored carbonated water. Compared to many other alcoholic drinks, hard seltzer is lower in alcohol content, calories, and sugar.

> combines alcohol with flavored carbonated water

COMBINES. That's exactly what I'm getting at. Water (alone) cannot make an alcoholic drink. Neither can regexes (alone) parse HTML. But of course, you can suspend alcohol within carbonated water and make a hard drink. In the same way you can utilize regexes to parse HTML.

Post reply on HN