Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Solving the regex of madness, and snarky answers on StackOverflow (2019)

cargocultcode.com

Post reply on HN
Print view View original

121–130 of 136 posts

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#121

Post by goto11 »

IshKebab wrote: post #119
goto11 wrote: post #115

Earlier quoted context omitted.

Regular expressions can be as robust as you need them to be, just like any other kind of code. They are a DSL to create lexers, and they are exactly as robust (or hacky) as if you wrote the same lexer by hand.

C code can be as robust as you need it to be. So why bother with formal verification, safe C coding standards, Rust, etc? The answer is that it can be robust, but the effort required to do that is so large that in practice it usually isn't.

Are you arguing that the effort required to make a regex robust and correct is larger than the effort required to make some hand-rolled character-by-character based lexer robust and correct?

Because that sounds counter-intuitive to me. A regex is a higher level DSL for lexing.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#122

Post by goto11 »

chubot wrote: post #63

This conversation would be a lot clearer with a distinction between "regexes" and "regular languages". The former is what Perl/Python/etc. have, and the latter is a mathematical concept (and automata-based non-backtracking engines like RE2, re2c, and rust/regex are closer to this set-based definition). https://www.oilshell.org/blog/2020/07/eggex-theory.html With those definitions, this part of the snarky answer is wr…

> This conversation would be a lot clearer with a distinction between "regexes" and "regular languages".

This is often an important distinction, but the point of the article is that the Stackoverflow question does not require recursion or any other non-standard regex features, and therefore can be solved using a vanilla regular expression.

So in this particular question the distinction is not important.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#123

Post by crispyambulance »

goto11 wrote: post #117
crispyambulance wrote: post #104

Earlier quoted context omitted.

Who said regular expressions are bad practice? Regexes are great but they can get abused easily. Looking more carefully at the SO question, I am inclined to ask "why?" at least a couple of times because I suspect the answer to the deeper problem the SO OP needs to solve can be worked out with a DOM parser. If not, then definitely a SAX parser could solve that specific problem and it would be more robust than handcraf…

As far as I can tell, a SAX parser does not expose the distinction between an opening tag and a self-closing tag. The tag ` ` would just emit a startElement event followed by an endElement, exactly the same as ` `. Which means it can't solve the specific problem the OP describes.

I would rather use a robust battle-tested parser and handle that edge case separately than to create regex for all of xhtml, copied off an opinionated blog, as a "hail-mary".

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#124

Post by goto11 »

jameshart wrote: post #62
jancsika wrote: post #34

I love seeing the weirdo CDATA thingy in there! CDATA ftw! E.g., you've got this enormous spec for SVG which includes CSS, but that CSS has syntax inside a style tag which could break XHTML parsers. Amateurs out there are probably thinking, "Well, why not just compromise in the spec and tell implementers to do the same thing that HTML does to parse style tags?" Well, professionals know that cannot work for myriad rea…

The 'weirdo' CDATA thing is the only thing that makes XHTML actually amenable to this approach, because XHTML is tokenizable using a regular expression-based grammar, whereas HTML without CDATA is not. As you're obviosuly aware, the language inside elements is not suitable for XML parsing. Nor is the language inside elements: var y = " ... a naive tokenizer thinks this is in a comment ... var z = "-->" There's a weir…

Correct me if I'm wrong, but I believe HTML tags can still be lexed with a regular expression. The syntax of the script element is cryptic, but it does not contain any recursive productions, so it should still be possible to lex correctly with a regular expression.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#125

Post by goto11 »

crispyambulance wrote: post #123
goto11 wrote: post #117

Earlier quoted context omitted.

As far as I can tell, a SAX parser does not expose the distinction between an opening tag and a self-closing tag. The tag ` ` would just emit a startElement event followed by an endElement, exactly the same as ` `. Which means it can't solve the specific problem the OP describes.

I would rather use a robust battle-tested parser and handle that edge case separately than to create regex for all of xhtml, copied off an opinionated blog, as a "hail-mary".

But the question is specifically about that edge case! So how to handle it, if the "robust battle-tested parser" is of no help?

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#126

Post by burnished »

IshKebab wrote: post #118
inopinatus wrote: post #116

Earlier quoted context omitted.

Well, that is snide, petty, personal, and wrong. The strangest part of this whole discussion has been the remarkable number of accounts making head-first personal character attacks. And as with that comment further back, the personal invective comes coupled to some strange language, like “StackOverflow people” - what are they, even? It sure ain’t a tribe I’d identify with. Does anyone with a login qualify? Where’s al…

> “StackOverflow people” - what are they, even? StackOverflow has a real problem with attracting strict rule followers who love over-moderating. I expect Wikipedia suffers from a similar issue but it's not such an interactive site so most people aren't exposed to it. > Where’s all that anger even coming from? StackOverflow can be an extremely frustrating experience due to people who probably think they are helping ca…

It's called the 'CodingHelp' subreddit. The quality is very low and you will see many similar questions, each with a screen shot and little explanation.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#127

Post by IshKebab »

burnished wrote: post #126
IshKebab wrote: post #118

Earlier quoted context omitted.

> “StackOverflow people” - what are they, even? StackOverflow has a real problem with attracting strict rule followers who love over-moderating. I expect Wikipedia suffers from a similar issue but it's not such an interactive site so most people aren't exposed to it. > Where’s all that anger even coming from? StackOverflow can be an extremely frustrating experience due to people who probably think they are helping ca…

It's called the 'CodingHelp' subreddit. The quality is very low and you will see many similar questions, each with a screen shot and little explanation.

That's not remotely the same.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#128

Post by IshKebab »

goto11 wrote: post #121
IshKebab wrote: post #119

Earlier quoted context omitted.

C code can be as robust as you need it to be. So why bother with formal verification, safe C coding standards, Rust, etc? The answer is that it can be robust, but the effort required to do that is so large that in practice it usually isn't.

Are you arguing that the effort required to make a regex robust and correct is larger than the effort required to make some hand-rolled character-by-character based lexer robust and correct? Because that sounds counter-intuitive to me. A regex is a higher level DSL for lexing.

That's exactly what I'm arguing. Especially because it's very unlikely that you'd write an XML/HTML parser yourself instead of using somebody else's well-tested library.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#129

Post by jameshart »

goto11 wrote: post #124
jameshart wrote: post #62

Earlier quoted context omitted.

The 'weirdo' CDATA thing is the only thing that makes XHTML actually amenable to this approach, because XHTML is tokenizable using a regular expression-based grammar, whereas HTML without CDATA is not. As you're obviosuly aware, the language inside elements is not suitable for XML parsing. Nor is the language inside elements: var y = " ... a naive tokenizer thinks this is in a comment ... var z = "-->" There's a weir…

Correct me if I'm wrong, but I believe HTML tags can still be lexed with a regular expression. The syntax of the script element is cryptic, but it does not contain any recursive productions, so it should still be possible to lex correctly with a regular expression.

Your lexer now needs to understand JavaScript too, though.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#130

Post by inopinatus »

jcelerier wrote: post #120
inopinatus wrote: post #103

Earlier quoted context omitted.

If I understand correctly, you're suggesting "Who are you" wasn't directed at me personally, "the person" wasn't referring to the OP but all possible authors, and "the question" wasn't referring to, well, the original SO question at hand, but the class of all possible questions. If so, then I see, I think: perhaps it was more intended as "Who is anyone to know the purpose and utility (of a question) better than the p…

> If I understand correctly, you're suggesting "Who are you" wasn't directed at me personally, "the person" wasn't referring to the OP but all possible authors, and "the question" wasn't referring to, well, the original SO question at hand, but the class of all possible questions. yes, exactly ? but maybe it is less common to speak in such a general way in english than in my mother tongue

It is a form in English, example idiom might be, “the man in the street”, which in general relates “anyone in any street” and thereby simply means “ordinary people”.

The potential for confusion arises when we’re already talking about a particular person, and he’s outside on the road.

Post reply on HN