Live data from Hacker News

Solving the regex of madness, and snarky answers on StackOverflow (2019)

cargocultcode.com

131–136 of 136 posts

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#131
post #124

Earlier quoted context omitted.

Correct me if I'm wrong, but I believe HTML tags can still be lexed with a regular expression. The syntax of the script element is cryptic, but it does not contain any recursive productions, so it should still be possible to lex correctly with a regular expression.

Your lexer now needs to understand JavaScript too, though.

[deleted]

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#132
post #124

Earlier quoted context omitted.

Correct me if I'm wrong, but I believe HTML tags can still be lexed with a regular expression. The syntax of the script element is cryptic, but it does not contain any recursive productions, so it should still be possible to lex correctly with a regular expression.

Your lexer now needs to understand JavaScript too, though.

The content model of the script element is gnarly for historical reasons, but it does not depend on the syntax of the scripting language used:

https://html.spec.whatwg.org/multipage/scripting.html#restri...

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#133

I love seeing the weirdo CDATA thingy in there! CDATA ftw! E.g., you've got this enormous spec for SVG which includes CSS, but that CSS has syntax inside a style tag which could break XHTML parsers. Amateurs out there are probably thinking, "Well, why not just compromise in the spec and tell implementers to do the same thing that HTML does to parse style tags?" Well, professionals know that cannot work for myriad rea…

The 'weirdo' CDATA thing is the only thing that makes XHTML actually amenable to this approach, because XHTML is tokenizable using a regular expression-based grammar, whereas HTML without CDATA is not. As you're obviosuly aware, the language inside elements is not suitable for XML parsing. Nor is the language inside elements: var y = " ... a naive tokenizer thinks this is in a comment ... var z = "-->" There's a weir…

If I understand the HTML spec correctly, these two examples are not valid HTML.

The script element content model has the constraint that if the substring "" must occur before the end-tag. (Technically, this would not be a comment though, since it is not discarded but parsed as character data.)

If I put the first example through the (X)HTML validator (https://html5.validator.nu/) I get:

> Error: The text content of element script was not in the required format: Content contains the character sequence .

I suspect this constraint on the script-element is exactly to avoid these parsing/lexing ambiguities.

A regex would still have to special-case script elements (and also style-elements I guess?) because content can contain unescaped "I don't believe there is any two-way interaction between the HTML parser and the JavaScript parser. The HTML parser passes the character data content of the script element to the JavaScript parser, but the JavaScript parse does not have any effect back on the HTML parse. (After all, it is legal for a user agent to not support JavaScript, but this should obviously not affect the parsing of the HTML.)

(Thanks for sending me down this rabbit-hole. HTML is weird!)

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#134
post #121

Earlier quoted context omitted.

Are you arguing that the effort required to make a regex robust and correct is larger than the effort required to make some hand-rolled character-by-character based lexer robust and correct? Because that sounds counter-intuitive to me. A regex is a higher level DSL for lexing.

That's exactly what I'm arguing. Especially because it's very unlikely that you'd write an XML/HTML parser yourself instead of using somebody else's well-tested library.

OK but these are two separate question.

Of course you should use an existing library if it solves the exact problem you have. Don't waste time re-implementing the wheel unless you are doing if for educational purposes. Whether such a library used regexes or not under the hood would be irrelevant as long as it works and it well tested.

But I would certainly like to hear an argument why you think a regex is less robust that a similar manual character-by-character matcher.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#135
post #28

Earlier quoted context omitted.

The headline says XHTML. HTML does not even have the self-closing tags the question is concerned about.

The headline said that the author was specifically looking to avoid 'xhtml self-contained tags', but that doesn't mean they could assume the document they were looking in was valid XHTML - just that it might contain XHTML-style 'self-closing' tags. I've seen plenty of plain HTML files that aren't well-formed XML yet contain tags.

Yeah the trailing slash in is legal in HTML, but it doesn't actually make the tag self-closing. For example is still an opening tag which require a .

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#136

Earlier quoted context omitted.

Well, that is snide, petty, personal, and wrong. The strangest part of this whole discussion has been the remarkable number of accounts making head-first personal character attacks. And as with that comment further back, the personal invective comes coupled to some strange language, like “StackOverflow people” - what are they, even? It sure ain’t a tribe I’d identify with. Does anyone with a login qualify? Where’s al…

> “StackOverflow people” - what are they, even? StackOverflow has a real problem with attracting strict rule followers who love over-moderating. I expect Wikipedia suffers from a similar issue but it's not such an interactive site so most people aren't exposed to it. > Where’s all that anger even coming from? StackOverflow can be an extremely frustrating experience due to people who probably think they are helping ca…

StackOverflow has its share of problems, including overzealous moderation. I do civic duty by often voting for reopen and moderating the reopen queue.

That said, removing moderation is not the solution. If you look at the unmoderated new questions, lots of questions are literally incomprehensible. A significant portion are homework or exam questions directly copy-pasted, but without enough context to answer - e.g. referring to table or figures or code examples which is not included in the question. Such is the sad reality of a widely known site in todays eternal september.

Post reply on HN