Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Solving the regex of madness, and snarky answers on StackOverflow (2019)

cargocultcode.com

Post reply on HN
Print view View original

21–30 of 136 posts

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#21

Post by nixpulvis »

dataflow wrote: post #16
lifthrasiir wrote: post #14

Earlier quoted context omitted.

Regular expressions in the formal language theory do not have captures anyway. The name collision is unfortunate, but we have already established that regexes in practice means a pattern language largely modelled after theoretical regular expressions and not the theoretical regular expressions themselves. At the very least the writing could have mentioned this discrepancy.

Captures are a red herring here. They don't fundamentally alter the nature of what a regex does, which is to recognize regular languages. Pointing to them as if they're some kind of justification is like calling pilots "drivers" because drivers originally drove wagons, and wagons didn't have rubber tires like cars and planes anyway. It's completely missing that the point of the distinction between a plane and a wagon…

On a related note, lookaheads (and behinds) do somewhat change the fundamental expressive power I believe.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#22

Post by megous »

adament wrote: post #6

Does the proposed regular expression really handle embedded script content correctly? From my limited understanding of HTML, pretty much only counts as closing the script contents and everything else is treated as part of the script.

Does it matter? You can create regex with lookahead for . The point is that it's possible to solve the problem from SO this way, due to the nature of the problem, not that this particular expression is perfectly correct.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#23

Post by eCa »

dataflow wrote: post #5

Try that regex on console.log(" "); Edit 1: I'm unsure if the inner is valid (X)HTML, so it might not be an issue of being unable to parse correct (X)HTML, but rather an issue of being unable to detect invalid (X)HTML. (Can someone verify?) Edit 2: It seems Chrome chokes on the space... does anyone know if the initial space is valid? I'm pretty sure I've seen parsers that accept it...

I don’t think any part of that is valid XML. There cant be space between < and the tag name, and I believe content containing tags should be in a CDATA section.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#24

Post by megous »

03b17999-4268 wrote: post #4

I haven't read the rest of the thread, but the article is even wronger than the glib replies there. HTML needn't be well formed. There are adhoc rules which let major browsers parse broken HTML. If you do not follow the spec to the letter you will have your tooling break on input that every browser thinks is acceptable. Which is why you always use whatever html parsing library comes with your language. There is no si…

> Should be seriously mentored by someone.

Maybe people who think a basic regex such as this is difficult, need some mentoring.

It doesn't even use lookahead/lookbehind or other more complicated features. It only uses non-greedy matching 2 times, which is the only thing that's more complicated than the basic AND/OR logic expressed by the rest of the expression.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#25

Post by goto11 »

dataflow wrote: post #5

Try that regex on console.log(" "); Edit 1: I'm unsure if the inner is valid (X)HTML, so it might not be an issue of being unable to parse correct (X)HTML, but rather an issue of being unable to detect invalid (X)HTML. (Can someone verify?) Edit 2: It seems Chrome chokes on the space... does anyone know if the initial space is valid? I'm pretty sure I've seen parsers that accept it...

The question is about XHTML, not HTML. HTML does not even have the self-closing tags the question is concerned about.

In XHTML, either the opening angle bracket must be escaped, or the script should be in a CDATA section.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#26

Post by lifthrasiir »

dataflow wrote: post #16
lifthrasiir wrote: post #14

Earlier quoted context omitted.

Regular expressions in the formal language theory do not have captures anyway. The name collision is unfortunate, but we have already established that regexes in practice means a pattern language largely modelled after theoretical regular expressions and not the theoretical regular expressions themselves. At the very least the writing could have mentioned this discrepancy.

Captures are a red herring here. They don't fundamentally alter the nature of what a regex does, which is to recognize regular languages. Pointing to them as if they're some kind of justification is like calling pilots "drivers" because drivers originally drove wagons, and wagons didn't have rubber tires like cars and planes anyway. It's completely missing that the point of the distinction between a plane and a wagon…

> They don't fundamentally alter the nature of what a regex does, which is to recognize regular languages.

It's a bit subjective but captures are harder than recognition. Russ Cox has once noted [1] that the extraction has to be run as a separate step after the recognition and a fast DFA can't always be used for that, suggesting they are related but different problems.

[1] https://swtch.com/~rsc/regexp/regexp3.html

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#27

Post by tannhaeuser »

03b17999-4268 wrote: post #4

I haven't read the rest of the thread, but the article is even wronger than the glib replies there. HTML needn't be well formed. There are adhoc rules which let major browsers parse broken HTML. If you do not follow the spec to the letter you will have your tooling break on input that every browser thinks is acceptable. Which is why you always use whatever html parsing library comes with your language. There is no si…

You make this more complex than it actually is. HTML is basically SGML with a DTD declaring rules for tag omission/inference, empty elements, and attribute shortforms. This is what the majority of XML heads seem to struggle with, but which nevertheless is every bit as formal as XML is, XML being just a proper subset of SGML, though too small for parsing HTML.

Then HTML has a number of historical quirks including:

- the style and script elements have special rules for comments that would make older pre-CSS, pre-Javascript browsers just see markup comments and ignore those

- the URL syntax using & ampersand characters needs to be treated special because & starts an entity reference in SGML's default concrete syntax

- the HTML 5 spec added a procedural parsing algorithm (in addition to specifying a grammar-based spec) that basically parses every byte sequence as HTML via fallback rules; for most intents and purposes, the language recognized by these rules, taken to the extreme, is not what's commonly understood as HTML

- WHATWG have added a number of element content rules on top of the HTML 4.01/HTML 5.0 baseline with ill-defined parsing rules (such as the content models for tables and description lists); the reason is precisely that WHATWG, once Ian Hickson had distilled the HTML DTD 4.01 grammar rules into the HTML 5 grammar presentation as prose, a formal basis was no longer used for vocabulary extension

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#28

Post by goto11 »

jontro wrote: post #17
goto11 wrote: post #9

Earlier quoted context omitted.

The question is about XHTML though, not HTML which have a more complex syntax.

To me it’s a bit ambiguous if the original question is about both html and xhtml. It’s tagged with both

The headline says XHTML. HTML does not even have the self-closing tags the question is concerned about.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#30

Post by mannykannot »

lifthrasiir wrote: post #14
dataflow wrote: post #12

Earlier quoted context omitted.

> it is patently false that regular expressions cannot be recursive. No, it's more like the term "regular expression" has gotten hijacked and nowadays gets abused to colloquially include, shall we say, irregular expressions. i.e. people basically say "regex" when they mean "some succinct pattern language with syntax similarities to (classical) regular expressions".

Regular expressions in the formal language theory do not have captures anyway. The name collision is unfortunate, but we have already established that regexes in practice means a pattern language largely modelled after theoretical regular expressions and not the theoretical regular expressions themselves. At the very least the writing could have mentioned this discrepancy.

>At the very least the writing could have mentioned this discrepancy.

The original meaning of 'regular expression' is very specific, and has some significant implications which are lost with the now-common and less well-defined usage. Therefore, if anything needed having this discrepancy mentioned, it was your original statement "it is patently false that regular expressions cannot be recursive", as this is an issue where the distinction is crucial. It is good to see that you have now done so, though the way you have done so suggests there is nothing of practical interest in the formal definition, which, I suggest, would be patently false.

Post reply on HN