Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Solving the regex of madness, and snarky answers on StackOverflow (2019)

cargocultcode.com

Post reply on HN
Print view View original

61–70 of 136 posts

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#61

Post by SavantIdiot »

Oh I've seen this many times in different forms. Especially with regexes.

You know what this is a great example of? A case where hacking makes a mess, and thinking before coding solves the problem.

The madness comes from using the wrong tool for the problem. Yes, you can hack a regex to parse XHTML this might be "good enough", but it is more robust, cleaner and easier to explain if you use a lexical tokenizer and a grammar.

The lure is an illusion that comes from an initial effort assessment. Where the effort to hack a quick-and-dirty regex (call this Ehack) vs a "oh, man, you mean I gotta think about the problem" (call this Ethink) appears as "Equick >> Ethink," driven by the thought process, "I'm almost there, this regex just needs one more tweak." Aka, the gambler's fallacy: it comes into play and the sunk costs are ignored.

TL;DR - Use the right tool for the problem, even if it means a slightly larger up-front effort investment.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#62

Post by jameshart »

jancsika wrote: post #34

I love seeing the weirdo CDATA thingy in there! CDATA ftw! E.g., you've got this enormous spec for SVG which includes CSS, but that CSS has syntax inside a style tag which could break XHTML parsers. Amateurs out there are probably thinking, "Well, why not just compromise in the spec and tell implementers to do the same thing that HTML does to parse style tags?" Well, professionals know that cannot work for myriad rea…

The 'weirdo' CDATA thing is the only thing that makes XHTML actually amenable to this approach, because XHTML is tokenizable using a regular expression-based grammar, whereas HTML without CDATA is not. As you're obviosuly aware, the language inside elements is not suitable for XML parsing. Nor is the language inside elements:

   
     var y = "
   
... a naive tokenizer thinks this is in a comment ...

   
     var z = "-->"
There's a weird interaction here between the Javascript and HTML parsers, because 
   
      var a = 1;
      if (0
   
... and this should not be in a comment

   
     if (a-->-10) { document.write('a-- should still be > -10, so this should print, too'); }
The regex in the article will miss the opening

tags here, because it assumes that it's being given valid, tokenizable XHTML.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#63

Post by chubot »

This conversation would be a lot clearer with a distinction between "regexes" and "regular languages". The former is what Perl/Python/etc. have, and the latter is a mathematical concept (and automata-based non-backtracking engines like RE2, re2c, and rust/regex are closer to this set-based definition).

https://www.oilshell.org/blog/2020/07/eggex-theory.html

With those definitions, this part of the snarky answer is wrong:

HTML is not a regular language and hence cannot be parsed by regular expressions

That is, regular expressions as found in the wild can parse more than regular languages. (And that does happen to be useful in the HTML case!)

This answer is also irrelevant, since the poster is asking for a solution with regexes, NOT regular languages:

I think the flaw here is that HTML is a Chomsky Type 2 grammar (context free grammar) and a regular expression is a Chomsky Type 3 grammar (regular grammar).

In this post, the example given IS a regex, but it IS NOT a regular language:

     # comment
The nongreedy match of .*? isn't a mathematical construct; it implies a backtracking engine.

I gave my analysis here and listed 3 or 4 caveats: https://news.ycombinator.com/item?id=26359556

I prefer to use regular languages and an explicit stack, but this is not really what the original question was asking.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#64

Post by IshKebab »

inopinatus wrote: post #39

> The question is about finding opening tags in XHTML using a regular expression Bzzzt, wrong, sorry! The question is about finding open tags in the presence of XHTML self-closing tags. That difference alone places these interpretations gulfs apart. But there’s more: it does not specify that the input document is even XHTML, only that XHTML-style self-closing elements may be present. In fact the original question was…

Read the regex. It handles self-closing tags fine.

Also you're doing the intensely annoying thing that lots of StackOverflow people do of imagining that the asker really wanted to ask a different question. It happens sometimes. But you shouldn't just jump in and assume that they don't know what they want and you're so much smarter than them so you know what they really want.

Offer additional answers if you want, but answer the question they asked first.

(Sorry, pet peeve.)

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#65

Post by IshKebab »

wodenokoto wrote: post #36
NtrllyIntrstd wrote: post #11

The author seems to be missing the point, in my opinion. While it is certainly true that often one can solve simple, seemingly innocent sub-problems within more general languages, the transitions from "I see I can solve this simple program with regex'es!" to "Then I can probably solve this other, almost identical problem as well!" and have the problem explode right into your face are subtle (almost imperceivable to a…

The article goes as far as to say that a parser is not the right tool. > Not only can the task be solved with a regular expression - regular expressions are basically the only practical way to solve the problem. Which is why none of the clever answers actually suggest another way to solve the problem. So no, the author is not missing the point at all.

I mean that bit is clearly wrong. An XML/HTML parser is a perfectly practical way to solve the problem.

However I completely agree that they didn't miss the point. A regex to do this might be fine for hacky things that you don't need to be robust (e.g. for searching for stuff, measuring stats, one-off scripts etc.).

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#66

Post by inopinatus »

IshKebab wrote: post #64
inopinatus wrote: post #39

> The question is about finding opening tags in XHTML using a regular expression Bzzzt, wrong, sorry! The question is about finding open tags in the presence of XHTML self-closing tags. That difference alone places these interpretations gulfs apart. But there’s more: it does not specify that the input document is even XHTML, only that XHTML-style self-closing elements may be present. In fact the original question was…

Read the regex. It handles self-closing tags fine. Also you're doing the intensely annoying thing that lots of StackOverflow people do of imagining that the asker really wanted to ask a different question. It happens sometimes. But you shouldn't just jump in and assume that they don't know what they want and you're so much smarter than them so you know what they really want. Offer additional answers if you want, but…

You're not the first to take that line, so I'll refer you to my previous observations: https://news.ycombinator.com/item?id=27097403

There's no wild assumption going on here. I just bothered to keep reading, very carefully, everything the original author actually wrote.

Then, please, further reflect that Stack Overflow is not Codewars; it is a forum for practical, focused, and relevant problem-solving advice, and at its best the moderation and answer processes help folks to iteratively revise and improve their questions. A crucial step is, therefore, analytically clarifying both the parameters and the intended outcome.

Contextualisation and focusing of requirements is a familiar and essential skill for any programmer being handed a requirements statement, and answering S.O. questions involves the same exercise, just in vignette.

The handling of this question was not, in this sense, the best. But expecting anything less, would just be a load of fizzbuzz.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#68

Post by IshKebab »

h2odragon wrote: post #55
nooyurrsdey wrote: post #40

This is one of those things that people will debate about endlessly and ultimately it feels so silly. The poster asked how to do it, and this person provided a practical regex to cover most (if not all) cases. Everything else is just pedantic debate.

The pedantic discussions can be fun and educational, but the regex based hack gets the job done in a few minutes, while the pedants are still wrestling with parsers and libraries. ... and then there's the anticipated joy of seeing the pedants' complicated, theoretically correct solution explode because the input wasn't what they assumed, in the first place. The pedants that have that experience either become enlighte…

The issue is that sometimes you should use a robust parser and do it properly, and sometimes a hacky regex is fine. But people forget that when arguing about which you should use.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#69

Post by hansvm »

goto11 wrote: post #54
AzzieElbab wrote: post #32

Earlier quoted context omitted.

I am guilty of this type of thinking as well. I saw the errors of my ways when I had to fix someone’s else regular expressions

So what do you use instead of regular expressions for such tasks?

IMO the biggest problem with regexes as seen in the wild is a lack of composability. If you need some kind of pattern like "[setA][setA+setB]{0,n}" then you'll copy-paste the definition of setA in both places. If you need to re-use that entire regex you'll copy-paste it again and construct a monstrous string with a really well-defined structure that isn't even slightly apparent without a reverse engineering session.

Up to a point you can solve that by just giving names to relevant sub-expressions, using a regex builder, etc, but in my experience if I'm going to write even a moderately complicated regex I'll probably be better served with something like parsec (a python implementation here [0]) in whichever language I'm currently using.

That isn't to say that regexes don't have their place -- anything quick and dirty, situations where you need to handle unsanitized input (mind you, the builtin regex engine is probably vulnerable to exponential runtime attacks) and don't want to execute a turing-complete language, etc.... I just think it has bad ergonomics for any parser you might use more than once, and I haven't yet regretted using parsec in situations where a complex regex would have sufficed.

[0] https://pythonhosted.org/parsec/

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#70

Post by Rapzid »

03b17999-4268 wrote: post #4

I haven't read the rest of the thread, but the article is even wronger than the glib replies there. HTML needn't be well formed. There are adhoc rules which let major browsers parse broken HTML. If you do not follow the spec to the letter you will have your tooling break on input that every browser thinks is acceptable. Which is why you always use whatever html parsing library comes with your language. There is no si…

The CDATA section doesn't appear to match this first example from Wikipedia:

    John Smith]]>
So it's either bugged due to the spaces or there is something else going on I don't understand; complexity.

Agree on the sentiment. Through a "Simple made easy" lens, easy or hard, there is a lot of complexity in the task and this solution..

Post reply on HN