Live data from Hacker News

Solving the regex of madness, and snarky answers on StackOverflow (2019)

cargocultcode.com

81–90 of 136 posts

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#81

Earlier quoted context omitted.

Not sure which revision that's pasted from, but it is not the original text, which can be found at https://stackoverflow.com/revisions/1732348/1 and I strongly advise against neglecting the title; doing so is how some folks blunder into misinterpretation, since the wording of that title is telegraphing a quite different underlying need to "please solve this regex puzzle".

Reading both, I see little to no difference. Is there an important change between the two revisions that I am missing? Also, my reading of the title does not, in fact, telegraph anything other than "help me with this regex puzzle."

I recommend consulting my other remarks within this topic, but on this specific point:

The title's inconsistency with the body text, i.e. "open tags" vs "opening tags", is especially and immediately notable because they are (in context) grammatically interchangeable but have dissimilar meanings. This is immediately suggestive of (but not diagnostic of) a writer revealing context and then switching to detail. As a longtime reader of requirements documents and S.O. questions, a mental flag to check the intended meaning of both is already raised at this point.

The reference to XHTML is ambiguous, since it speaks to working around XHTML that is already present, rather than defining an input or output document, which means this is something thinking at the character level rather than in terms of DOM. This impression is verified by the comparison pairs in the question body, leaving the essential context question of what are the desired inputs and outputs?" glaringly unanswered.

The early construct, "I need to ..." is a secondary flag that immediately reinforces the likelihood of a critical gap in context, since it does not explain why the need has arisen.

To anyone familiar with the structure & semantics of the HTMLs, the omission of any mention of tag closing, having discussed "opening tags", boosts the sense that something relevant is missing from the question. The obviously flawed regular expression then attracts a "probable novice" qualifier, which is amplified by both the closing "what do you think?" and the original's emoticon.

At this point, we're about ten seconds into the read-through and there's already a ton of labels pointing to a beginner who may be conflating dissimilar concepts and (through inexperience) choosing the wrong tool for the task, whatever that may be. Another few seconds to review comments, at which point it appears that author does understand the distinction between an "open tag" and an "opening tag", and this reweights the XHTML reference in the title toward being a need to output strict XHTML.

Given their apparent beginner standing it's likely this is either their first S.O. post, or one of slightly too many, and a check in their profile for proximate questions immediately reveals the latter case. References to balancing of "dangling tags" abound, collapsing all preceding problem variants and likelihoods into the only focused understanding that fits: it's a PHP novice, bright but inexperienced, who wants to normalise a nonconforming document into XHTML by balancing the tags, and that the mostly likely use case is injecting an existing HTML fragment into a RSS XML feed.

Elapsed time ca.90 seconds.

Another minute to scroll through surrounding Q&A materials, to allow alternative options to appear (they don't).

A few seconds for a chuckle at that old friend of an amusing answer, and recognise that the advice within is sound: what they really need is, indeed, a parser. Then, a pause to consider a contrasting notion, that what they're trying to do is write a parser. Consider evidence that author has taken at least one step towards inventing stacks on their own from first principles; prospect loses to existing top answer because the experience level required to write parsers cannot, ultimately, be acquired from a single S.O. question.

Last but not least; the final boss: convincing the chorus of Hacker News to accept this interpretation. That takes much longer.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#82

Earlier quoted context omitted.

Reading both, I see little to no difference. Is there an important change between the two revisions that I am missing? Also, my reading of the title does not, in fact, telegraph anything other than "help me with this regex puzzle."

I recommend consulting my other remarks within this topic, but on this specific point: The title's inconsistency with the body text, i.e. "open tags" vs "opening tags", is especially and immediately notable because they are (in context) grammatically interchangeable but have dissimilar meanings. This is immediately suggestive of (but not diagnostic of) a writer revealing context and then switching to detail. As a lon…

Consider alternatively that this is a tiny piece in the much broader puzzle of what they are trying to accomplish, that they are aware both of their own beginner status but also that, in this case, good enough will be good enough, or that they don't have the time or inclination to switch to a real parser and that's why they didn't ask about it. Or maybe even that the goal here is to specifically learn to use regex to solve a real software problem that they have. It's all a matter of interpretation of context that we don't have. As an asker of questions on SO where people provide answers to questions I specifically intended to not ask, this pattern of reading deeply into context we don't have is actually kind of annoying. Unless you provide both the answer to the actual question and then some guidance on how you really should be doing something. Those answers are very, very helpful.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#83

> The question is about finding opening tags in XHTML using a regular expression Bzzzt, wrong, sorry! The question is about finding open tags in the presence of XHTML self-closing tags. That difference alone places these interpretations gulfs apart. But there’s more: it does not specify that the input document is even XHTML, only that XHTML-style self-closing elements may be present. In fact the original question was…

> For this task, the suggestion of "use a parser" is indeed sound advice. Perhaps technically, but it's also useless advice because a parser does not exist for their particular flavor of malformed XHTML. XHTML parsers parse XHTML, which you yourself have said it wasn't: > they were trying to fix a malformed document So in the absence of a reference to a particular malformed-XHTML-recovering parser (which may or may n…

[deleted]

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#84
post #63

This conversation would be a lot clearer with a distinction between "regexes" and "regular languages". The former is what Perl/Python/etc. have, and the latter is a mathematical concept (and automata-based non-backtracking engines like RE2, re2c, and rust/regex are closer to this set-based definition). https://www.oilshell.org/blog/2020/07/eggex-theory.html With those definitions, this part of the snarky answer is wr…

> This conversation would be a lot clearer with a distinction between "regexes" and "regular languages". Very much so. > In this post, the example given IS a regex, but it IS NOT a regular language: ` # comment` The nongreedy match of .*? isn't a mathematical construct; it implies a backtracking engine. Actually, that's [edit: "it IS NOT a regular language"] wrong, at least in principle. If you're limiting it to only…

Yeah I think you are right -- the nongreedy match can be simply written as a more awkward pattern. I think regular language negation and intersection also help (which the rarer derivatives-based implementations seem to have). They are still regular and equivalent in power, but there's a richer set of operators for writing patterns.

I would also divide it into the "recognition problem" and the "parsing/capturing problem".

Recognition is just yes/no -- does it match? And notably that excludes common regex APIs that either return the end position (Python's re.match) or search (re.search). It is more like math -- is it in the set or not?

For the recognition problem, there's no difference between greedy and non-greedy, in terms of the result. (It does matter in terms of performance, which shows up in practice in backtracking engines!)

But parsing / capturing is more relevant to programmers. I don't remember all the details but there is some discussion on the interaction between greediness and capturing here: https://swtch.com/~rsc/regexp/regexp2.html

It looks like it can all be done in linear time and RE2 supports it fine.

So in that case maybe the 2 questions are closer than I thought. I have a very similar set of regexes that I use for parsing (my own) HTML, but I use one regex for each kind of token, and I do it with the help of an explicit stack.

I'm using Python's engine so I think it is better to separate each case, for performance reasons. But maybe with RE2 it would be fine to combine them all into one big expression as is done here, for "parallel" matching. The expression is a little weird because it's solving a specific problem and not something more general (which is useful and natural).

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#85

Long regexes are the root of all evil

Regexes are computer programs just like any other computer program. A long program, terse syntax, embedding one programming language in another programming language, etc. are risk factors for unmaintainable code, but are manageable in the same way that you manage the risk inherent in other complex computer programs.

I did always like elisp's "rx"[1] library for removing some of the terseness. Not enough to ever use it, of course, but it's an interesting idea.

[1] https://www.emacswiki.org/emacs/rx

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#86

Earlier quoted context omitted.

I recommend consulting my other remarks within this topic, but on this specific point: The title's inconsistency with the body text, i.e. "open tags" vs "opening tags", is especially and immediately notable because they are (in context) grammatically interchangeable but have dissimilar meanings. This is immediately suggestive of (but not diagnostic of) a writer revealing context and then switching to detail. As a lon…

Consider alternatively that this is a tiny piece in the much broader puzzle of what they are trying to accomplish, that they are aware both of their own beginner status but also that, in this case, good enough will be good enough, or that they don't have the time or inclination to switch to a real parser and that's why they didn't ask about it. Or maybe even that the goal here is to specifically learn to use regex to…

These objections may seem relevant to your personal experience, but they don't pertain to the case at hand, for which ample context is available.

Dealing with responses from people who've rushed to write something without bothering to properly consider the problem is, of course, internet 101, but such answers generally take the short road to content oblivion.

For those answers of any quality, however, note this: if people are answering, for free, on their own time, questions that you don't think you asked, but they evidently do, then consider taking some personal responsibility for that outcome. Complaining about it would seem wholly entitled and ungrateful, and unconstructive besides. If they are well-meaning beginners, coach them: it's how the stone soup gets made.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#87

Earlier quoted context omitted.

Oh, are they here? It’d be fascinating to hear from them. Alternatively, perhap, that hostile tone is suggesting I’m personally unqualified to interpret loosely framed questions? I suppose, since I’ve only been doing it for a few decades, I’m definitely a novice by any standard, and my tendency to observe and follow up on anomalous, incomplete, subtly conflicting, or otherwise inexplicable requirements by investigati…

I have to ask, is this series of comments some kind of performance art or some sort of social experiment? Or is this unironically how you write/speak/act?

You don't have to ask. Isn't there a rule about fake curiosity here?

I hope this is the last comment in this bad bad thread.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#88

Earlier quoted context omitted.

Oh, are they here? It’d be fascinating to hear from them. Alternatively, perhap, that hostile tone is suggesting I’m personally unqualified to interpret loosely framed questions? I suppose, since I’ve only been doing it for a few decades, I’m definitely a novice by any standard, and my tendency to observe and follow up on anomalous, incomplete, subtly conflicting, or otherwise inexplicable requirements by investigati…

I have to ask, is this series of comments some kind of performance art or some sort of social experiment? Or is this unironically how you write/speak/act?

To someone paying enough attention to crystallize the trichotomy, all I can say is, thank you for reading.

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#89
post #63

This conversation would be a lot clearer with a distinction between "regexes" and "regular languages". The former is what Perl/Python/etc. have, and the latter is a mathematical concept (and automata-based non-backtracking engines like RE2, re2c, and rust/regex are closer to this set-based definition). https://www.oilshell.org/blog/2020/07/eggex-theory.html With those definitions, this part of the snarky answer is wr…

> This conversation would be a lot clearer with a distinction between "regexes" and "regular languages". Very much so. > In this post, the example given IS a regex, but it IS NOT a regular language: ` # comment` The nongreedy match of .*? isn't a mathematical construct; it implies a backtracking engine. Actually, that's [edit: "it IS NOT a regular language"] wrong, at least in principle. If you're limiting it to only…

Does your regex assumes that "-->" must be prefixed by a space? This is not the case in XML. (Also the string "--" must not occur inside a comment, so the last clause is not necessary.)

Re: Solving the regex of madness, and snarky answers on StackOverflow (2019)

#90

> The question is about finding opening tags in XHTML using a regular expression Bzzzt, wrong, sorry! The question is about finding open tags in the presence of XHTML self-closing tags. That difference alone places these interpretations gulfs apart. But there’s more: it does not specify that the input document is even XHTML, only that XHTML-style self-closing elements may be present. In fact the original question was…

What if we looked at the XHTML parsers trusted by the people who mindlessly dismiss the utility of regular expressions and found they were constructed using a lexer that relied on regular expressions.
Post reply on HN