Live data from Hacker News

Is there a regular expression to detect a valid regular expression?

stackoverflow.com

71–80 of 224 posts

Re: Is there a regular expression to detect a valid regular expression?

#71
post #54
post #48

Earlier quoted context omitted.

The question is about how to identify start and end-tags in XHTML. What would be an appropriate tool for that job?

A parser. Specifically, an XHTML parser.

How do you think an XHTML parser is written? In particular, how does an XHTML parser identify tokens like start and end tags?

Re: Is there a regular expression to detect a valid regular expression?

#72

I hate it how the accepted answer is incorrect and does not teach the fundamental property of regular expressions.

The fundamental property of regular expressions is that the programming/software engineering concept of "regex" or "regular expression" is fundamentally different from the mathematical concept of "regular language".

If you see "regex" without any special context and start thinking about what it can/can't do, then the first association should be with something like PCRE and its capabilities, not something that's bound by the pumping lemma.

Re: Is there a regular expression to detect a valid regular expression?

#73
post #2

I believe Zalgo has the answer to this, via an equivalent question. https://stackoverflow.com/questions/1732348/regex-match-open...

My favorite part of it is the moderators' note at the end: > Moderator's Note > This post is locked to prevent inappropriate edits to its content. The post looks exactly as it is supposed to look - there are no problems with its content. Please do not flag it for our attention.

The big problem with the content is suggesting to use an XML parser to parse HTML. That might often work, and several XML parsers have some sort of HTML "mode", but in general, no, not really. HTML documents are frequently invalid XML. HTML documents lack an XML declaration, they don't close all tags, etc. They might even lack a root element. Someone mentioned this in a comment on SO, but it was (as far as I could see) not really addressed or answered.

HTML 5 has it's own parsing rules, specified at [1]. However, I don't know of any implementation of these outside of browsers. I normally use Beatuiful Soup [2] (for Python) or HTML Agility Pack [3] (for .NET), and while I don't think these implement the exact standard, they're easier than struggling with a XML parser for HTML "in the wild".

1: https://html.spec.whatwg.org/multipage/parsing.html 2: https://www.crummy.com/software/BeautifulSoup/ 3: https://html-agility-pack.net/

Re: Is there a regular expression to detect a valid regular expression?

#74

> Is there a regular expression to detect a valid regular expression? No, there is not. For example, parentheses in a regex must be balanced, and (famously) there is no regex to detect balanced parentheses.

More precisely, it is arbitrarily deep nested clauses that cannot be parsed, on account of true REs not being able to either use recursion or keep count of how deep they are.

https://blogs.msdn.microsoft.com/jaredpar/2008/10/15/regular...

Re: Is there a regular expression to detect a valid regular expression?

#75
post #64

Earlier quoted context omitted.

The fact that someone might make a mistake in doing something does not show it cannot be done. More generally, You seem to be mistaking validating a program's source with the question of whether it performs its intended purpose. These are different things, and attempting to conflate them will only lead to confusion.

> You seem to be mistaking validating a program's source with the question of whether it performs its intended purpose. In general, that is a useful distinction to make, but you forgot about the edge case where the distinction is meaningless. A self-hosting compiler's intended purpose is to validate its own source code.

Then your mistake appears to be in failing to see that your perceived edge case does not invalidate the first sentence of my reply. If the sole purpose of a parser is to syntactically validate its own source (which is not the case for a compiler's parser, by the way, not even if we expand 'its own source' to 'arbitrary input'), then if it does that correctly, that's all there is to it.

Re: Is there a regular expression to detect a valid regular expression?

#76
post #70
post #44

Earlier quoted context omitted.

Here is the answer to the question: https://www.cargocultcode.com/solving-the-zalgo-regex/ tl;dr: It can indeed be solved relatively easily with a regex.

It doesn't work for me with regex101. "The preceding token is not quantifiable" on this part: |

See, this is kinda what I mean. Maybe you can detect tags with regex, but maybe you shouldn't, given the widespread but subtle differences in regex engines.

Perhaps the entire approach of "why are you trying to parse X?" Needs to be traced and re-evaluated.

Re: Is there a regular expression to detect a valid regular expression?

#77

> Is there a regular expression to detect a valid regular expression? No, there is not. For example, parentheses in a regex must be balanced, and (famously) there is no regex to detect balanced parentheses.

More precisely, it is arbitrarily deep nested clauses that cannot be parsed, on account of true REs not being able to either use recursion or keep count of how deep they are. https://blogs.msdn.microsoft.com/jaredpar/2008/10/15/regular...

Can regex engines actually process arbitrarily deep nestings themselves though?

It seems to me it would start getting computationally expensive very quickly to search for those kinds of regexes, so is 'arbitrarily deep' a practical concern for todays hardware?

Re: Is there a regular expression to detect a valid regular expression?

#78
post #64

Earlier quoted context omitted.

> You seem to be mistaking validating a program's source with the question of whether it performs its intended purpose. In general, that is a useful distinction to make, but you forgot about the edge case where the distinction is meaningless. A self-hosting compiler's intended purpose is to validate its own source code.

Then your mistake appears to be in failing to see that your perceived edge case does not invalidate the first sentence of my reply. If the sole purpose of a parser is to syntactically validate its own source (which is not the case for a compiler's parser, by the way, not even if we expand 'its own source' to 'arbitrary input'), then if it does that correctly, that's all there is to it.

Your mistake appears to be - ignoring the alternative hypothesis.

You are mistaken, not me.

The consequences of Godel's incompleteness theorem are such that a mathematical system (such as a compiler) cannot prove its own correctness.

It can only prove that it is free from known errors. Once you define what an "error" is.

Re: Is there a regular expression to detect a valid regular expression?

#79

> Is there a regular expression to detect a valid regular expression? No, there is not. For example, parentheses in a regex must be balanced, and (famously) there is no regex to detect balanced parentheses.

This is true if you want to handle arbitrarily deep nested parents.

But any given string you are asked to validate to see if it is a regex will be a finite length, and contain a finite number of opening paren characters. So it’s maximum possible nesting depth is known.

And you can construct, fairly trivially, a regex that can validate paren nesting up to a fixed depth.

So, in practice, you could use a regular expression to validate the paren nesting of any given string - if you were allowed to prepare the regex based on the string length, or on running another regex over the string first.

What I suspect you might not be able to validate is that in a regex, while [a-z] is valid, [z-a] is not.

Re: Is there a regular expression to detect a valid regular expression?

#80
post #38
post #17

Earlier quoted context omitted.

It’s a famous post on StackOverflow, but I don’t find it particularly helpful.

If you were looking for the reason why a regex cannot parse HTML, it is because HTML has matching nested tags and regex parsers are finite state machines (FSM). What this means is that a regex parser is like a goldfish. It only knows about the state it is currently in (what it just read) and which possible states it may transition to (what is legally allowed to come next). The fish never remembers where it was before…

>What this means is that a regex parser is like a goldfish.

Like a drunk goldfish, or a sober goldfish?

'A method to study short-term memory (STM) in the goldfish.'

'Twenty-one common goldfish (13-15.5 cm long) were randomly divided into alcohol (A) and nonalcohol (NA) groups and were trained in an alcohol solution of 400 mg/100 ml or in water, respectively. All alcohol fish were placed in an alcohol solution of 400 mg/100 ml for 3 hr before training in the same alcohol concentration. Fish were trained on a position discrimination task for 2 consecutive days. The door used for training was that opposite to each fish's spontaneous preference. Savings in relearning on Day 2 was taken as a measure of long term memory strength. Only fish which reached criterion on both days were immediately given 10 forced reversal trails in the opposite direction (i.e., a fish trained on right door was forced to choose the left door.) A and NA subjects were then tested after a 5 min (STM) delay, respectively, in a free choice situation for 10 trails (i.e., neither door was blocked). The results suggest that alcohol facilitates the STM of the forced reversal information.'

https://www.ncbi.nlm.nih.gov/pubmed/935220

Post reply on HN