Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The dangers of single line regular expressions

greg.molnar.io

Post reply on HN
Print view View original

61–70 of 133 posts

Re: The dangers of single line regular expressions

#61

Post by medstrom »

js2 wrote: post #39
neilk wrote: post #6

In my experience `$` does reliably mean end of string for regular expressions, unless you specifically ask for "multiline" mode. Ruby seems to be in multiline mode all the time? $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foobar") else "no"' yes $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\nbar") else "no"' no $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\…

The potential trouble with $ (even in single-line mode) is that it matches the end of a string BOTH with AND without a newline at the end. If you're using it to ensure the string has no newline before doing something with it, this can lead to trouble. $ python3 -c 'import re; print("yes" if re.search(r"^foo$", "foo") else "no")' yes $ python3 -c 'import re; print("yes" if re.search(r"^foo$", "foo\n") else "no")' yes…

Correct me if I'm wrong, but if you extract a capture group (^foo$), you would get "foo" without the "\n", right?

If so, it is not "matching the end of a string" at all. Just end of line. That's exactly as expected in single-line mode, so it's good. May mismatch your expectations in multi-line mode though.

Re: The dangers of single line regular expressions

#62

Post by brobinson »

neilk wrote: post #6

In my experience `$` does reliably mean end of string for regular expressions, unless you specifically ask for "multiline" mode. Ruby seems to be in multiline mode all the time? $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foobar") else "no"' yes $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\nbar") else "no"' no $ python -c 'import re; print "yes" if re.match(r"^[a-z ]+$", "foo\…

Note that Ruby also has \z which is what you generally want instead of \Z.

(\Z allows a trailing newline, \z does not)

Re: The dangers of single line regular expressions

#63

Post by Akronymus »

sfink wrote: post #12
Akronymus wrote: post #7

Earlier quoted context omitted.

> Alternatively, don't validate and then use the original. Instead, pull out the acceptable input and use that. Parse don't validate https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

Heh. I wrote up my comment, and then thought "hey, I bet that's what that 'Parse don't validate' article meant, the one I never quite got around to reading." So I pulled it up — great article! — but then didn't post the link because it uses the type system to record the results of the parse. Whereas here, you'd probably parse from a string into another string. But philosophically I agree, that's exactly the relevant…

parsing from a string to a string runs the risk of erroneously assigning the original value to the new string. Which kinda defeats the whole parsing, not validating.

What would work is having a small object holding a readonly string which parses the original on creation, then becomes immutable.

Re: The dangers of single line regular expressions

#64

Post by JonChesterfield »

Regular expressions make me sad about our industry.

If you read the early papers, you get a very clear language for pattern matching on sequences. They have really nice properties - the compilation to finite automata gives you decidable equality and decidable minimisation. As in you can compile equivalent regex to exactly the same state machine however they were expressed.

At some point perl happened and that seems to have sent us down a path to encoding the regular expression in an illegible subset of ascii. The backtracking implementation cost us negation and intersection. What should be linear time matching becomes exponential.

Emacs will let you write regex in s-expressions at which point they're much easier to read. Everywhere else has gone with "looks like Perl but has different semantics, which we kind of document, be lucky".

I started writing tests to check that regex I'd begrudgingly converted to the perl style behaved the same under different engines and the divergence is rough. Granted I was parsing regex with regex which is possibly a path to insanity but things like a literal [ were a real puzzle to match on different implementations.

I don't know that the horrible syntax on semantic beauty is due to perl but it looks likely from a superficial standpoint.

Re: The dangers of single line regular expressions

#65

Post by [deleted] »

JonChesterfield wrote: post #64

Regular expressions make me sad about our industry. If you read the early papers, you get a very clear language for pattern matching on sequences. They have really nice properties - the compilation to finite automata gives you decidable equality and decidable minimisation. As in you can compile equivalent regex to exactly the same state machine however they were expressed. At some point perl happened and that seems t…

[deleted]

Re: The dangers of single line regular expressions

#66

Post by jacobolus »

JonChesterfield wrote: post #64

Regular expressions make me sad about our industry. If you read the early papers, you get a very clear language for pattern matching on sequences. They have really nice properties - the compilation to finite automata gives you decidable equality and decidable minimisation. As in you can compile equivalent regex to exactly the same state machine however they were expressed. At some point perl happened and that seems t…

If you read the early papers you get a very limiting mathematical tool of mainly theoretical interest. At some point perl happened and regular expressions became a ubiquitous practical tool saving programmers collectively millions of hours of labor.

Re: The dangers of single line regular expressions

#68

Post by wlesieutre »

JonChesterfield wrote: post #64

Regular expressions make me sad about our industry. If you read the early papers, you get a very clear language for pattern matching on sequences. They have really nice properties - the compilation to finite automata gives you decidable equality and decidable minimisation. As in you can compile equivalent regex to exactly the same state machine however they were expressed. At some point perl happened and that seems t…

Swift's RegexBuilder DSL from a couple years ago gets away from the illegible subset of ASCII.

Easy to explode into a lot lines, but I'd rather have a 50 line RegexBuilder implementation than try to keep track of what the equivalent single-line version is doing. Especially if you ever have to come back to it later and understand it again.

And if you ever make revisions in RegexBuilder you have useful diffs instead of "the one line that does everything is different than before."

https://developer.apple.com/documentation/regexbuilder

Are there similar tools in any other languages?

Post reply on HN