Live data from Hacker News

Regex character "$" doesn't mean "end-of-string"

sethmlarson.dev

151–160 of 382 posts

Re: Regex character "$" doesn't mean "end-of-string"

#151
post #12

Does anyone consider RegEx to be standardised? Moving to a new context is always a relearning exercise in my experience.

Delightfully, RFC 9485 https://datatracker.ietf.org/doc/rfc9485/ "I-Regexp: An Interoperable Regular Expression Format" was published just back in October last year!

Re: Regex character "$" doesn't mean "end-of-string"

#152
post #140

Earlier quoted context omitted.

The line delimiter is a newline. If you have a file containing `A\nB\nC` in a file, the file is three lines long. I guess it could be argued that a file containing `A\nB\nC\n` has four lines, with the fourth having zero length. That a regex is applying to an in memory string vs a file doesn't feel to me like it should have different semantics. Digging into the history a little, it looks like regexes were popularized…

Technically the “newline” character is actually a line _terminator_. Hence “A\n” is one line, not two. The “\n” is always at the end of a line by definition.

So if you have "A" in a file with no newline, there are no lines in that file?

Re: Regex character "$" doesn't mean "end-of-string"

#153
post #150
post #97

Earlier quoted context omitted.

…which both excludes addresses allowed by the RFC and includes addresses disallowed by the RFC. (For example, the RFC disallows two consecutive dots in the local-part.)

I take the descriptivist approach to email validation, rather than the prescriptivist. I know an email has to have a domain name after the @ so I know where to send it. I also know it has to have something before the @ so the domain’s email server knows how to handle it. But do I care if the email server is supports sub addresses, characters outside of the commonly supported range (eg quotation marks and spaces), or…

I generally agree, but the two consecutive dots (or leading/trailing dots) are an example that would very likely be a typo and that you wouldn’t particularly want to send. Similar for unbalanced quotes, angle brackets, and other grammar elements.

Re: Regex character "$" doesn't mean "end-of-string"

#154

Earlier quoted context omitted.

regex is useful but chatgpt is amazing at it, so why spend a minute keeping such useless knowledge in mind. if you know where to find something no point in knowing it.

Does gpt produce efficient regex? Are there any experts here that can assess the quality and correctness of gpt-generated regex? I wonder how regex responses by gpt are validated if the prompter does not have the knowledge to read the output.

That was one of my first uh oh moments with gpt. Getting code that clearly had untestable/unreadable regexen, which given the source must have meant the regex were gpt generated. So much is going to go wrong, and soon.

Re: Regex character "$" doesn't mean "end-of-string"

#155
post #150
post #97

Earlier quoted context omitted.

…which both excludes addresses allowed by the RFC and includes addresses disallowed by the RFC. (For example, the RFC disallows two consecutive dots in the local-part.)

I take the descriptivist approach to email validation, rather than the prescriptivist. I know an email has to have a domain name after the @ so I know where to send it. I also know it has to have something before the @ so the domain’s email server knows how to handle it. But do I care if the email server is supports sub addresses, characters outside of the commonly supported range (eg quotation marks and spaces), or…

Yeah, that's about as far as I've ever been comfortable going in terms of validating email addresses too: some stuff followed by "@" followed by more stuff.

Though I guess adding a check for invalid dot patterns might be worthwhile.

Re: Regex character "$" doesn't mean "end-of-string"

#156

Seriously, just write one unit test for your regex.

Indeed, one should test any regex one puts any trust in, but the problem is that if you take as a fact something that is actually a false assumption (as the author did here), your test may well fail to find errors which may cause faults when the regex is put to use.

This, in a nutshell, is the sort of problem which renders fallacious the notion that you can unit-test your way to correct software.

Re: Regex character "$" doesn't mean "end-of-string"

#157

Earlier quoted context omitted.

Technically the “newline” character is actually a line _terminator_. Hence “A\n” is one line, not two. The “\n” is always at the end of a line by definition.

So if you have "A" in a file with no newline, there are no lines in that file?

Yes, that is a file with zero lines that ends with an "incomplete line". Processing of such files by standard line-oriented utilities is undefined in the opengroup spec. So, for instance, the effect of "grep"ping such a file is not defined. Heck, even "cat"ting such a file gives non-ideal results, such as colliding with the regular shell prompt. For this reason, a lot of software projects I work on check and correct this condition whenever creating a commit.

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... ("text file")

Re: Regex character "$" doesn't mean "end-of-string"

#158
post #99

Earlier quoted context omitted.

Because even after disabling multi-line you are still dealing with line-based semantics when you use ^ or $, the newline at the end is still not part of the content. You have to use \A and \Z if you want to treat all characters as a string instead of one or multiple lines.

> Because even after disabling multi-line you are still dealing with line-based semantics when you use ^ or $ No, you're not, except for this weird corner case where `$` can match before the last `\n` in a string. It's not just any `\n` that non-multiline `$` can match before. It's when it's the last `\n` in the string. See: >>> re.search('cat$', 'cat\n') >>> re.search('cat$', 'cat\n\n') >>> This is weird behavior. I…

But that is exactly what it means, the end of the line is before the terminating newline or at the end of the string if there is no terminating newline. Both ^ and $ always match at start or end of lines, \A and \Z match at the start or end of the string. The difference between multi-line and not is whether or not internal newlines end and start lines, it does not change the semantics from end of line to end of string. And if you are not in multi-line mode but have internal newlines, then you might also want single-line/dot-all mode.

One could certainly have a debate whether this behavior is too strongly tied to the origins of regular expressions and now does more harm than good, but I am not convinced that this would be an easy and obvious choice to have breaking change.

Re: Regex character "$" doesn't mean "end-of-string"

#159

Earlier quoted context omitted.

Technically the “newline” character is actually a line _terminator_. Hence “A\n” is one line, not two. The “\n” is always at the end of a line by definition.

So if you have "A" in a file with no newline, there are no lines in that file?

[deleted]

Re: Regex character "$" doesn't mean "end-of-string"

#160
The table in the article makes this look complicated, but it really isn't. All the cases in the article can be grouped into two families:

- The JS/Go/Rust family, which treats $ like \z and does not support \Z at all

- The Java, .NET, PHP, Python family, which treats $ like \Z and may or may not (Python) support \z.

\Z does away with \n before the end of the string, while \z treats \n as a regular character. For multiline $ the distinction doesn't matter, because \n is the end.

Really the only deviation from the rule is Python's \Z, which is indeed weird.

Post reply on HN