Regex character "$" doesn't mean "end-of-string" • Hacker News Board

Re: Regex character "$" doesn't mean "end-of-string"

#11

Post by wolletd » Wed, Mar 20, 2024, 8:54 AM UTC

ikiris wrote: post #2

this is mostly due to the different types of regex and less about it being platform dependent. $ was end of string in pcre which is the "old" perl compatible regex. python has its own which has quirks as mentioned, re2 is another option in go for example, and i think rust has its own version as well iirc.

The differences of the various regex "dialects" came to me over the years of using regular expressions for all kinds of stuff.

Matching EOL feels natural for every line-based process.

What I find way more annoying is escaping characters and writing character groups. Why can't all regex engines support '\d' and '\w' and such? Why, in sed, is an unescaped '.' a regex-dot matching any character, but an unescaped '(' is just a regular bracket?

Re: Regex character "$" doesn't mean "end-of-string"

#12

Post by beardyw » Wed, Mar 20, 2024, 8:55 AM UTC

Does anyone consider RegEx to be standardised? Moving to a new context is always a relearning exercise in my experience.

Re: Regex character "$" doesn't mean "end-of-string"

#13

Post by 7bit » Wed, Mar 20, 2024, 9:00 AM UTC

pjmlp wrote: post #3

ikiris wrote: post #2

this is mostly due to the different types of regex and less about it being platform dependent. $ was end of string in pcre which is the "old" perl compatible regex. python has its own which has quirks as mentioned, re2 is another option in go for example, and i think rust has its own version as well iirc.

Indeed, there isn't any kind of universal regexp standard.

We should create a new RegEx flavour that standardises RegEx for good!

Re: Regex character "$" doesn't mean "end-of-string"

#14

Post by rusk » Wed, Mar 20, 2024, 9:00 AM UTC

beardyw wrote: post #12

Does anyone consider RegEx to be standardised? Moving to a new context is always a relearning exercise in my experience.

My understanding is it was standardised for Posix but the variants in popular use have so many variations.

I consider sed to be the baseline. If you can do sed you can do anything but it’s seriously limited.

Re: Regex character "$" doesn't mean "end-of-string"

#15

Post by telotortium » Wed, Mar 20, 2024, 9:00 AM UTC

beardyw wrote: post #12

Does anyone consider RegEx to be standardised? Moving to a new context is always a relearning exercise in my experience.

Languages invented after Perl will generally use some flavor of Perl regex syntax, but there are always some minor differences. The issue of the meaning of `$` and changing it via multi-line mode is usually consistent though.

Re: Regex character "$" doesn't mean "end-of-string"

#16

Post by wolletd » Wed, Mar 20, 2024, 9:01 AM UTC

beardyw wrote: post #12

Does anyone consider RegEx to be standardised? Moving to a new context is always a relearning exercise in my experience.

At some point, I felt like I knew them all. There are probably more regex dialects out there, but I don't encounter them and my set of knowledge works most of the time.

I feel it's like driving a rental car. It behaves slightly different than your own car, some features missing, some other features added, but in general, most of the things are pretty similar.

Re: Regex character "$" doesn't mean "end-of-string"

#17

Post by MattHeard » Wed, Mar 20, 2024, 9:04 AM UTC

beardyw wrote: post #12

Does anyone consider RegEx to be standardised? Moving to a new context is always a relearning exercise in my experience.

My working assumption has always been to check the docs of your specific regexp parser, and to write some tests (either automated or manually in a REPL) with specific patterns that you are interested in using.

Re: Regex character "$" doesn't mean "end-of-string"

#18

Post by ghusbands » Wed, Mar 20, 2024, 9:04 AM UTC

> Note: The table of data was gathered from regex101.com, I didn't test using the actual runtimes.

Has anyone confirmed this behaviour directly against the runtimes/languages? Newlines at the end of a string are certainly something that could get lost in transit inside an online service involving multiple runtimes.

Re: Regex character "$" doesn't mean "end-of-string"

#19

Post by masswerk » Wed, Mar 20, 2024, 9:06 AM UTC

As for the good old reference implementation (not "Parameter Efficient Reinforcement Learning"):

  my $string = "cat\n";
  /cat$/s  -> true
  /cat\Z/s -> true
  /cat\z/s -> false

Re: Regex character "$" doesn't mean "end-of-string"

#20

Post by susam » Wed, Mar 20, 2024, 9:07 AM UTC

rusk wrote: post #14

beardyw wrote: post #12

Does anyone consider RegEx to be standardised? Moving to a new context is always a relearning exercise in my experience.

My understanding is it was standardised for Posix but the variants in popular use have so many variations. I consider sed to be the baseline. If you can do sed you can do anything but it’s seriously limited.

POSIX specifies two flavours of regular expressions: basic regular expressions (BRE) and extended regular expressions (ERE). There are subtle differences between the two and ERE supports more features than BRE. For example, what is written as a$bc$\{3\}d in BRE is written as a(bc){3}d in ERE. See https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... for more details.

The regular expression engines available in most mainstream languages go well beyond what is specified in POSIX though. An interesting example is named capturing group in Python, e.g., (?Pf[o]+).