Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Regex character "$" doesn't mean "end-of-string"

sethmlarson.dev

Post reply on HN
Print view View original

41–50 of 382 posts

Re: Regex character "$" doesn't mean "end-of-string"

#41

Post by jpgvm »

jasonjayr wrote: post #33
beardyw wrote: post #12

Does anyone consider RegEx to be standardised? Moving to a new context is always a relearning exercise in my experience.

The three big ones I know of are POSIX, Perl/PCRE(aka Perl-Compatible Regular Expression), and Go came along and added used re2, which is a bit different from the first too. A lot of systems implemented PCRE, including JavaScript, since Perl extended the POSIX system with many useful extensions. IIRC, re2 tries to reign in on some of the performance issues and quirks the original systems had, while implementing the w…

re2 predates Go and was written in C++.

Re: Regex character "$" doesn't mean "end-of-string"

#42

Post by PuffinBlue »

This seems like the perfect opportunity to introduce those unfamiliar to Robert Elder. He makes cool YouTube[0] and blog content[1] and has a series on regular expressions[2] and does some quite deep dives into the differing behaviour of the different tools that implement the various versions.

His latest on the topic is cool too: https://www.youtube.com/watch?v=ys7yUyyQA-Y

He's has quite a lot of content that HN folks might be interested in I think, like the reality and woes of consulting[3]

[0] https://www.youtube.com/@RobertElderSoftware

[1] https://blog.robertelder.org/

[2] https://blog.robertelder.org/regular-expressions/

[3] https://www.youtube.com/watch?v=cK87ktENPrI

Re: Regex character "$" doesn't mean "end-of-string"

#43

Post by IshKebab »

danbruc wrote: post #30
IshKebab wrote: post #9

Earlier quoted context omitted.

Yes and every implementation gets that right. The point was when multi-line matching is disabled and only Javascript, Go and Rust get that right. I'm not too surprised by PHP and Python getting it wrong. Java and C# is a slight surprise though.

I don't think it is correct to say some get it right and some get it wrong, it is more of an design decision.

It's possible to get design decisions wrong. Clearly people expect `$` to only match end-of-string so they did make the wrong decision. It may not have been clear it was the wrong decision at the time.

Re: Regex character "$" doesn't mean "end-of-string"

#45

Post by jamesmunns »

Karellen wrote: post #28

> Folks who've worked with regular expressions before might know about ^ meaning "start-of-string" and correspondingly see $ as "end-of-string". Huh. I always think of them as "start-of-line" and "end-of-line". I mean, a lot of the time when I'm working with regexes, I'm working with text a line at a time so the effect is the same, but that doesn't change how I think of those operators. Maybe because a fair amount of…

Same, tho it'd be interesting to see if this behavior holds if the file ends without a trailing newline and your match is on the final newline-less line.

Re: Regex character "$" doesn't mean "end-of-string"

#46

Post by tankenmate »

danbruc wrote: post #30
IshKebab wrote: post #9

Earlier quoted context omitted.

Yes and every implementation gets that right. The point was when multi-line matching is disabled and only Javascript, Go and Rust get that right. I'm not too surprised by PHP and Python getting it wrong. Java and C# is a slight surprise though.

I don't think it is correct to say some get it right and some get it wrong, it is more of an design decision.

Not quite, there are standards for this behaviour (formal and de jure).

Re: Regex character "$" doesn't mean "end-of-string"

#47

Post by aquariusDue »

PuffinBlue wrote: post #42

This seems like the perfect opportunity to introduce those unfamiliar to Robert Elder. He makes cool YouTube[0] and blog content[1] and has a series on regular expressions[2] and does some quite deep dives into the differing behaviour of the different tools that implement the various versions. His latest on the topic is cool too: https://www.youtube.com/watch?v=ys7yUyyQA-Y He's has quite a lot of content that HN folk…

I'm glad to see someone else that has stumbled over his content. Seconding the recommendation.

Re: Regex character "$" doesn't mean "end-of-string"

#48

Post by jwilk »

susam wrote: post #20
rusk wrote: post #14

Earlier quoted context omitted.

My understanding is it was standardised for Posix but the variants in popular use have so many variations. I consider sed to be the baseline. If you can do sed you can do anything but it’s seriously limited.

POSIX specifies two flavours of regular expressions: basic regular expressions (BRE) and extended regular expressions (ERE). There are subtle differences between the two and ERE supports more features than BRE. For example, what is written as a\(bc\)\{3\}d in BRE is written as a(bc){3}d in ERE. See https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... for more details. The regular expression engines availab…

> what is written as \(f..\)\1 in BRE is written as (f..)\1 in ERE

Oddly, there are no backreferences in POSIX EREs.

Re: Regex character "$" doesn't mean "end-of-string"

#49

Post by danbruc »

IshKebab wrote: post #43
danbruc wrote: post #30

Earlier quoted context omitted.

I don't think it is correct to say some get it right and some get it wrong, it is more of an design decision.

It's possible to get design decisions wrong. Clearly people expect `$` to only match end-of-string so they did make the wrong decision. It may not have been clear it was the wrong decision at the time.

Things are obviously more complicated than that, lines are a complicated issue for historical reasons. There are two conventions, line termination and line separation. In case of line termination, the newline is part of the line and a string without a newline is not a [complete] line. In case of line separation, the newline is not part of the line but separates two lines. Also the way newlines are encoded is not universal.

Re: Regex character "$" doesn't mean "end-of-string"

#50

Post by tankenmate »

somat wrote: post #23
wolletd wrote: post #11

Earlier quoted context omitted.

The differences of the various regex "dialects" came to me over the years of using regular expressions for all kinds of stuff. Matching EOL feels natural for every line-based process. What I find way more annoying is escaping characters and writing character groups. Why can't all regex engines support '\d' and '\w' and such? Why, in sed, is an unescaped '.' a regex-dot matching any character, but an unescaped '(' is…

> Why, in sed, is an unescaped '.' a regex-dot matching any character, but an unescaped '(' is just a regular bracket? It is because sed predates the very influential second generation Extended Regular Expression engine and by default uses the first generation Basic Regular Expression engine. So really it is for backwards compatibility. http://man.openbsd.org/re_format#BASIC_REGULAR_EXPRESSIONS you can usually pass s…

BRE and ERE was created at the same time. Prior to this there wasn't a clear standard for Regex. From my memory this was standardised in 1996 (IEEE Std 1003.1-1996).

The work originally came from work by Stephen Cole Kleene in the 1950s. It was introduced into Unix fame via the QED editor (which later became ed (and sed), then ex, then vi, then vim; all with differing authors) when Ken Thompson added regex when he ported QED to CTSS (an OS developed at MIT for the IBM 709, which was later used to develop Multics, and hence lead to Unix).

Also the "grep" command got its name from "ed"; "g" (the global ed command) "re" (regular expression), and "p" (the print ed command). Try it in vi/vim, :g/string/p it is the same thing as the grep command.

Post reply on HN