Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Regex character "$" doesn't mean "end-of-string"

sethmlarson.dev

Post reply on HN
Print view View original

71–80 of 382 posts

Re: Regex character "$" doesn't mean "end-of-string"

#71

Post by rusk »

psd1 wrote: post #51
rusk wrote: post #14

Earlier quoted context omitted.

My understanding is it was standardised for Posix but the variants in popular use have so many variations. I consider sed to be the baseline. If you can do sed you can do anything but it’s seriously limited.

No gnu tool can balance brackets, afaics. So you can't do everything in sed. And sed is, by design, useless for matching text that spans lines, so good luck picking out paragraphs with it.

Sorry I meant to write “if you can do it in sed you can do it in anything” thereby implying it is a subset of the more generally available flavours. The issue at hand however is that there isn’t much in the way of standardisation but 95% of sed should work across all of them. Of course you should get more into the specifics of whatever your solution space supports.

Re: Regex character "$" doesn't mean "end-of-string"

#72

Post by wodenokoto »

> So if you're trying to match a string without a newline at the end, you can't only use $ in Python! My expectation was having multiline mode disabled wouldn't have had this newline-matching behavior, but that isn't the case.

A reproducible example would be nice. I don’t understand what it is he cannot do. `re.search('$', 'no new lines')` returns a match.

Re: Regex character "$" doesn't mean "end-of-string"

#73

Post by divingdragon »

skrebbel wrote: post #31
pjc50 wrote: post #21

Special misery case: Visual Studio supports regex search, where '$' matches \n. The end of line character is usually the standard Windows \r\n. Yes, that means if you want to really match the end of line you have to match "\r$". So broken.

FWIW, and I know this doesn't really address your complaint: I use Windows and I've set all my text editors to use LF exclusively years ago and Things Are Great. No more weird Git autocrlf warnings, no quirks when copying files over to/from people on Macs or Linuxes, etc. Even Notepad supports LF line endings for quite a long time now - to my practical experience, there's little remaining in Windows that makes CRLF "…

> Even CMD.EXE batch files support LF.

I don't know if it is the case on Windows 11, but I have surely been bitten by CMD batch files using LF line endings. I don't remember the exact issue but it may have been the one bug affecting labels. [1]

[1]: https://www.dostips.com/forum/viewtopic.php?t=8988#p58888

Re: Regex character "$" doesn't mean "end-of-string"

#74

Post by ykonstant »

username_my1 wrote: post #61
hoc wrote: post #58

Earlier quoted context omitted.

"I will hire you anyway, but I will pay you less" Regex, useful in any job...

regex is useful but chatgpt is amazing at it, so why spend a minute keeping such useless knowledge in mind. if you know where to find something no point in knowing it.

Does gpt produce efficient regex? Are there any experts here that can assess the quality and correctness of gpt-generated regex? I wonder how regex responses by gpt are validated if the prompter does not have the knowledge to read the output.

Re: Regex character "$" doesn't mean "end-of-string"

#75

Post by rusk »

tankenmate wrote: post #35
susam wrote: post #20

Earlier quoted context omitted.

POSIX specifies two flavours of regular expressions: basic regular expressions (BRE) and extended regular expressions (ERE). There are subtle differences between the two and ERE supports more features than BRE. For example, what is written as a\(bc\)\{3\}d in BRE is written as a(bc){3}d in ERE. See https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1... for more details. The regular expression engines availab…

Indeed, and the most common is Perl since it was the source of many of the extensions.

I would hazard that nowadays it’s Java due to its broad permeation of the application space

Re: Regex character "$" doesn't mean "end-of-string"

#76

Post by fauigerzigerk »

danbruc wrote: post #49
IshKebab wrote: post #43

Earlier quoted context omitted.

It's possible to get design decisions wrong. Clearly people expect `$` to only match end-of-string so they did make the wrong decision. It may not have been clear it was the wrong decision at the time.

Things are obviously more complicated than that, lines are a complicated issue for historical reasons. There are two conventions, line termination and line separation. In case of line termination, the newline is part of the line and a string without a newline is not a [complete] line. In case of line separation, the newline is not part of the line but separates two lines. Also the way newlines are encoded is not univ…

Why is this relevant when multi-line is disabled?

Re: Regex character "$" doesn't mean "end-of-string"

#77

Post by ghusbands »

ghusbands wrote: post #18

> Note: The table of data was gathered from regex101.com, I didn't test using the actual runtimes. Has anyone confirmed this behaviour directly against the runtimes/languages? Newlines at the end of a string are certainly something that could get lost in transit inside an online service involving multiple runtimes.

I've now tested C#, directly, and got the same result as the article. It also documents the behavior:

> The ^ and $ language elements indicate the beginning and end of the input string. The end of the input string can be a trailing newline \n character.

Re: Regex character "$" doesn't mean "end-of-string"

#78

Post by dfawcus »

IshKebab wrote: post #43
danbruc wrote: post #30

Earlier quoted context omitted.

I don't think it is correct to say some get it right and some get it wrong, it is more of an design decision.

It's possible to get design decisions wrong. Clearly people expect `$` to only match end-of-string so they did make the wrong decision. It may not have been clear it was the wrong decision at the time.

Given that in unix they sort started as:

    ed -> sed
    ed -> grep
The line oriented mature makes sense.

There is some sed multi-line capability if one uses the hold space, but it is much easier to just use awk.

Re: Regex character "$" doesn't mean "end-of-string"

#79

Post by ykonstant »

psd1 wrote: post #51
rusk wrote: post #14

Earlier quoted context omitted.

My understanding is it was standardised for Posix but the variants in popular use have so many variations. I consider sed to be the baseline. If you can do sed you can do anything but it’s seriously limited.

No gnu tool can balance brackets, afaics. So you can't do everything in sed. And sed is, by design, useless for matching text that spans lines, so good luck picking out paragraphs with it.

I am pretty sure even pure Awk can do it; or am I mistaken? I thought there was an even more sophisticated example in the Awk book.

Edit: oh, you mean via regex engines available in GNU tools; I am dumb. Hmm... is there no GNU extension with PCRE?

Re: Regex character "$" doesn't mean "end-of-string"

#80

Post by iainmerrick »

wodenokoto wrote: post #72

> So if you're trying to match a string without a newline at the end, you can't only use $ in Python! My expectation was having multiline mode disabled wouldn't have had this newline-matching behavior, but that isn't the case. A reproducible example would be nice. I don’t understand what it is he cannot do. `re.search('$', 'no new lines')` returns a match.

This unexpectedly matches:

re.match('^bob$', 'bob\n')

I didn't want the trailing newline to be included.

Post reply on HN