Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Regex character "$" doesn't mean "end-of-string"

sethmlarson.dev

Post reply on HN
Print view View original

271–280 of 382 posts

Re: Regex character "$" doesn't mean "end-of-string"

#271

Post by Izkata »

michaelt wrote: post #267
Izkata wrote: post #188

Earlier quoted context omitted.

Same here; when I saw the title I was like "well obviously not, where did you hear that?" In nearly two decades of using regex I think this might be the first time I've heard of $ being end of string. It's always been end of line for me.

Take a look at, for example, these stackoverflow answers about a regex to validate and e-mail address: https://stackoverflow.com/a/8829363 These people are I think not intending to say a newline character is permitted at the end of an e-mail address. (Of course people using 'grep' would have different expectations for obvious reasons)

Even disregarding whether or not end-of-string is also an end-of-line or not (see all the other comments below), $ doesn't match the newline, similar to zero-width matches like \b, so the newline wouldn't be included in the matched text either way.

I think this series of comments might be clearest: https://news.ycombinator.com/item?id=39764385

Re: Regex character "$" doesn't mean "end-of-string"

#272

Post by LK5ZJwMwgBbHuVI »

rerdavies wrote: post #217
akdev1l wrote: post #142

Earlier quoted context omitted.

Technically the “newline” character is actually a line _terminator_. Hence “A\n” is one line, not two. The “\n” is always at the end of a line by definition.

Technically, that is one of two possible interpretations, and you seem to have invented a "by definition" out of thin air. Very very technically a "newline" character indicates the start of a new line, which is why it is not called the "end-of-line" character.

It doesn't indicate the start of a new line, or files would start with it. Files end with it, which is why it is a line terminator. And it is by definition: by the standard, by the way cat and/or your shell and/or your terminal work together, and by the way standard utilities like `wc` treat the file.

Re: Regex character "$" doesn't mean "end-of-string"

#273

Post by LK5ZJwMwgBbHuVI »

joshjje wrote: post #253
akdev1l wrote: post #142

Earlier quoted context omitted.

Technically the “newline” character is actually a line _terminator_. Hence “A\n” is one line, not two. The “\n” is always at the end of a line by definition.

“A\n” is two lines.

Factually incorrect.

Re: Regex character "$" doesn't mean "end-of-string"

#274

Post by PaulDavisThe1st »

coryrc wrote: post #241
rovr138 wrote: post #165

Earlier quoted context omitted.

> Yes, that is a file with zero lines that ends with an "incomplete line". It's a file with zero complete lines. But it has 1 line, that's incomplete, right? The file starts empty. Anything in it starts "a line". So it's 1 incomplete line. I hate weird states.

Pedantically, if it doesn't end with a newline, it's considered a binary file and not a text file. Binary files don't have lines. In practice, most utilities expecting text files will still operate on it.

No file has lines.

"Lines" are a convention established by (or not) software reading a data stream.

Re: Regex character "$" doesn't mean "end-of-string"

#275

Post by LK5ZJwMwgBbHuVI »

keybored wrote: post #235
knome wrote: post #196

Earlier quoted context omitted.

the idea of changing a decades old convention to instead use, as I assume you are implying, some character that requires special entry, is beyond silly.

It’s not that silly. You constantly get into escape conundrums because you need to use a metacharacter which is also a metacharacter three levels deep in some embedding. (But that might not solve that problem? Maybe the problem is mostly about using same-character delimiters for strings.) And I guess that’s why Perl is so flexible with regards to delimiters and such.

Yes, languages really need some sort of "raw string" feature like Python (or make regex literals their own syntax like Perl does). That's the solution here, not using weird characters...

Re: Regex character "$" doesn't mean "end-of-string"

#276

Post by LK5ZJwMwgBbHuVI »

amelius wrote: post #224
Yujf wrote: post #198

Earlier quoted context omitted.

Why not? Common characters are easier to type and presumbly if you are using regex on a unicode string they might include these special characters anyway so what have you gained?

In theory yes, in practice no. What you have gained is that the regex is now much easier to read.

> In theory yes, in practice no.

That's like "in theory we need 4 bytes to represent Unicode, but in practice 3 bytes is fine" (glances at universally-maligned utf8mb3)

Re: Regex character "$" doesn't mean "end-of-string"

#277

Post by dumbo-octopus »

layer8 wrote: post #153
KMnO4 wrote: post #150

Earlier quoted context omitted.

I take the descriptivist approach to email validation, rather than the prescriptivist. I know an email has to have a domain name after the @ so I know where to send it. I also know it has to have something before the @ so the domain’s email server knows how to handle it. But do I care if the email server is supports sub addresses, characters outside of the commonly supported range (eg quotation marks and spaces), or…

I generally agree, but the two consecutive dots (or leading/trailing dots) are an example that would very likely be a typo and that you wouldn’t particularly want to send. Similar for unbalanced quotes, angle brackets, and other grammar elements.

I wonder whether simply (regex) replacing a sequence of .'s with a single one as part of a post-processing step would be effective.

Re: Regex character "$" doesn't mean "end-of-string"

#278

Post by danbruc »

masklinn wrote: post #266
danbruc wrote: post #121

Earlier quoted context omitted.

The POSIX definition of a line is a sequence of non-newline characters - possibly zero - followed by a newline. Everything that does not end with a newline is not a [complete] line. So strictly speaking it would even be correct that cat$ does not match cat because there is no terminating newline, it should only match cat\n. But as lines missing a terminating newline is a thing, it seems reasonable to be less strict.

> a line is a sequence of non-newline characters Works for me. How do you square that with your assertion that in your invention of "single-line mode" you implicitly define "line" as matching \n\n?

If you are not in multi-line mode, then a single line is expected and consequently there is at most one newline at the end of the string. You can of course pick an input that violates this, run it against a multi-line string with several newlines in it. cat\n\n will not match cat$ because there is something between cat and the end of the line, it just happens to be a newline but without any special meaning because it is not the last character and you did not say that the input is multi-line.

Re: Regex character "$" doesn't mean "end-of-string"

#279

Post by ydant »

pizzafeelsright wrote: post #204
xlii wrote: post #8

Regexp was one of the first things I truly internalized years ago when I was discovering Perl (which still lives in a cozy place in my heart due to a lovely “Camel” book). Today most important bit of information is knowledge that implementations differ and I made a habit of pulling reference sheet for a thing I work with. E.g. Emacs Regexp annoyingly doesn’t have word in form of “\w” but uses “\s_-“ (or something no…

How did you internalize it? Perl looks like cat keyboarding.

For me, Perl hit me at exactly the right time in my development. One or more of the various O'Reilly Perl books caught my attention in the bookstore, the foreword and the writing style was unlike anything else I'd read in programming up to that point, and I read the book and just felt a strong connection to how the language was structured, the design concepts behind it, the power of regex being built in to the language, etc. The syntax favored easy to write programs without unnecessary scaffolding (of course, leading to the jokes of it being write-only - also the jokes I could make about me programming largely in Java today), and the standard functionality plus the library set available felt like magic to me at that point.

Learning Perl today would be a very different experience. I don't think it would catch me as readily as it did back then. But it doesn't matter - it's embedded into me at a deep level because I learned it through a strong drive of fascination and infatuation.

As for the regex themselves? It's powerful and solved a lot of the problems I was trying to solve, was built fundamentally into Perl as a language, so learning it was just an easy iterative process. It didn't hurt that the particular period of time when I learned Perl/regex the community was really big on "leetcode" style exercises, they just happened to be focused around Perl Golf, being clever in how you wrote solutions to arbitrary problems, and abusive levels of regex to solve problems. We were all playing and play is a great way to learn.

Re: Regex character "$" doesn't mean "end-of-string"

#280

Post by LK5ZJwMwgBbHuVI »

Izkata wrote: post #271
michaelt wrote: post #267

Earlier quoted context omitted.

Take a look at, for example, these stackoverflow answers about a regex to validate and e-mail address: https://stackoverflow.com/a/8829363 These people are I think not intending to say a newline character is permitted at the end of an e-mail address. (Of course people using 'grep' would have different expectations for obvious reasons)

Even disregarding whether or not end-of-string is also an end-of-line or not (see all the other comments below), $ doesn't match the newline, similar to zero-width matches like \b, so the newline wouldn't be included in the matched text either way. I think this series of comments might be clearest: https://news.ycombinator.com/item?id=39764385

Problem is, plenty of software doesn't actually look at the match but rather just validates that there was a match (and then continues to use the input to that match).
Post reply on HN