Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Regex character "$" doesn't mean "end-of-string"

sethmlarson.dev

Post reply on HN
Print view View original

221–230 of 382 posts

Re: Regex character "$" doesn't mean "end-of-string"

#221

Post by pksebben »

berkes wrote: post #95
username_my1 wrote: post #61

Earlier quoted context omitted.

regex is useful but chatgpt is amazing at it, so why spend a minute keeping such useless knowledge in mind. if you know where to find something no point in knowing it.

> if you know where to find something no point in knowing it. Nonsense. And you know it. First, you need to know what to find, before knowing where to find it. And knowing what to find requires intricate knowledge of the thing. Not intricate implementation details, but enough to point yourself in the right direction. Secondly, you need to know why to find thing X and not thing Y. If anything, ChatGPT is even worse th…

While I agree with your point that knowing things matters, it is entirely possible with the current batch of LLMs to get to an answer you don't know much about. It's actually one of the few things they do reliably well.

You start with what you do know, asking leading questions and being clear about what you don't, and you build towards deeper and deeper terminology until you get to the point where there are docs to read (because you still can't trust them to get the specifics right).

I've done this on a number of projects with pretty astonishing results, building stuff that would otherwise be completely out of my wheelhouse.

Re: Regex character "$" doesn't mean "end-of-string"

#222

Post by DougBTX »

rovr138 wrote: post #165
jepler wrote: post #157

Earlier quoted context omitted.

Yes, that is a file with zero lines that ends with an "incomplete line". Processing of such files by standard line-oriented utilities is undefined in the opengroup spec. So, for instance, the effect of "grep"ping such a file is not defined. Heck, even "cat"ting such a file gives non-ideal results, such as colliding with the regular shell prompt. For this reason, a lot of software projects I work on check and correct…

> Yes, that is a file with zero lines that ends with an "incomplete line". It's a file with zero complete lines. But it has 1 line, that's incomplete, right? The file starts empty. Anything in it starts "a line". So it's 1 incomplete line. I hate weird states.

Another way to look at it is that concatenating files should sum the line count. Concatenating two empty files produces an empty file, so 0 + 0 = 0. If “incomplete lines” are not counted as lines, then the maths still works out. If they counted as lines, it would end up as 1 + 1 = 1.

Re: Regex character "$" doesn't mean "end-of-string"

#224

Post by amelius »

Yujf wrote: post #198
amelius wrote: post #194

Earlier quoted context omitted.

What is driving me nuts is that we have Unicode now, so there is no need to use common characters like $ or ^ to denote special regex state transitions.

Why not? Common characters are easier to type and presumbly if you are using regex on a unicode string they might include these special characters anyway so what have you gained?

In theory yes, in practice no.

What you have gained is that the regex is now much easier to read.

Re: Regex character "$" doesn't mean "end-of-string"

#225

Post by bregma »

account42 wrote: post #133
bregma wrote: post #70

Earlier quoted context omitted.

The ISO/IEC 14882 C++ standard library mandates [0] implementations for six de jure standard regex grammars: IEEE Std 1003.1-2008 (POSIX) [1] BRE, ERE, awk, grep, and egrep and ECMA-262 EcmaScript 3 [2]. So, yes, at least someone (me) considers regex to be standardized in several published de jure standards. [0] https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3690.pdf#chapter.28 [1] https://pubs.opengroup.o…

is not exactly an example anyone should follow.

You may be prejudiced against C++, but ISO/IEC 14882 is a published international standard that links to recognized regex standards, so answers the question "does anyone consider RegEx standardised?" very much in the affirmative.

Re: Regex character "$" doesn't mean "end-of-string"

#226

Post by jfhufl »

jewel wrote: post #174

This has security implications! Example exploitable ruby code: unless person_id =~ /^\d+$/ abort "Bad person ID" end sql = "select * from people where person_id = #{person_id}" In addition to injection attacks, this also can bite people when parsing headers, where a bad header is allowed to sneak past a filter.

Unsure what you mean?

    $ ruby -e 'x = "25" ; if x =~ /^\d+$/ ; puts "yes" ; else ; puts "no" ; end'
    yes
    $ ruby -e 'x = "25\n" ; if x =~ /^\d+$/ ; puts "yes" ; else ; puts "no" ; end' 
    yes
    $ ruby -e 'x = "a25\n" ; if x =~ /^\d+$/ ; puts "yes" ; else ; puts "no" ; end'
    no
Also, you'd want to use something that parameterizes the query with '?' (I use the Sequel gem) instead of just stuffing it into a sql string.

Re: Regex character "$" doesn't mean "end-of-string"

#227

Post by cortesoft »

rerdavies wrote: post #217
akdev1l wrote: post #142

Earlier quoted context omitted.

Technically the “newline” character is actually a line _terminator_. Hence “A\n” is one line, not two. The “\n” is always at the end of a line by definition.

Technically, that is one of two possible interpretations, and you seem to have invented a "by definition" out of thin air. Very very technically a "newline" character indicates the start of a new line, which is why it is not called the "end-of-line" character.

I mean, the person you are responding to didn't invent the definition out of thin air... the POSIX standard did:

3.206 Line A sequence of zero or more non- characters plus a terminating character.

https://pubs.opengroup.org/onlinepubs/9699919799.2018edition...

Re: Regex character "$" doesn't mean "end-of-string"

#228

Post by TillE »

toyg wrote: post #138
tyingq wrote: post #112

Seems odd to leave Perl off the list, given it's regex related. Here's the explanation for $ in the perlre docs: $ Match the end of the string (or before newline at the end of the string; or before any newline if /m is used)

Yeah, omitting what is arguably the language most associated with regexes seems a bit of an oversight. I guess it shows how far off the radar Perl currently is.

PHP uses PCRE, so it more or less serves as a stand-in for Perl in this case.

Re: Regex character "$" doesn't mean "end-of-string"

#229

Post by danbruc »

burntsushi wrote: post #182
danbruc wrote: post #158

Earlier quoted context omitted.

But that is exactly what it means, the end of the line is before the terminating newline or at the end of the string if there is no terminating newline. Both ^ and $ always match at start or end of lines, \A and \Z match at the start or end of the string. The difference between multi-line and not is whether or not internal newlines end and start lines, it does not change the semantics from end of line to end of strin…

re.search does not accept a "line." It accepts a "string." There is no pretext in which re.search is meant to only accept a single line. And giving it a `string` with multiple new lines doesn't necessarily mean you want to enable multi-line mode. They are orthogonal things. > Both ^ and $ always match at start or end of lines This is trivially not true, as I showed in my previous example. The haystack `cat\n\n` conta…

re.search does not accept a "line." It accepts a "string." There is no pretext in which re.search is meant to only accept a single line.

Sure, it takes a string which might be a line or multiple or whatever. Does not change the fact that $ matches at the end of a line. If you want the end of the string, use \Z.

This is trivially not true, as I showed in my previous example. The haystack `cat\n\n` contains two lines and the regex `cat$` says it should match `cat` followed by the "end of a line" according to your definition.

In multi-line mode it matches, in single-line mode it does not because there is a newline between cat and the end of the line. A newline is only a terminating newline if it is the last character, the newline after cat is not a terminating newline. You need cat\n$ or cat\n\n to match.

Re: Regex character "$" doesn't mean "end-of-string"

#230

Post by cortesoft »

FranOntanaya wrote: post #210
knome wrote: post #196

Earlier quoted context omitted.

the idea of changing a decades old convention to instead use, as I assume you are implying, some character that requires special entry, is beyond silly.

I don't think anyone that writes regex would feel specially challenged by using the Alt+ | Ctrl+Shift+u key combos for unicode entry. Having to escape less things in a pattern would be nice.

I write regexes all the time, and I don't know if I would be CHALLENGED by that, but it would be annoying. Escaping things is trivial, and since you do it all the time it is not anything extra to learn. Having to remember bespoke keystrokes for each character is a lot more to learn.
Post reply on HN