Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Regex character "$" doesn't mean "end-of-string"

sethmlarson.dev

Post reply on HN
Print view View original

361–370 of 382 posts

Re: Regex character "$" doesn't mean "end-of-string"

#361

Post by wtetzner »

coryrc wrote: post #241
rovr138 wrote: post #165

Earlier quoted context omitted.

> Yes, that is a file with zero lines that ends with an "incomplete line". It's a file with zero complete lines. But it has 1 line, that's incomplete, right? The file starts empty. Anything in it starts "a line". So it's 1 incomplete line. I hate weird states.

Pedantically, if it doesn't end with a newline, it's considered a binary file and not a text file. Binary files don't have lines. In practice, most utilities expecting text files will still operate on it.

That's a weird way to look at it. Binary files might not have "lines", but there's no reason they couldn't include a byte with value 10 (the ASCII value for \n). Software reading that file wouldn't know the difference, right?

Also, why couldn't you have a text file without any lines?

Re: Regex character "$" doesn't mean "end-of-string"

#362

Post by wtetzner »

simonh wrote: post #247
rerdavies wrote: post #220

Earlier quoted context omitted.

The opengroup spec says no such thing.

3.206 Line A sequence of zero or more non- characters plus a terminating character. See also ‘3.403 Text File’ for the definition of a text file. No new line characters, no lines. No lines, not a text file.

> No lines, not a text file.

That seems like a broken (maybe just bad?) definition/specification to me. A blob of JSON in a file isn't "text" if there's no newline character trailing it?

Re: Regex character "$" doesn't mean "end-of-string"

#363

Post by wtetzner »

LK5ZJwMwgBbHuVI wrote: post #269
wtetzner wrote: post #152

Earlier quoted context omitted.

So if you have "A" in a file with no newline, there are no lines in that file?

Why don't you go ask? $ echo -n foo | wc -l 0

wc just counts newline characters. I'm not sure why it would be the ultimate authority on anything.

Re: Regex character "$" doesn't mean "end-of-string"

#364

Post by Karellen »

keybored wrote: post #331
LK5ZJwMwgBbHuVI wrote: post #275

Earlier quoted context omitted.

Yes, languages really need some sort of "raw string" feature like Python (or make regex literals their own syntax like Perl does). That's the solution here, not using weird characters...

Fine enough. But I wonder why strings have to use the same delimiter. Imagine if you had a list delimiter `|` and the answer to nested lists was “ohh, use raw list syntax, just make `###||` when you are three levels deep or something”.

It is quite nice what `sed` does. A sed search-and-replace is typically shown as `s/foo/bar/`, but you can actually use any punctuation character to separate the parts. Whatever follows the "s" will be used for that statement, so you can write `s|foo|bar|` or `s:foo:bar:`, even mixing and matching in the same script to have `s|baz|quux|; s:xyzzy:blorp:` and it will all work.

Re: Regex character "$" doesn't mean "end-of-string"

#365

Post by lsaferite »

nomel wrote: post #255
cortesoft wrote: post #227

Earlier quoted context omitted.

I mean, the person you are responding to didn't invent the definition out of thin air... the POSIX standard did: 3.206 Line A sequence of zero or more non- characters plus a terminating character. https://pubs.opengroup.org/onlinepubs/9699919799.2018edition...

Posix getline() includes EOF as a line terminator: getline() reads an entire line from stream, storing the address of the buffer containing the text into *lineptr. The buffer is null-terminated and includes the newline character, if one was found. ... ... a delimiter character is not added if one was not present in the input before end of file was reached. EOF seems same as end-of-string.

Your quoted documentation says otherwise. It says that a 'line' include the delimiter, '\n', in the line buffer. It also says that is no delimiter is found before the EOF is reached that the line buffer will not include the delimiter. That means the line buffer can clearly indicate an incomplete line by the absence of the delimiter. To be clear, EOF isn't a 'line terminator', it's the end of the data stream.

Re: Regex character "$" doesn't mean "end-of-string"

#366

Post by nomel »

lsaferite wrote: post #365
nomel wrote: post #255

Earlier quoted context omitted.

Posix getline() includes EOF as a line terminator: getline() reads an entire line from stream, storing the address of the buffer containing the text into *lineptr. The buffer is null-terminated and includes the newline character, if one was found. ... ... a delimiter character is not added if one was not present in the input before end of file was reached. EOF seems same as end-of-string.

Your quoted documentation says otherwise. It says that a 'line' include the delimiter, '\n', in the line buffer. It also says that is no delimiter is found before the EOF is reached that the line buffer will not include the delimiter. That means the line buffer can clearly indicate an incomplete line by the absence of the delimiter. To be clear, EOF isn't a 'line terminator', it's the end of the data stream.

Yes, "EOF seems same as end-of-string."

Re: Regex character "$" doesn't mean "end-of-string"

#367

Post by coryrc »

wtetzner wrote: post #361
coryrc wrote: post #241

Earlier quoted context omitted.

Pedantically, if it doesn't end with a newline, it's considered a binary file and not a text file. Binary files don't have lines. In practice, most utilities expecting text files will still operate on it.

That's a weird way to look at it. Binary files might not have "lines", but there's no reason they couldn't include a byte with value 10 (the ASCII value for \n). Software reading that file wouldn't know the difference, right? Also, why couldn't you have a text file without any lines?

All I'm addressing is GP's comment:

    It's a file with zero complete lines. But it has 1 line, that's incomplete, right?
Because the Unix definition of text file requires the file to end with a newline. "Lines" only exist in the context of text files. If there's no terminating newline, it's (pedantically) not a text file and so has no lines. Now, in practice, if you open() that file in text mode, it doesn't TMK return an error if the terminating newline isn't present, but it's undefined behaviour.

And if you do have a terminating newline, then you have at least one line :).

Re: Regex character "$" doesn't mean "end-of-string"

#368

Post by pxc »

rovr138 wrote: post #165
jepler wrote: post #157

Earlier quoted context omitted.

Yes, that is a file with zero lines that ends with an "incomplete line". Processing of such files by standard line-oriented utilities is undefined in the opengroup spec. So, for instance, the effect of "grep"ping such a file is not defined. Heck, even "cat"ting such a file gives non-ideal results, such as colliding with the regular shell prompt. For this reason, a lot of software projects I work on check and correct…

> Yes, that is a file with zero lines that ends with an "incomplete line". It's a file with zero complete lines. But it has 1 line, that's incomplete, right? The file starts empty. Anything in it starts "a line". So it's 1 incomplete line. I hate weird states.

Here's another way to think about this:

This isn't a weird state. It's a language problem. An 'incomplete line' isn't a type of line, it's an unfortunate name for a thing that is not a line. Just like how the 'wor' is an incomplete word (the word 'word'), but 'wor' is, of course, not a word.

Same thing for formalisms like equations in algebra or formulas in propositional logic— we have the phrase 'well-formed formula', and we might describe some sequences of terms as 'incomplete formulas' or perhaps 'ill-formed formulas', but those phrases don't describe anything that meets the formal system's definition of 'formula' at all— they are not formulas. 'Ill-formed formula' is not a compositional phrase where 'ill-formed' describes a feature of a 'formula'. It's a bit of convenient language for what we can intuitively or metaphorically recognize as a formula-ish thing.

Re: Regex character "$" doesn't mean "end-of-string"

#369

Post by simonh »

wtetzner wrote: post #362
simonh wrote: post #247

Earlier quoted context omitted.

3.206 Line A sequence of zero or more non- characters plus a terminating character. See also ‘3.403 Text File’ for the definition of a text file. No new line characters, no lines. No lines, not a text file.

> No lines, not a text file. That seems like a broken (maybe just bad?) definition/specification to me. A blob of JSON in a file isn't "text" if there's no newline character trailing it?

There are other definitions of a text file than the opengroup spec, particularly for specific OS platforms. I’m not sure what convention JSON follows.

As a spec it’s fine. It defines a text file in such a way that you can easily write code to process such a file deterministicaly.

Re: Regex character "$" doesn't mean "end-of-string"

#370

Post by deaddodo »

int_19h wrote: post #311
deaddodo wrote: post #236

Earlier quoted context omitted.

The "Windows way" is the "right way" for a few reasons. This is definitely not one of them.

Which are the valid reasons, legacy meanings of those characters aside?

I mean, it was what everyone had agreed upon previously. Microsoft was the only party to follow through. For all the guff they get for not following standards, it was the one standard they did.

You don't have to love a company to acknowledge they did something right.

Post reply on HN