Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

EOF is not a character

ruslanspivak.com

Post reply on HN
Print view View original

51–60 of 136 posts

Re: EOF is not a character

#51

Post by schoen »

anonymousiam wrote: post #16

CP/M and DOS use ^Z (0x1A) as an EOF indicator. More modern operating systems use the file length (if available). Unix/Linux will treat ^D (0x04) as EOF within a stream, but only if the source is "cooked" and not "raw". (^D is ASCII "End Of Transmission or EOT" so that seems appropriate, except in the world of unicode.)

Strictly speaking, as discussed elsewhere in this thread, ^D can cause a terminal device to signal an EOF condition; other kinds of Unix byte streams don't make this association.

For example,

  $ python3 -c 'print("".join(chr(c) for c in range(10)))' | python3 -c 'print(list(ord(c) for c in input()))'
will confirm that it doesn't happen in a pipe (the ASCII 4 character there is totally unrelated to EOF).

Re: EOF is not a character

#52

Post by jcrawfordor »

mark-r wrote: post #40
jcrawfordor wrote: post #13

Earlier quoted context omitted.

Perhaps a marginally better title would be "EOF is not a character [on Unix]". There are some OS that have an explicit EOF character, but it seems to have been the less common approach historically. CP/M featured an explicit end of file marker because the file system didn't bother to handle the problem of files which were not block-aligned, so the application layer needed to detect where the actual end of the file wa…

I think CP/M copied that convention from an even older OS but I can't remember which one.

CP/M was developed on TOPS-10 and copied a lot of concepts from it. I can't immediately tell whether or not this is an example, but for any given eccentricity of CP/M it's a good bet that it came from TOPS-10.

It's amusing that almost the same can be said about NT: for any given eccentricity of Windows NT it's a good bet that it came from VMS, since the two had the same principal designer.

Re: EOF is not a character

#54

Post by mikekchar »

nixpulvis wrote: post #7
reidacdc wrote: post #5

Seems like the confusion arises because getchar() (or its equivalent in langauges other than c) can produce an out-of-band result, EOF, which is not a character. Procedural programmers don't generally have a problem with this -- getchar() returns an int, after all, so of course it can return non-characters, and did you know that IEEE-754 floating point can represent a "negative zero" that you can use for an error cod…

What does "Procedural" vs "Functional" have to do with this? It's a choice in data type. If by procedural you mean, nonsense, then sure... I agree that a function named `getchar` returning an `int` is procedural. :P

I suspect it was a product of the OP's musing about errors. Side effects are common in programming languages outside of pure functional languages. When you have a pure functional language, what do you do if the type you are returning can't represent an error? You also can't have side effects (for example throw an exception), so it's doubly important that you make sure your return type can encode errors. I suspect that's all they meant. The choice of wording was just unfortunate (especially the use of "procedural" -- what do I do if I can't return values??? ;-) ).

Re: EOF is not a character

#55

Post by saagarjha »

snek wrote: post #33
reidacdc wrote: post #5

Seems like the confusion arises because getchar() (or its equivalent in langauges other than c) can produce an out-of-band result, EOF, which is not a character. Procedural programmers don't generally have a problem with this -- getchar() returns an int, after all, so of course it can return non-characters, and did you know that IEEE-754 floating point can represent a "negative zero" that you can use for an error cod…

> and did you know that IEEE-754 floating point can represent a "negative zero" that you can use for an error code in functions that return float or double? I am begging, please never ever do this. NaN literally exists for this reason. NaN even allows you to encode additional error context and details into the value.

+DBL_MAX. Negative zero is an entirely valid, if rare, result of certain computations.

Re: EOF is not a character

#56

Post by samatman »

nixpulvis wrote: post #30
samatman wrote: post #26

Earlier quoted context omitted.

It's not a great snark given that the C standard considers the signedness of char to be implementation defined, making -1 a valid option, sometimes.

I'm sorry you don't find it great (I still do). Integers are not characters. Integers are numbers like -1337, 0, and 42. Characters are things that compose strings of text. These are not the same kind of thing at all. Just because APIs may be leaky, and some of these APIs are held in very high regard doesn't change that fact.

a char isn't a character, though. you can't add two characters together and get another character. it's a number.

getchar() gets a char. not a character.

Re: EOF is not a character

#57

Post by pwdisswordfish2 »

wongarsu wrote: post #21
fennecfoxen wrote: post #12

Earlier quoted context omitted.

I taught myself C on MS-DOS in middle school, decades ago. Could have sworn that ASCII 26 was named “EOF” even if modern text files don’t include it. This is a supplementary source of confusion.

Wikipedia supports this: > Character 26 was used to mark "End of file" even if the ASCII calls it Substitute, and has other characters for this. Number 28 which is called "File Separator" has also been used for similar purposes. [1] I think today we would think of character 4 (End of Transmission, Ctrl-D) as the end of file/input marker, but historically Character 26/Ctrl-Z was used, even on disk. 1: https://en.wikip…

See, this is why you should not believe Wikipedia.

The DOS syscall interface has no concept of an EOF character. ^Z being considered EOF was a feature of the COPY command, later replicated by the runtimes of various languages targetting DOS.

http://jdebp.info/FGA/dos-character-26-is-not-special.html

Re: EOF is not a character

#58

Post by pwdisswordfish2 »

saagarjha wrote: post #55
snek wrote: post #33

Earlier quoted context omitted.

> and did you know that IEEE-754 floating point can represent a "negative zero" that you can use for an error code in functions that return float or double? I am begging, please never ever do this. NaN literally exists for this reason. NaN even allows you to encode additional error context and details into the value.

+DBL_MAX. Negative zero is an entirely valid, if rare, result of certain computations.

IEEE 754 has infinities as well, no need to constrain yourself to DBL_MAX :)

Re: EOF is not a character

#59

Post by pwdisswordfish2 »

anonymousiam wrote: post #16

CP/M and DOS use ^Z (0x1A) as an EOF indicator. More modern operating systems use the file length (if available). Unix/Linux will treat ^D (0x04) as EOF within a stream, but only if the source is "cooked" and not "raw". (^D is ASCII "End Of Transmission or EOT" so that seems appropriate, except in the world of unicode.)

That is a common misconception.

http://jdebp.info/FGA/dos-character-26-is-not-special.html

Re: EOF is not a character

#60

Post by pwdisswordfish2 »

kylek wrote: post #42
_8j50 wrote: post #36

Banged my head against the wall once after trying to figure out why Ctrl+D generates some character in bash but I can't send that character in a pipe to simulate termination.

Fun fact, ctrl-v in bash sets "verbatim insert" mode for the next character, so you can type a ^D "character" by doing "ctrl-v ctrl-d".

It’s not bash, it’s the tty device driver. Applications can switch between the ‘cooked’ mode (which recognises it as EOF) and ‘raw’ mode (which passes it through) by performing some ioctl I don’t really want to look up right now.
Post reply on HN