Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CRLF is obsolete and should be abolished

fossil-scm.org

Post reply on HN
Print view View original

221–230 of 273 posts

Re: CRLF is obsolete and should be abolished

#221

Post by eqvinox »

fijiaarone wrote: post #188
mattmerr wrote: post #35

Earlier quoted context omitted.

ASCII already has designated bytes for unit, group, and record separators. That aside, a big drawback of using unprintable bytes like these is they're more difficult for humans to read in dumps or type on a keyboard than a newline (provided newline has a strict definition CRLF, LF, etc)

There is no reason those ascii characters need to stay unprintable. You could use other characters like an interpunct, silcrow, or down carat.

There is in fact a reason those ASCII 'characters' should stay unprintable: the 0x00-0x1f (except Tab, CR, LF) range is explicitly excluded as invalid in a whole bunch of standards, e.g. XML.

Re: CRLF is obsolete and should be abolished

#222

Post by Joker_vD »

tptacek wrote: post #38
amluto wrote: post #32

Earlier quoted context omitted.

> I'm hoping this is satire. Why intentionally introduce potential bugs for the sake of making a point? It’s worse than satire. Postel’s Law is definitively wrong, at least in the context of network protocols, and delimiters, especially, MUST be precise. See, for example: https://www.postfix.org/smtp-smuggling.html Send exactly what the spec requires, and parse exactly as the spec requires. Do not accept garbage. And…

If two systems agree, independent of any specification someone somewhere else wrote, to accept a bare NL where a CRLF is specified, that is not "garbage". Standards documents are not laws; the horse drags the cart.

This is, by the way, exactly the stance the Microsoft used to have in the 90-s and 00-s on the standards (it probably still has). And MS caught a lot of flak for that, for a very good reason.

Re: CRLF is obsolete and should be abolished

#223

Post by Joker_vD »

romwell wrote: post #128
jfengel wrote: post #113

Earlier quoted context omitted.

LF only? Huh. In its original terms for printing terminals, carriage return might be ambiguous. It could means either "just send the print head to column zero" or "print head to 0 and advance the line by one". The latter is what typewriters do for the Return key. But LF always meant Line Feed, moving the paper but not the print head. These are of course wildly out of date concepts. But it still strikes me as odd to s…

>The latter is what typewriters do for the Return key. Minor correction: mechanical typewriters do not have a Return key, but they have both operations (line feed, as well as carriage return). The carriage return lever is typically rigged to also do line feed at the same time, by a preset amount of lines (which can be set to 0), or you can push the carriage without engaging line feed. Technically, the lever would do…

Naked CR is an almost (baring legacy Mac OS) universally supported cross-platform way to print progress bars on CRT terminals.

Re: CRLF is obsolete and should be abolished

#224

Post by isThereClarity »

chasil wrote: post #12
michaelmior wrote: post #3

> various protocols (HTTP, SMTP, CSV) still "require" CRLF at the end of each line What would be the benefit to updating legacy protocols to just use NL? You save a handful of bits at the expense of a lot of potential bugs. HTTP/1(.1) is mostly replaced by HTTP/2 and later by now anyway. Sure, it makes sense not to require CRLF with any new protocols, but it doesn't seem worth updating legacy things. > Even if an est…

FYI, Sendmail accepts LF without CR, but Exchange doesn't.

sendmail 8.18.1 includes patches to correct this behaviour (and options to turn it back on) due to its role in SMTP smuggling, CVE-2023-51765. See https://ftp.sendmail.org/RELEASE_NOTES

  sendmail is now stricter in following the RFCs and rejects
  some invalid input with respect to line endings
  and pipelining:
  ...snip...
  - Accept only CRLF . CRLF as end of an SMTP message
    as required by the RFCs, which can disabled by the
    new srv_features option 'O'.
  - Do not accept a CR or LF except in the combination
    CRLF (as required by the RFCs).  These checks can
    be disabled by the new srv_features options
   'U' and 'G', respectively.  In this case it is
   suggested to use 'u2' and 'g2' instead so the server
   replaces offending bare CR or bare LF with a space.
   It is recommended to only turn these protections off
   for trusted networks due to the potential for abuse.

Re: CRLF is obsolete and should be abolished

#225

Post by eqvinox »

moomin wrote: post #41

Counterpoint: Unix deciding on a non-standard line ending was always a mistake. It has produced decades of random incompatibility for no particular benefit. CRLF isn’t a convention: it’s two different pieces of the base terminal API. You have no idea how many programs rely on CR and LF working correctly.

Counter-counterpoint: using 2 bytes to signal one relevant operation creates ambiguity out of thin air. If all you care about is "where does the line end?", having CRLF as a line ending creates edge cases for "there is only CR" and "there is only LF". Are those line endings or not? How do you deal with them? And what's LFCR?

Personally speaking, I've always written my parsers to be permissive and accept either CR¹, LF, or CRLF as line endings. And it always meant keeping a little extra boolean for "previous byte was CR" to ignore the LF to not turn CRLF into 2 line endings.

¹ CR-only was used on some ancient (m68k era?) Macintosh computers I believe.

P.S.: LFCR is 2 line endings in my parsers :D

Re: CRLF is obsolete and should be abolished

#226

Post by eqvinox »

matheusmoreira wrote: post #69
moomin wrote: post #41

Counterpoint: Unix deciding on a non-standard line ending was always a mistake. It has produced decades of random incompatibility for no particular benefit. CRLF isn’t a convention: it’s two different pieces of the base terminal API. You have no idea how many programs rely on CR and LF working correctly.

Yeah. It's weird how Unix picked LF given its love of terminals. CRLF is the semantically correct line ending considering terminal semantics. It's present in the terminal subsystem to this day, people just don't notice because they have OPOST output post processing enabled which automatically converts LF into CRLF.

I'd argue (but have no historical context) that it's a distinction between storage format and presentation interface, and IMHO that makes a lot of sense. A terminal has other operations too, backspaces and deletes being the most basic. Which coincidentally are one hell of a mess across different terminal types between ^H / 0x08 and DEL / 0x7f as well…

(And these distinctions predate UNIX — if I were confronted with an inconsistent mess I'd go for simplicity too, and a 2-byte newline is definitely not simple just by merit of being 2 bytes. I personally wouldn't have cared whether it was CR or LF, but would have cared to make it a single byte.)

Re: CRLF is obsolete and should be abolished

#227

Post by account42 »

hifromwork wrote: post #99
tptacek wrote: post #45

Earlier quoted context omitted.

This would be more persuasive if HTTP servers didn't already widely accept bare 0ah line termination. What's the first major public web site you can find that doesn't?

As the parent mentioned, it's security critical that every HTTP parser in the world - including every middleware, proxy, firewall, WAF - parses the headers in the same way. If you write a HTTP parser for a server application it's imperative you don't introduce random inconsistences with the standard (I can't believe I have to write this). On the other hand, as a client, it's OK to send malformed requests, as long as…

> As the parent mentioned, it's security critical that every HTTP parser in the world - including every middleware, proxy, firewall, WAF - parses the headers in the same way. If you write a HTTP parser for a server application it's imperative you don't introduce random inconsistences with the standard (I can't believe I have to write this).

No it isn't, at least not critical to all those parsers. My HTTP server couln't care less if some middle boxes that people go through are less or more strict in their HTTP parsing. This only becomes a concern when you operate something like a reverse proxy AND implement security-relevant policies in that proxy.

Re: CRLF is obsolete and should be abolished

#228

Post by srg0 »

I would also like to point out that English spelling is obsolete and should be abolished (/s). The text of the CRLF abolition proposal itself contains more digraphs, trigraphs, diphthongs, and silent letters than line-ending sequences. The last letter of the word "obsolete" is not necessary. "Should" can be written as only three letters in Shavian "𐑖𐑫𐑛".

According to ChatGPT, the original proposal had:

Number of sentences: 60 Number of diphthongs: 128 (pairs of vowels in the same syllable like "ai", "ea", etc.) Number of digraphs: 225 (pairs of letters representing a single sound, like "th", "ch", etc.) Number of trigraphs: 1 (three-letter combinations representing a single sound, like "sch") Number of silent letters: 15 (common silent letter patterns like "kn", "mb", etc.)

For all intents and purposes, CRLF is just another digraph.

Re: CRLF is obsolete and should be abolished

#229

Post by ksp-atlas »

srg0 wrote: post #228

I would also like to point out that English spelling is obsolete and should be abolished (/s). The text of the CRLF abolition proposal itself contains more digraphs, trigraphs, diphthongs, and silent letters than line-ending sequences. The last letter of the word "obsolete" is not necessary. "Should" can be written as only three letters in Shavian "𐑖𐑫𐑛". According to ChatGPT, the original proposal had: Number of s…

I'm a big fan of English spelling reform and know Shavian and sometimes write in it, but I feel shavian is limited due to how heavily it uses letter rotation. Dyslexics already have trouble with b, d, p and q, having most letters have a rotated form would be challenging

Re: CRLF is obsolete and should be abolished

#230

Post by perching_aix »

eqvinox wrote: post #216
perching_aix wrote: post #189

Earlier quoted context omitted.

> There's no such thing as a "trailing newline," there is only a line-terminating newline. Is "line-terminating newline" a controlled / established term I'm unfamiliar with or am I right to hold deep contempt against you? Because "trailing newline", contrary to what you claim, is 100% established terminology (in programming anyways), so I'd most definitely consider it "existing", and I find it actively puzzling that…

A trailing newline to me is "\n\n" at the end of a file, i.e. a superfluous empty line. That doesn't seem to be what the root comment is referring to, though?

I'm referring to files ending with a \n, and I do not see why this wouldn't be a trailing newline. It's a newline... at the end.
Post reply on HN