Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CRLF is obsolete and should be abolished

fossil-scm.org

Post reply on HN
Print view View original

231–240 of 273 posts

Re: CRLF is obsolete and should be abolished

#231

Post by eviks »

akira2501 wrote: post #215
eviks wrote: post #206

Earlier quoted context omitted.

They also add to the time wasted by not removing needless complexity in due time

The complexity of CRLF? The balance here, of course, being backwards compatability. I'd sooner kill EBCDIC, bad ASCII and Code Pages than worry about CRLF if we didn't have to care about ancient systems. Programming languages still retain C's operator precedence hierarchy even though it was itself meant to be a backwards compatible compromise and leads to errors around logical operator expressions. Anyways, this arti…

> The complexity of CRLF?

You didn't limit your general admiration of standards to CRLF, so no, not only that.

> about actively breaking systems like some kind of protocol terrorist in order to achieve an outcome at any cost,

That's simply false, he isn't

> Almost all implementations of these protocols will accept a bare NL as an end-of-line mark, even if it is technically incorrect.

Re: CRLF is obsolete and should be abolished

#232

Post by michaelmior »

Spooky23 wrote: post #124
FiloSottile wrote: post #40

Earlier quoted context omitted.

Exactly. Please DO NOT mess with protocols, especially legacy critical protocols based on in-band signaling. HTTP/1.1 was regrettably but irreversibly designed with security-critical parser alignment requirements. If two implementations disagree on whether `A:B\nC:D` contains a value for C, you can build a request smuggling gadget, leading to significant attacks. We live in a post-Postel world, only ever generate and…

What a weird reaction. Microsoft’s use of CRLF is an archaic pain in the ass. Taking a position that it should be deprecated isn’t radical or irresponsible — Microsoft makes gratuitous changes to things all of the time, why not this one? Hipp is probably one of the better engineering leaders out there. His point of view carries weight because of who he is, but should be evaluated on its merits. If Microsoft got rid o…

I didn't say we shouldn't get rid of it. I'm saying we shouldn't intentionally break existing protocols.

Re: CRLF is obsolete and should be abolished

#233

Post by michaelmior »

jcul wrote: post #219
michaelmior wrote: post #3

> various protocols (HTTP, SMTP, CSV) still "require" CRLF at the end of each line What would be the benefit to updating legacy protocols to just use NL? You save a handful of bits at the expense of a lot of potential bugs. HTTP/1(.1) is mostly replaced by HTTP/2 and later by now anyway. Sure, it makes sense not to require CRLF with any new protocols, but it doesn't seem worth updating legacy things. > Even if an est…

Not disagreeing with you, but implementation diverges from spec a lot anyway. I've had to write decoders for things like HTTP, SMTP, SIP (VoIP), and there's so many edge cases and undocumented behavior from different implementations that you have to still support. I find that it affects text based protocols, a lot more than binary protocols. Like TLS, or RTP, to stick with the examples above, have much less divergenc…

That's fair, but I don't see that as an argument for intentionally deviating from the spec.

Re: CRLF is obsolete and should be abolished

#234

Post by michaelmior »

tfehring wrote: post #137
michaelmior wrote: post #3

> various protocols (HTTP, SMTP, CSV) still "require" CRLF at the end of each line What would be the benefit to updating legacy protocols to just use NL? You save a handful of bits at the expense of a lot of potential bugs. HTTP/1(.1) is mostly replaced by HTTP/2 and later by now anyway. Sure, it makes sense not to require CRLF with any new protocols, but it doesn't seem worth updating legacy things. > Even if an est…

At least for CSV, there's a divergence between usage in practice and the spec. The spec requires CRLF, but all of the commonly used tools I've encountered for reading and writing CSVs can read files with CR, LF, or CRLF line endings, and when writing CSVs they'll default to either LF or platform-specific line endings. (Even Excel for Mac doesn't default to CRLF!) I think this divergence is bad and should be fixed. Bu…

For CSV, "breaking" changes seem like less of a big deal to me. Partially because there is already so much variation in how CSV is implemented.

Re: CRLF is obsolete and should be abolished

#235

Post by vitus »

nsnshsuejeb wrote: post #89
Ekaros wrote: post #57

Earlier quoted context omitted.

If there truly is a problem with existing protocols, propose and properly design new one that can replace it. Then if it is technically superior solution it should win in long run.

No need. Just convince the king (e.g. Google for HTTP) to make a tweak in the next standard version.

Good news! Both HTTP/2 and HTTP/3 use a packed binary representation for headers, so CRLF is nowhere to be found in RFC 9113 / 9114.

I don't see value in picking on Google or HTTP here, even if it is fashionable to do so.

Re: CRLF is obsolete and should be abolished

#236

Post by LegionMammal978 »

eviks wrote: post #231
akira2501 wrote: post #215

Earlier quoted context omitted.

The complexity of CRLF? The balance here, of course, being backwards compatability. I'd sooner kill EBCDIC, bad ASCII and Code Pages than worry about CRLF if we didn't have to care about ancient systems. Programming languages still retain C's operator precedence hierarchy even though it was itself meant to be a backwards compatible compromise and leads to errors around logical operator expressions. Anyways, this arti…

> The complexity of CRLF? You didn't limit your general admiration of standards to CRLF, so no, not only that. > about actively breaking systems like some kind of protocol terrorist in order to achieve an outcome at any cost, That's simply false, he isn't > Almost all implementations of these protocols will accept a bare NL as an end-of-line mark, even if it is technically incorrect.

> Almost all implementations of these protocols will accept a bare NL as an end-of-line mark, even if it is technically incorrect.

See https://news.ycombinator.com/item?id=41832555 as far as HTTP/1.1 goes, it's definitely common but far from universal. The big problem with "it's 100% safe to make this change, since it doesn't break anything I know about" is that there are always a lot of things you don't know about, not all of which can be boiled down to being negligible weirdos.

Re: CRLF is obsolete and should be abolished

#237

Post by immibis »

Aeolun wrote: post #153
michaelmior wrote: post #51

Earlier quoted context omitted.

We're talking about servers and clients here. The best way to ensure things work is to adhere to an established protocol. Aside from saving a few bytes, there doesn't seem to be any good reason to deviate.

Well, you can achieve the desired behavior in all situations by ignoring CR and treating any seen LF as NL. I just don’t see why you’d not want to do that as the implementer. If there’s some way to exploit that behavior I can’t see it.

The exploit is that your request went through a proxy which followed the standard (but failed to reject the bare NL) and the client sent a header after a bare NL which you think came from the proxy but actually came from the client - such as the client's IP address in a fake X-Forwarded-For, which the proxy would have removed if it had parsed it as a header.

This attack is even worse when applied to SMTP because the attacker can forge emails that pass SPF checking, by inserting the end of one message and start of another. This can also be done in HTTP if your reverse proxy uses a single multiplexed connection to your origin server, and the attacker can make their response go to the next user and desync all responses after that.

Re: CRLF is obsolete and should be abolished

#238

Post by djha-skin »

Kwpolska wrote: post #120
djha-skin wrote: post #5

> Let's make CRLF one less thing that your grandchildren need to know about or worry about. The struggle is real, the problem is real. Parents, teach your kids to use .gitattribute files[1]. While you're at it, teach them to hate byte order marks[2]. 1: https://stackoverflow.com/questions/73086622/is-a-gitattribu... 2: https://blog.djhaskin.com/blog/byte-order-marks-must-diemd/

Nope. Git should not mess with line endings, the remote repository not matching the code in your local clone can bite you when you least expect it. On Windows, one should disable the autocrlf misfeature (git config --global core.autocrlf false) and configure their text editor to default to LF.

Sure, don't use autocrlf. But some _windows_ tools need crlf, like for powershell or batch build scripts. Defaulting to lf in the editor will not save you.

Don't use `auto`, full marks, but the gitattributes file is indispensable as a safety net when explicit entries are used in it.

I mean, the whole point of the file is not everyone who is working on the project has their editors set to lf. Furthermore, not every tool is okay with line endings that are not CRLF.

When used properly (sure, ideally without auto), the git attributes file as a lifesaver.

Re: CRLF is obsolete and should be abolished

#239

Post by throwaway19972 »

djbusby wrote: post #87
tptacek wrote: post #38

Earlier quoted context omitted.

If two systems agree, independent of any specification someone somewhere else wrote, to accept a bare NL where a CRLF is specified, that is not "garbage". Standards documents are not laws; the horse drags the cart.

That's just two systems that happen to agree on garbage.

At that point, what does garbage even mean? There's just functional software and non-functional software.

Re: CRLF is obsolete and should be abolished

#240

Post by 0points »

naikrovek wrote: post #186
Spooky23 wrote: post #124

Earlier quoted context omitted.

What a weird reaction. Microsoft’s use of CRLF is an archaic pain in the ass. Taking a position that it should be deprecated isn’t radical or irresponsible — Microsoft makes gratuitous changes to things all of the time, why not this one? Hipp is probably one of the better engineering leaders out there. His point of view carries weight because of who he is, but should be evaluated on its merits. If Microsoft got rid o…

CRLF was the correct way to implement a new line the way we think of it now, because teletypes and typewriters considered the “return to the 0th column” and “go to the next line” to be different things that are each valid on their own. CRLF was the standardized way to implement “go down one line and return to column zero” and they’re the only ones who implemented new lines correctly at the outset. Blaming Microsoft n…

Additionally it is also dishonest to bring Microsoft into the discussion like that. The discussion revolved around _standardized_ network protocols, which is entirely unrelated to MS-DOS text formats.
Post reply on HN