Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CRLF is obsolete and should be abolished

fossil-scm.org

Post reply on HN
Print view View original

151–160 of 273 posts

Re: CRLF is obsolete and should be abolished

#151

Post by theamk »

tptacek wrote: post #104
hifromwork wrote: post #99

Earlier quoted context omitted.

As the parent mentioned, it's security critical that every HTTP parser in the world - including every middleware, proxy, firewall, WAF - parses the headers in the same way. If you write a HTTP parser for a server application it's imperative you don't introduce random inconsistences with the standard (I can't believe I have to write this). On the other hand, as a client, it's OK to send malformed requests, as long as…

That appears to be an argument in favor of accepting bare-0ah, since as a positive statement that is the situation on the Internet today.

Wouldn't the safest thing, security-wise, to fail fast on bare 0ah?

As a web server, you may not know which intermediate proxies did the request traverse before arriving to your port. Given that request smuggling is a thing, failing fast with no further parsing on any protocol deviations seems to be the most secure thing.

Re: CRLF is obsolete and should be abolished

#152

Post by tptacek »

LegionMammal978 wrote: post #146
tptacek wrote: post #132

Earlier quoted context omitted.

You sure about this? www.pinterest.com, for instance, does not appear to care whether I 0d0a or just 0a.

My apologies, I was using a client which kept the connection alive between the 0D0A and 0A requests, which has an effect on www.pinterest.com. Rerunning the test with separate connections for 0D0A and 0A requests, www.pinterest.com and phys.org are no longer affected (I've removed the two from the list), but all other URLs are still affected.

I picked one at random --- hhs.gov --- and it too appears to work?

For what it's worth: I'm testing by piping the bytes for a bare-newline HTTP request directly into netcat.

Re: CRLF is obsolete and should be abolished

#153

Post by Aeolun »

michaelmior wrote: post #51
tptacek wrote: post #45

Earlier quoted context omitted.

This would be more persuasive if HTTP servers didn't already widely accept bare 0ah line termination. What's the first major public web site you can find that doesn't?

We're talking about servers and clients here. The best way to ensure things work is to adhere to an established protocol. Aside from saving a few bytes, there doesn't seem to be any good reason to deviate.

Well, you can achieve the desired behavior in all situations by ignoring CR and treating any seen LF as NL.

I just don’t see why you’d not want to do that as the implementer. If there’s some way to exploit that behavior I can’t see it.

Re: CRLF is obsolete and should be abolished

#154

Post by convolvatron »

tptacek wrote: post #107
perching_aix wrote: post #70

Earlier quoted context omitted.

Apart from colloquially considering standards not-necessarily-normative being, in my opinion, nonsensical (see below), to the best of my knowledge at the very least the STD subseries of IETF standards documents are normative in nature: https://datatracker.ietf.org/doc/std > They are not the only tool, and they don't carry any moral force. Indeed there are countless other standards bodies in the world also producing n…

Ok, but just to be clear: the standards-track HTTP RFC says you can use a single LF. I don't think this issue is as clear as people seem to want it to be.

none of this is as clear as anyone wants it to be. if standards _could_ be completely formally described, it would be an entirely different world. I did quite a bit of work implementing draft standards in the IETF, and and the end of the day the standard is the best we can make it, but for non-trivial things good luck actually implementing it without something to test against or a reference implementation.

thats the context in which Postel's law make absolute sense. not that you should forgo any sanity checking, or attempt to interpret garbage or make up frame boundaries. but when there is a potential ambiguity, and you can safely tolerate it, then its really helpful for you to do so.

Re: CRLF is obsolete and should be abolished

#155

Post by tptacek »

theamk wrote: post #151
tptacek wrote: post #104

Earlier quoted context omitted.

That appears to be an argument in favor of accepting bare-0ah, since as a positive statement that is the situation on the Internet today.

Wouldn't the safest thing, security-wise, to fail fast on bare 0ah? As a web server, you may not know which intermediate proxies did the request traverse before arriving to your port. Given that request smuggling is a thing, failing fast with no further parsing on any protocol deviations seems to be the most secure thing.

I mean the safest thing would be to send an RST as soon as you see a SYN for 80/tcp.

Re: CRLF is obsolete and should be abolished

#156

Post by tptacek »

rtpg wrote: post #136
tptacek wrote: post #45

Earlier quoted context omitted.

This would be more persuasive if HTTP servers didn't already widely accept bare 0ah line termination. What's the first major public web site you can find that doesn't?

Gunicorn expects `\r\n` for lines (see gunicorn/http/message.py:read_line), though it's possible that every middleware that is in front of gunicorn in practice normalizes lines to avoid this issue.

Yep, tested it locally, you're right; gotta CRLF to gunicorn.

Re: CRLF is obsolete and should be abolished

#158

Post by dang »

mvdtnz wrote: post #28
fweimer wrote: post #8

SMTP https://datatracker.ietf.org/doc/html/rfc2821#section-4.1.1.... > is pretty clear that the message termination sequence is CR LF . CR LF, not LF . LF, and disagreements in this spot are known to cause problems (include undesirable message injection). But then enough alternative implementations that recognize LF . LF as well are out there, so maybe the original SMTP rules do not matter anymore.

[flagged]

"Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that.""

https://news.ycombinator.com/newsguidelines.html

Re: CRLF is obsolete and should be abolished

#159

Post by elcritch »

WesolyKubeczek wrote: post #29

> Even if an established protocol (HTTP, SMTP, CSV, FTP) technically requires CRLF as a line ending, do not comply. Send only NL. Now just go pound sand. Seriously. And you owe me 5 minutes of my life wasted on reading the whole thing. My god, I would have thought all those “simplification” ideas die off once you have 3 years of experience or more. Some people won’t learn. P. S. Guess even the most brilliant people t…

Conversely, I'd argue most brilliant people tend to have more dumb ideas than others, usually on oddly specific topics which most people would find inconsequential.

Re: CRLF is obsolete and should be abolished

#160

Post by akira2501 »

rkeene2 wrote: post #21
perching_aix wrote: post #7

Well, at least the title is honest. Straight up asking people to break standards out of sheer conviction is a new one for me personally, but it's definitely one of the attitudes of all time, so maybe it's just me being green. Can we ask for the typical *nix text editors to disobey the POSIX standard of a text file next, so that I don't need to use hex editing to get trailing newlines off the end of files?

People don't seem to mind when Chrome does it [0]. The response "standards aren't a death pact" stands out in particular. [0] https://news.ycombinator.com/item?id=13860682

Death pact? Jeez. Standards simply prevent people from having to waste time debugging dumb issues that rightfully could have been avoided.
Post reply on HN