Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CRLF is obsolete and should be abolished

fossil-scm.org

Post reply on HN
Print view View original

171–180 of 273 posts

Re: CRLF is obsolete and should be abolished

#171

Post by perching_aix »

bityard wrote: post #165
perching_aix wrote: post #145

Earlier quoted context omitted.

My problem is that what I input (and observe!) doesn't match what's persisted. Worse still, editors lie about it to me until I close the file and reopen it. And just to really turn the knife, various programs will then throw a fit that a character that I did not input and my editor lies about not being present, is present. I hope it's appreciable why I find this frustrating. I expect my editor to do what I say, not s…

I have been using and programming Unix systems for almost 30 years and have not run into anything like what you are describing. What Unix program "throws a fit" when encountering a perfectly normal newline in the last line in a file?

"Unix programs" I haven't ran into throwing a fit per se. That's why I didn't write that.

What I ran into issues with was contemporary software that's shipped to Linux, such as Neo4j, which expects its license files to have no newline at the end of the file, and will actively refuse to start otherwise.

I have a feeling I'll now experience the "well that's that software's problem then" part of this debate. Just like how software not being able to handle CRLF / CR-only / LF-only, is always the problem - instead of text files being a joke, and platforms assuming things about them being the problem.

Re: CRLF is obsolete and should be abolished

#172

Post by lifthrasiir »

Eduard wrote: post #109

> Stop using "linefeed" as the name for the U+000a code point. stop reinventing terms. it's literally standardized with the name "LF" / "line feed" in Unicode.

Just in case... Unicode doesn't define anything about C0 control characters. Everything you see from the code chart is from ISO/IEC 6429 and only shown there for information. Some parts of Unicode and related standards do assign a special meaning to U+000A, but often also to U+000D for the obvious reason.

Re: CRLF is obsolete and should be abolished

#174

Post by LegionMammal978 »

tptacek wrote: post #152
LegionMammal978 wrote: post #146

Earlier quoted context omitted.

My apologies, I was using a client which kept the connection alive between the 0D0A and 0A requests, which has an effect on www.pinterest.com. Rerunning the test with separate connections for 0D0A and 0A requests, www.pinterest.com and phys.org are no longer affected (I've removed the two from the list), but all other URLs are still affected.

I picked one at random --- hhs.gov --- and it too appears to work? For what it's worth: I'm testing by piping the bytes for a bare-newline HTTP request directly into netcat.

Make sure you're contacting hhs.gov and not www.hhs.gov, the www. subdomain reacts differently.

  $ printf 'GET / HTTP/1.1\r\nHost: hhs.gov\r\n\r\n' | nc hhs.gov 80
  HTTP/1.1 302 Found
  Date: Mon, 14 Oct 2024 01:38:29 GMT
  Server: Apache
  Location: http://www.hhs.gov/web/508//
  Content-Length: 212
  Content-Type: text/html; charset=iso-8859-1
  
  
  
  302 Found
  
  Found
  
The document has moved here.

  
  ^C
  $ printf 'GET / HTTP/1.1\nHost: hhs.gov\n\n' | nc hhs.gov 80
  HTTP/1.1 400 Bad Request
  Date: Mon, 14 Oct 2024 01:38:40 GMT
  Server: Apache
  Content-Length: 226
  Connection: close
  Content-Type: text/html; charset=iso-8859-1
  
  
  
  400 Bad Request
  
  Bad Request
  
Your browser sent a request that this server could not understand.

Re: CRLF is obsolete and should be abolished

#175

Post by abhinavk »

MatthiasPortzel wrote: post #121

They acted on these words, updating their HTTP server to serve just \n. => https://sqlite.org/althttpd/info/8d917cb10df3ad28 Send bare \n instead of \r\n for all HTTP reply headers. While browser aren't effected, this broke compatibility with at least Zig's HTTP client. => https://github.com/ziglang/zig/issues/21674 zig fetch does not work with sqlite.org

It has been reverted.

Re: CRLF is obsolete and should be abolished

#176

Post by dwattttt »

layer8 wrote: post #164
Kwpolska wrote: post #120

Earlier quoted context omitted.

Nope. Git should not mess with line endings, the remote repository not matching the code in your local clone can bite you when you least expect it. On Windows, one should disable the autocrlf misfeature (git config --global core.autocrlf false) and configure their text editor to default to LF.

This is impractical in many situations, because tools that process build-source files (for example XML files that control the build, or generated source files) inherently generate CRLF on Windows. These are many, many, many tools, not just one’s text editor. The correct solution is to use .gitattributes.

If you're using tools that only support one particular line ending, the solution isn't to convert your source automatically in the repo, it's to know your source is always stored in one format, and convert it back/forth when a tool needs it to be different.

How would you handle two different tools only supporting disjoint line endings?

Re: CRLF is obsolete and should be abolished

#177

Post by dwattttt »

nsnshsuejeb wrote: post #86
dijit wrote: post #82

Earlier quoted context omitted.

I think the parent is equally denigrating the situation. Leaders choose the standards, especially as they approach monopoly. Worse still: people will come out of the woodwork to actively defend the monopolist de facto standard producer.

Not defending the producer, just making pragmatic choices!

Pragmatic choices is what Postel is known for now, and it makes for a pretty confused world when everyone makes different pragmatic choices.

Re: CRLF is obsolete and should be abolished

#178

Post by SQLite »

Author here:

My title was imprecise and unclear. I didn't mean that you should raise errors if CRLF is used as a line terminator in (for example) HTTP, only that a bare NL should be allowed as an acceptable line terminator. RFC2616 recommends as much (section 19.3 paragraph 3) but doesn't require it. The text of my proposal does say that CRLF should continue to be accepted, for backwards compatibility, just not required and not generated by default. I failed to make that point clear.

My initial experiments suggested that this idea would work fine and that few people would even notice. Initially, it appeared that when systems only generate NL instead of CRLF, everything would just keep working seamlessly and without problems. But, alas, there are more systems in circulation that are unable to deal with bare NLs than I knew. And I didn't sell my idea very well. So there was breakage and push-back.

I have revised the document accordingly and reverted the various systems that I control to generate CRLFs again. The revolution is over. Our grandchildren will have to continue dealing with CRLFs, it seems. Bummer.

Thanks to everyone who participated in my experiment. I'm sorry it didn't work out.

Re: CRLF is obsolete and should be abolished

#179

Post by fijiaarone »

Line feed is exactly what you do when you are editing text. But nobody uses it.

CR + LF was meant as an instruction for teletype printers, so it is outdated, and looks like he withdrew the proposal (which couldn’t have ever been serious) after some feedback.

Fossil SCM, btw, was written by the creator of SQLite, so his opinion shouldn’t be discounted as some random nobody.

Re: CRLF is obsolete and should be abolished

#180

Post by dwattttt »

deltaknight wrote: post #31
tedunangst wrote: post #27

No mention of what happened the last time we mixed and matched line endings? https://smtpsmuggling.com/

Doesn’t this show that ignoring CR and only processing LFs is a good idea? If I’m understanding right (probably wrong), this vuln relied on some servers using CRLF only as endings, and others supporting both CRLF and LF. If every server updated to line-end of LF, thereby supporting both types, this vuln wouldn’t happen? Of course if there’s is a mixed bag then I guess this is still possible, if your server only suppo…

As I mentioned else-thread: it doesn't matter as much which option is chosen, so long as everyone agrees. If everyone agrees that LF on its own is enough (and we stop sending CR's to make sure it's not part of whatever comes before LF), that's fine. But it's just as fine for everyone to agree that CRLF is right, and reject plain LF.
Post reply on HN