Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postel’s Principle is a Bad Idea

programmingisterrible.com

Post reply on HN
Print view View original

41–50 of 62 posts

Re: Postel’s Principle is a Bad Idea

#41

Post by btilly »

salgernon wrote: post #35
btilly wrote: post #27

Earlier quoted context omitted.

I can't read this comment without thinking about SOAP.

I could kiss you for that. If the use of a format for interoperability can only be reasonably used by a single vendor, it has no benefit over a binary protocol. The entire SOAP and XML-RPC space is postels law writ large.

Yup.

I had to only face the true horrors on one occasion, for a Responsys integration. They had the C# examples and the Java examples. The API that they offered for the two had differences because some methods would work with one, some with the other.

I'm a Perl programmer, so tried that. After all you just have to translate the language, right? Wrong. After banging my head against that mess for a week or so, I finally gave up, wrote the communication in Java, and had a Perl launcher for it.

Re: Postel’s Principle is a Bad Idea

#42

Post by derleth »

kens wrote: post #7
gruseom wrote: post #3

Earlier quoted context omitted.

when instead they've accidentally built dependencies on other people's implementation details I think it would be very interesting if you could give an example or two of this.

Windows software is a similar case - a program works on one version of Windows due to accidental dependencies on Windows implementation details (e.g. memory management), and then fails on the next version of Windows when the underlying implementation changes. Raymond Chen has written about the huge difficulty for Windows to maintain backwards compatibility with "broken" programs. New versions of Windows provide speci…

> http://www.joelonsoftware.com/articles/APIWar.html

In the terminology of this article, Torvalds is firmly in the Raymond Chen camp as far as "The kernel is not allowed to break user software" is concerned. The difference between Windows and Linux (and especially between Windows 95 era Windows and Linux of the same vintage) is, apparently, that Linux didn't come from MS-DOS, and so never had to allow application software to get hooks into low-level parts of the kernel.

There was never an official version of Linux for hardware without memory protection, and there never will be. Scope is important.

Re: Postel’s Principle is a Bad Idea

#43

Post by badgar »

> Treat input handling computational power as a privilege, and reduce it whenever possible.

A great example of this was Google's Code Search product, before it was canceled. Since full backtracking search was blowing out the tiny thread stacks in servers, they had to reduce what they allowed to actually regular expressions - expressions generating a regular language. Queries could be turned into DFAs of linear size with respect to input, making arbitrary public regex searches over code indices feasible.

Ross Cox's regular expressions write-ups are quite a fascinating deep-dive: http://swtch.com/~rsc/regexp/

Re: Postel’s Principle is a Bad Idea

#44

Post by Arnt »

It's a pity so many people grossly misunderstand Postel's Principle.

Postel didn't talk about off-spec behaviour. He talked about the borderline details, which were often quite hazy in early RFCs. When an RFC says the line length is at most 512 bytes and the terminator is CRLF, does that mean 510+CRLF or 512+CRLF? Postel says to accept 512+CRLF and send 510+CRLF.

If a write a receiver and want to accept 1024 bytes instead, maybe that's a good idea and maybe it's a bad idea. But if you do that, don't invoke Postel's Principle in defense.

Re: Postel’s Principle is a Bad Idea

#45

Post by DougBTX »

colomon wrote: post #23
MichaelGG wrote: post #21

Earlier quoted context omitted.

Possible attack: Because the strings are not ASCII, implementations now need to bring another library in to decode those strings. Now lets say someone encodes an end-string char (single quote?) using some alternative encoding that doesn't use the ASCII quote char. When an implementation saves this file, it normalizes that other encoding to use an ASCII single quote, then proceeds to write out the rest of the string.…

But this is where "be conservative in what you do" comes into play. The STEP format has formal rules for exporting all ASCII, Unicode, and ISO-8859 characters. A well-written STEP string exporter should handle them all without difficulty, no matter what goofy things are in the string. And again, if you're worried that there may be an attack vector, change high-bit-set characters to "[Illegal character value N]". Thou…

The tl;dr of the article is to define handling of invalid input, so that all conforming implementations will handle it in the same way, without having to reverse-engeneer eachother to be interoperable.

Re: Postel’s Principle is a Bad Idea

#46

Post by emn13 »

tedunangst wrote: post #39
barrkel wrote: post #26

Earlier quoted context omitted.

A nontechnical user, given a choice between two environments, one of which nags them pedantically over technical details, and another which displays the gist of entered content but perhaps with sometimes screwy formatting which one would win? Word processors won out over text processors for the nontechnical user partially for this reason. Postel's law applied to HTML let nontechnical users get things done with less i…

Writing correct HTML is not significantly harder than writing crappy HTML. Comparing it to latex vs word isn't a good analogy IMO. And wouldn't nontechnical users be using higher level HTML editors anyway? At least some bad HTML comes from lazy developers who should know better, and could have done better with a stricter tool.

It's not entirely trivial to write out correct HTML. In addition to properly closing tags, you need to deal with optional end tags (many get this wrong), optional start tags, the odd comment syntax, weird exceptionality wrt escaping esp. in script tags, context-dependant validity such as no block-level elements in inline elements (or nested a tags, or tags whose "type" depends on their attributes).

It's obviously easier to write than to read; but it's definitely easy to make a mistake. There's a lot of illogical cruft that's accumulated in HTML; so even a careful implementer might make a mistake (and might not detect it since most other implementations are so liberal).

Re: Postel’s Principle is a Bad Idea

#47

Post by mattmanser »

inopinatus wrote: post #6
gruseom wrote: post #3

Earlier quoted context omitted.

when instead they've accidentally built dependencies on other people's implementation details I think it would be very interesting if you could give an example or two of this.

Two off the top of my head: * A classic would be IE's abuse of TCP RST: http://www.stroppykitten.com/cms/index.php?option=com_conten... * A decent chunk of email server code (SMTP & IMAP implementations in particular) is there to handle erroneous client behaviours. The worst cases are those where the workaround leads to misbehaviours (or less optimal behaviours) for conforming clients. If I remember correctly, the po…

Yes, there were major changes between the way IMAP worked in Outlook 2003 & Outlook 2007 too, I think they re-wrote their IMAP code as it behaved completely differently. Admittedly it was much better, but it broke our custom IMAP server which I ended up fixing. I can't remember exactly why now, but Outlook 2003's implementation was bizarre, as if they'd not read the RFC.

That reminds me of the bizarre big because they used a short uint to store the message UID. Maybe it wasn't a short but it definitely wasn't 32-bit as per the spec, there was some magic number that if you went over 'boom'. As an end user it appeared that some messages just disappeared.

Re: Postel’s Principle is a Bad Idea

#48

Post by bartwe »

derleth wrote: post #42
kens wrote: post #7

Earlier quoted context omitted.

Windows software is a similar case - a program works on one version of Windows due to accidental dependencies on Windows implementation details (e.g. memory management), and then fails on the next version of Windows when the underlying implementation changes. Raymond Chen has written about the huge difficulty for Windows to maintain backwards compatibility with "broken" programs. New versions of Windows provide speci…

> http://www.joelonsoftware.com/articles/APIWar.html In the terminology of this article, Torvalds is firmly in the Raymond Chen camp as far as "The kernel is not allowed to break user software" is concerned. The difference between Windows and Linux (and especially between Windows 95 era Windows and Linux of the same vintage) is, apparently, that Linux didn't come from MS-DOS, and so never had to allow application sof…

No this is actually very different. Linus doesn't want breaking API changes to documented behavior, in the Raymond Chen case its not breaking applications that misbehave or abuse undocumented behavior.

Re: Postel’s Principle is a Bad Idea

#49

Post by colomon »

tedunangst wrote: post #29
colomon wrote: post #25

Earlier quoted context omitted.

Maybe I'm missing something here, but a valid STEP string can already encode any arbitrary Unicode code point. It just does it using 7-bit ASCII. If your code is somehow executing these strings without examining their content, then you are already in big, big trouble. Trying to do something with 8-bit characters -- whether skipping them, indicating an illegal character in the string, or trying to guess what was reall…

The problem is if you decode a particular byte sequence that causes a bad action (if that's possible with step files) in a different way than some other program that is supposed to keep you safe. In the case of ie, ie decoded one way and forum software might decode a different way. So the forum software says the string is safe for the browser (according to its decoding rules) but then the browser applies different ru…

As jbert pointed out, if your program's main job is to say whether or not something is safe, and it liberally says "Oh yeah, I think that's safe", that's pretty much the exact opposite of "be conservative in what you do".

Re: Postel’s Principle is a Bad Idea

#50

Post by colomon »

DougBTX wrote: post #45
colomon wrote: post #23

Earlier quoted context omitted.

But this is where "be conservative in what you do" comes into play. The STEP format has formal rules for exporting all ASCII, Unicode, and ISO-8859 characters. A well-written STEP string exporter should handle them all without difficulty, no matter what goofy things are in the string. And again, if you're worried that there may be an attack vector, change high-bit-set characters to "[Illegal character value N]". Thou…

The tl;dr of the article is to define handling of invalid input, so that all conforming implementations will handle it in the same way, without having to reverse-engeneer eachother to be interoperable.

So you're saying that every time I find a STEP file written in an invalid fashion, I should convene an ISO 10303 committee and wait for years to find out how everyone should handle it? That's doubly insane, because it would take many bugs that can be fixed in a day and make my customers suffer from them for years, while at the same time requiring me to modify my program to handle every bug found by every STEP software vendor or cease to be conforming.
Post reply on HN