Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Post Mortem: A single whitespace character

eatabit.com

Post reply on HN
Print view View original

161–170 of 209 posts

Re: Post Mortem: A single whitespace character

#161

Post by sixwing »

goleksiak wrote: post #157

Heroku came back and said: Looking through the system, I see that you were sent two emails (in August and September) as several of your apps were migrated to the new routing stack ( https://devcenter.heroku.com/articles/heroku-improved-router ). As mentioned in the documentation, the new router follows stricter adherence to the RFC specification, including sensitivity to spaces. ...and sure enough, there is a line th…

The team at Heroku (where I currently PM) is constantly trying to improve our communication and documentation. We're definitely sorry that this caused problems, and we'll work even harder to make sure that our communication calls out any potential issues. Again - thanks for reaching out to us, and let us know if we can help.

Re: Post Mortem: A single whitespace character

#162

Post by pilif »

ldng wrote: post #158
pilif wrote: post #50

Likely "Cowboy" is a transparent proxy added by your mobile service provider. I had a similar thing happening a year ago when the mobile provider used by most of our barcode scanners decided to add a transparent proxy into the loop (without telling anybody). The solution for this problem: Use SSL. I mean: There are already many good reasons to use SSL, but whenever you need to send any kind of mission critical data o…

What's wrong with transparent proxy ? Isn't how HTTP caching is supposed to work ? I would think the cache headers are the solution rather than SSL. It feels like you are kind of throwing the baby with the bath water. IMHO, badly configured transparent proxy does not mean the concept is bad, does it ?

The problem is that you have no recourse. When the carrier decides to f up your connections somehow, you can try to work around the issue, until they break it some more to the point where the one thing you really needed also stopped working.

Then you can hope that you are big enough to have priority with the carrier or you know somebody who knows somebody who can fix it.

Or you don't deal with any of this and just go SSL. A certificate will cost you $100 per year in the worst case. Thats about one hour of your time spent fixing proxy issues (not including customers and/or end users breathing down your neck because their software just stopped working for some as yet unknown reason)

Re: Post Mortem: A single whitespace character

#164

Post by ldng »

pilif wrote: post #162
ldng wrote: post #158

Earlier quoted context omitted.

What's wrong with transparent proxy ? Isn't how HTTP caching is supposed to work ? I would think the cache headers are the solution rather than SSL. It feels like you are kind of throwing the baby with the bath water. IMHO, badly configured transparent proxy does not mean the concept is bad, does it ?

The problem is that you have no recourse. When the carrier decides to f up your connections somehow, you can try to work around the issue, until they break it some more to the point where the one thing you really needed also stopped working. Then you can hope that you are big enough to have priority with the carrier or you know somebody who knows somebody who can fix it. Or you don't deal with any of this and just go…

Except the carrier, or the that case the hoster, did not break anything. It's their software that don't comply to the RFC. HTTP is supposed to be cacheable when it makes sens. With SSL to work that around you break the Internet somehow.

To me, there are valid usecase for SSL, using it to work around proxies is not one. That said, I get your point, you prefer the possibly easier and safer way. But you still might run into another set of problems (https://news.ycombinator.com/item?id=8471877).

Re: Post Mortem: A single whitespace character

#165

Post by zimpenfish »

ajanuary wrote: post #55
zimpenfish wrote: post #45

Earlier quoted context omitted.

Unless I'm reading RFC 3986 incorrectly, that's valid because you can't have an empty segment in the path part of a URI.

I think you're reading it incorrectly. You can have an empty segment in the path. The BNF for a segment is: segment = *pchar Which according to RFC2234 section 3.6 means zero or more repetitions.

The difference between `path-abempty` and `path-absolute` is bloody confusing but I think you're right.

Re: Post Mortem: A single whitespace character

#166

Post by zimpenfish »

userbinator wrote: post #53
zimpenfish wrote: post #49

Earlier quoted context omitted.

Sounds like something that could be added to http://httpbin.org

That runs on Python/Flask, which is already a layer of abstraction above where HTTP conformance testing would be; what you need is something that listens on a TCP socket and parses the requests itself.

Actually, thinking about it, didn't Zed Shaw make a Ragel-based strict-conformance HTTP parser?

> Simply being more explicit about what is valid HTTP means that most of the security attacks that worked on Apache were rejected outright when tried on Mongrel.

Which I guess is a qualified "sounds like it, maybe?"

Re: Post Mortem: A single whitespace character

#167

Post by fulafel »

ldng wrote: post #158
pilif wrote: post #50

Likely "Cowboy" is a transparent proxy added by your mobile service provider. I had a similar thing happening a year ago when the mobile provider used by most of our barcode scanners decided to add a transparent proxy into the loop (without telling anybody). The solution for this problem: Use SSL. I mean: There are already many good reasons to use SSL, but whenever you need to send any kind of mission critical data o…

What's wrong with transparent proxy ? Isn't how HTTP caching is supposed to work ? I would think the cache headers are the solution rather than SSL. It feels like you are kind of throwing the baby with the bath water. IMHO, badly configured transparent proxy does not mean the concept is bad, does it ?

No, "transparent proxying" is a clear violation of HTTP specs (as well as TCP protocol, and IP's "thou shall not mess with packets in transit" principle/specs). It's essentially a MITM attack and all bets are off wrt correctness.

Re: Post Mortem: A single whitespace character

#168

Post by MichaelGG »

linuxlizard wrote: post #142
goleksiak wrote: post #69

Earlier quoted context omitted.

We would really like to use HTTPS but it's not supported by the Arduino chipset as I understand it. Though I'm not the hardware guy here at eatabit...

Why not ROT13? Or a simple substitution cypher? Not trying to be silly. But if the only goal is to prevent man-in-the-middle attacks such as someone mangling the data, why not "corrupt" the data such that the phone company in the middle can't read it? You control both ends. You can make your own "security". You're not explicitly worried about security. You're not worried about Evil Person reading your messages. You j…

They could try a different port - some systems won't bother.

They might also use TLS with null cipher. That should be not-so-intensive, even on a tiny processor. And it could be enough to defeat some packet-modifiers (they may notice it's TLS and not analyze), while maintaining HTTPS compatibility.

Re: Post Mortem: A single whitespace character

#169

Post by tedchs »

goleksiak wrote: post #69
pilif wrote: post #50

Likely "Cowboy" is a transparent proxy added by your mobile service provider. I had a similar thing happening a year ago when the mobile provider used by most of our barcode scanners decided to add a transparent proxy into the loop (without telling anybody). The solution for this problem: Use SSL. I mean: There are already many good reasons to use SSL, but whenever you need to send any kind of mission critical data o…

We would really like to use HTTPS but it's not supported by the Arduino chipset as I understand it. Though I'm not the hardware guy here at eatabit...

SPDY is an alternative to HTTP that avoids at least Verizon's carrier shenanigans.

Re: Post Mortem: A single whitespace character

#170

Post by MichaelGG »

Xylakant wrote: post #18
jrochkind1 wrote: post #7

This very example -- requests were technically illegal all the time without devs realizing, but something in the stack changed to start rejecting them -- demonstrates the fallacy of the "be liberal in what you accept, strict in what you issue" principal. If all the web servers involved had been strict in rejecting the illegal request from the start, they would have noticed the bug in development before deploying to f…

I don't agree that "be liberal in what you accept, strict in what you issue" is a fallacy. The client actually failed to adhere to the "be strict in what you issue" principal, just as the Cowboy was not liberal in accepting. All software will sooner or later exhibit bugs or be stricter or more lenient about a standard. I think the fallacy is to assume that once stuff works in production, only your changes can trigger…

The liberal/strict thing is a terrible idea. It introduces completely busted behavior.

Consider a client that emits \n instead of \r\n. How do you handle it? Liberally? OK, treat 'em like CRLFs. Now you read \n\n. Everything after that is content, right?

Oops, you're now ignoring headers, potentially security-sensitive ones.

I've run into this exact bug in production, leading to a security problem. The client, proxy, and endpoints had different ways of handling CRLF. Some would treat \n\n as the end of headers, some not. Exploiting this, clients could route requests through the proxy and add special headers that only the proxy should have been able to add (like X-Client-IP).

Apart from this, the whole "robustness principle" just leads to a bunch of guessing and even more incompatible implementations. See HTML as another example mess.

Post reply on HN