51–60 of 80 posts
TL;DR: 1. Rails 2.3.11 introduced two subtle changes: - CSRF tokens have to be included in XHR POST requests - failing the CSRF check silently resets the session instead of throwing an exception 2. A/Bingo (his A/B testing library) checks if visitors are human with an XHR POST request. He did not notice that he needed to patch it to include the Rails CSRF token. 3. Race Condition: When the login/signup page is loaded…
Earlier quoted context omitted.
How is this a DoS? It's not significantly more resource-intensive than any other request. If you could submit enough of these requests to DoS the server, you could submit enough of any request to DoS the server, so the server is no more vulnerable this way than otherwise.
It allows you to grief the users of any rails site. You can log them out at will (for some value of at will). In the dawn of time, you could use a CSRF exploit and really grief the users. Then CSRF checking made that go away. This is now a step backwards, where 3rd party sites can affect your users.
Earlier quoted context omitted.
Habitually taking responsibility for things outside your control is probably a bad thing though.
This is very much not the cultural norm over in these parts. I'd be curious to hear your reason as to why you think that. (Genuinely curious, not passive-aggressive "curious.") Particularly with specific regards to engineering, even things which are theoretically not within our control (e.g. the server going down because a technician tripped over the cable) are often inside our ability to affect (by picking a better…
The rest of my thoughts on the matter center around the sense I get that, in continually taking responsibility (blame) for things outside your ability to influence, you will basically form a habit of it, which will lead you down a dark road of learned helplessness, omega behavior, and never being able to please your clients because everything bad that happens is "your" fault. If you think about it, if you form a habit of it and it begins to internalize, you are basically developing something similar to Imposter Syndrome.
Personally, when something is my fault I try to own it. When something is not, and is outside my sphere of influence, I try to skip past the issue of "fault" by briefly explaining the cause (in an attempt to show it was not incompetence on my part, while doing my best to avoid "pointing fingers") and focus on resolution.
Simple example: Coworker erases your work from centralized revision control (history and all), causing you to miss a deadline. Do you declare, "There was a glitch in revision control, it is my fault for trusting it" or do you state the facts and try to move on? The former certainly appears to be noble, but to me it appears to be the tragic hero sort of noble.
(I will add that I am very conflicted in writing this. I am saying to myself, "You cold heartless bastard!". I would, and do, cover for other people sometimes, but fundamentally I do believe in the untenable nature of regularly making yourself the scapegoat.)
Earlier quoted context omitted.
This is very much not the cultural norm over in these parts. I'd be curious to hear your reason as to why you think that. (Genuinely curious, not passive-aggressive "curious.") Particularly with specific regards to engineering, even things which are theoretically not within our control (e.g. the server going down because a technician tripped over the cable) are often inside our ability to affect (by picking a better…
I wasn't thinking of things you have the power to address. I was thinking more of making yourself the fall guy for things you didn't cause and cannot fix. I didn't know how to express exactly why, but scott_s summarizes part of my feelings on the matter quite well. The rest of my thoughts on the matter center around the sense I get that, in continually taking responsibility (blame) for things outside your ability to…
Taking responsibility for things that you did not cause and have utterly no power to fix has a long, long and documented history of (at least temporarily) destroying people. Obviously this is an extreme example, but I feel it illustrates rather well.
In the later afternoon a marketing guy on the traffic team came over and told me there was a problem with our trial signups. They'd dropped ~7% today for no apparently reason. We looked at the charts and today, for most of the day, compared to many other days and the past week there was indeed a dip. There were no changes to their marketing campaign or traffic levels either so they concluded it was a software bug.
But we hadn't pushed any new code that day. All that got pushed was some css changes from a designer so I concluded that it couldn't be a dev problem. I started looking from an ops perspective but couldn't find anything abnormal there either. There weren't a lot of memcached evictions, the db was doing fine, server loads were normal, etc.
I spent easily a couple of hours trying to find the source of the problem when marketing found it for me. Someone else was going through the metrics and noticed that we had the same trial signup rates for firefox, chrome and mobile browsers, it was only ie that had dropped. When filtered by ie and broken down by browser version he noticed that IE9, IE8 and IE7 were all the same as other days but IE6 had a 0%.
When you signed up to our site you signed up as a free member but were immediately offered a 5 day trial for a premium account and we had enough volume that the signup rates were predicatable and didn't vary much. When I created an account with IE6 it put me directly into the home page without showing me the trial offer page first.
Turns out the problem was indeed from the stylesheet change. The designer had not only changed a couple of buttons, he'd also added a font-face declaration. This font-face was not yet used anywhere and the font itself hadn't been uploaded to assets in production. What happened was IE6 would try to download the font-face and, when logged into our site we don't output 404 for pages that don't exist we redirect to the home page. So IE6 would get the redirect inside a stylesheet and follow that redirect in the browser. All the later IEs and other browsers simply ignored the redirect.
It was a very strange bug that would've resolved itself on its own the next day when the designer actually used and uploaded the font but it sure gave me a lot of head scratching and I never would've found it without the analytics.
Earlier quoted context omitted.
It allows you to grief the users of any rails site. You can log them out at will (for some value of at will). In the dawn of time, you could use a CSRF exploit and really grief the users. Then CSRF checking made that go away. This is now a step backwards, where 3rd party sites can affect your users.
You can't log any arbitrary out of their sessions. It only resets your session, as in the session cookie is reset.
While reading this article, it reminded me of an odd bug I encountered working with the Nitrogen Web Framework. I was finding that, similar to Patrick, it was losing sessions (though much more consistently), but only in Chrome and only with Yaws as the webserver. Switching to the Mochiweb server or Firefox caused the problem to go away. Firebug and Chrome's dev tools both reliably stated that the cookie header was in…
Similarly, one of the most banging-my-head-on-the-table bugs I've ever encountered caused all SSL connections from my iOS app to fail with an unhelpful 'something is wrong' message. Turned out my development iPhone's clock had just set itself back a few years, but I was so focused on the code that it took me a long time to figure that out.
tl;dr: A Rails upgrade last January caused previously valid code to reset the session (but only once per user per session). The session reset caused a logged-in user to be suddenly logged out; but most users' use-flow (and Patrick's test-flow) made the session reset happen before login, so the bug only manifested visibly about 1% of the time. Moral: race conditions are hard! (Alternate moral: read the release notes c…
Many people like to pretend that they can follow a discipline like Semantic Versioning (http://semver.org/). But minor changes like this can cause subtle bugs when you are upgrading between patch-level versions of a component that you're working with.