Live data from Hacker News

Debugging file corruption on iOS

code.facebook.com

1–10 of 22 posts

Re: Debugging file corruption on iOS

#2
> "The SSL layer instead handled a raw file descriptor and, consequently, lifetime handling was not automatically synchronized ... We worked with the networking team and fixed this issue within hours."

Why on earth is Facebook writing their own SSL layer for iOS?

Re: Debugging file corruption on iOS

#4
I would love to see a standalone testcase demonstrating this bug. Although, what was Facebook's solution to fix it? Was it to make a fix to the SSL library to use a CFSocker wrapper?

| It turns out that abandoning manual code analysis was a good strategy.

Wait a minute, wasn't this manual code analysis? They were certainly digging around the codebase and a particular slice of commits to figure out why the crash kept occurring.

Re: Debugging file corruption on iOS

#6
This is the challenge of modern software development. When I first started my career, I spent two months looking for a random crashing bug in a video game due to misplaced free(); this was in code that was 100% ours. Now, there likely isn't an application in existence that does anything interesting where 100% of the code was written for the app itself. As such, you are no longer debugging your own code, but other people's as well.

Vernor Vinge coined the idea of a software archeologist, people who could sift through layers of code, all the way back to the beginning of time (Jan 1, 1970 and Unix, of course), to understand how systems work and to make modifications to them. We aren't that far from that point; already, there are people who seem to specializing in digital spelunking to find and fix bugs in these underlying layers.

While I love building new code, some of the most satisfying moments in my career have been when I've gone back through somebody else's code, untied it (oh, you built a polymorphic type system including class initializers and destructors in a decidedly not object oriented language? Cool.) and fixed it. There's something interesting about getting into somebody else's head and seeing how they approached the problem, then finding out where they were wrong.

Re: Debugging file corruption on iOS

#7
post #2

> "The SSL layer instead handled a raw file descriptor and, consequently, lifetime handling was not automatically synchronized ... We worked with the networking team and fixed this issue within hours." Why on earth is Facebook writing their own SSL layer for iOS?

I would guess one of NIH or SPDY. And they may not be (probably aren't is my guess) writing their own, just packaging one up that they know works with whatever depends on it.

Re: Debugging file corruption on iOS

#8

This is the challenge of modern software development. When I first started my career, I spent two months looking for a random crashing bug in a video game due to misplaced free(); this was in code that was 100% ours. Now, there likely isn't an application in existence that does anything interesting where 100% of the code was written for the app itself. As such, you are no longer debugging your own code, but other peo…

I love the idea of software archaeology in the context of video game software. Usually video game platforms (the earlier the better) cause developers do resort to wacky/unique tricks to make the most of their limited resources. And I feel like that ends up giving video games most of their unique flavor.

Re: Debugging file corruption on iOS

#9
post #4

I would love to see a standalone testcase demonstrating this bug. Although, what was Facebook's solution to fix it? Was it to make a fix to the SSL library to use a CFSocker wrapper? | It turns out that abandoning manual code analysis was a good strategy. Wait a minute, wasn't this manual code analysis? They were certainly digging around the codebase and a particular slice of commits to figure out why the crash kept…

Although fixing a bug requires analyzing the offending code, we weren't able to effectively narrow down the area of inspection through manual means (diff analysis, stack trace, git bisect). We instead narrowed down the area of code using sandboxing and non-trivial conditional breakpointing.

To fix the SSL library, we first used dup() to properly refcount the FD and then did more long-term restructuring to properly couple the FD & SSL object lifetime later.

Re: Debugging file corruption on iOS

#10
post #2

> "The SSL layer instead handled a raw file descriptor and, consequently, lifetime handling was not automatically synchronized ... We worked with the networking team and fixed this issue within hours." Why on earth is Facebook writing their own SSL layer for iOS?

SPDY (pre-Apple's version) + Open Source SSL Library + Perf Mods
Post reply on HN