Live data from Hacker News

Tracking down a memory leak in Ruby's EventMachine

blog.nelhage.com

11–20 of 33 posts

Re: Tracking down a memory leak in Ruby's EventMachine

#11
post #10
post #7

Earlier quoted context omitted.

Me too. I wonder what would make for more posts like this.

Who says there aren't already? Note that what's on the front page is just up-voted content. Tons of stuff (some quite good) slips through the cracks every day.

When I look at /newest, I don't see a lot of high-quality technical posts falling through the cracks. Do you have examples?

Re: Tracking down a memory leak in Ruby's EventMachine

#13
post #3

There are two things I really like about this walkthrough: 1. It shows how bugs can be something quite conceptually simple 2. It shows the value of logical, detective-like thinking in tracking these bugs. Even as a programmer, I still think of bug-hunting as something requiring an encyclopedia knowledge of the trivial and arcane. Obviously, it looks easier in hindsight, but the OP does a great job of demonstrating ho…

Deduction is great, but what I read here was how much experience matters in debugging. You have to know what to look and test for to provide fodder for your reasoning capabilities, or else you're just shooting in the dark. The OP here had a plan informed by what I'm guessing are years of experience solving this kind of thing ("It's probably small objects. I should dump the core and look for repeating memory patterns."). Then there's a lot of accumulated wisdom that reprioritizes your search as you go along: "That looks like a flag because flags tend to look like that.", etc.

I have no doubt that the OP is good at reasoning logically, but the take-home lesson here is, if you want to be good at debugging, do a lot of debugging.

Re: Tracking down a memory leak in Ruby's EventMachine

#15
post #7

Earlier quoted context omitted.

Me too. I wonder what would make for more posts like this.

Go here every day and upvote interesting articles after reading them. http://news.ycombinator.com/newest

No, what I'm saying (cf my other comment in this thread) is that it's actually a supply problem.

Re: Tracking down a memory leak in Ruby's EventMachine

#16
post #15

Earlier quoted context omitted.

Go here every day and upvote interesting articles after reading them. http://news.ycombinator.com/newest

No, what I'm saying (cf my other comment in this thread) is that it's actually a supply problem.

Not sure I agree there. There are lots of interesting posts which sink on that queue and fall off it without getting sufficient votes to be seen, while less interesting but more controversial stuff floats to the top. All it would take is a few score more people voting technical stories up consistently to change the story mix.

Re: Tracking down a memory leak in Ruby's EventMachine

#17
"If you’ve stared at too many Linux coredumps, as I have, that number looks suspicious. Interpreted in little-endian, that is 0x00007f1b5358a800, which points near the top of the userspace portion of the address space on an amd64 Linux machine.

In fewer words: It’s most likely a pointer."

As someone who has not stared at any core dump for more than about 2 seconds, I admire this level of skill.

Re: Tracking down a memory leak in Ruby's EventMachine

#19
post #3

There are two things I really like about this walkthrough: 1. It shows how bugs can be something quite conceptually simple 2. It shows the value of logical, detective-like thinking in tracking these bugs. Even as a programmer, I still think of bug-hunting as something requiring an encyclopedia knowledge of the trivial and arcane. Obviously, it looks easier in hindsight, but the OP does a great job of demonstrating ho…

Deduction is great, but what I read here was how much experience matters in debugging. You have to know what to look and test for to provide fodder for your reasoning capabilities, or else you're just shooting in the dark. The OP here had a plan informed by what I'm guessing are years of experience solving this kind of thing ("It's probably small objects. I should dump the core and look for repeating memory patterns.…

Yes, this is true, so as I said, debugging looks easier in hindsight, especially someone else's hindsight :)

Summarizing the reasoning process:

1. The program's object space doesn't contain an absurd number of small objects, so inspect the core dump

2. 95% of the core dump is leaked objects, so a random sample should contain clues to the composition of the leaked objects.

3. A repeated pattern in every leaked object indicates a common pointer, i.e. a single type of object.

4. The signature helps find what file in the program is being referenced, which indicates that the pointer's object type is a BIO struct

5. This kind of leak isn't possible in straight Ruby. So between OpenSSL and EventMachine's C/C++ code, the latter is more likely to have something awry.

6. Search the EM code for BIO constructors

7. Check each constructor call to see if the BIO instance is ultimately freed

---

Steps 3 and 4 are the most arcane to me...I am pretty sure I do not know enough about memory to look at a hex signature and realize where in the address space it refers to, or even the significance of it.

Post reply on HN