Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Go: Severe memory problems on 32bit Linux

groups.google.com

Post reply on HN
Print view View original

21–30 of 159 posts

Re: Go: Severe memory problems on 32bit Linux

#21

Post by r00tbeer »

_delirium wrote: post #3

This is the relevant bug: http://code.google.com/p/go/issues/detail?id=909 As the discussion there and in the thread says, the root problem is that Go uses a conservative garbage collector, and on 32-bit a lot more values look like pointers than on 64-bit, so many more things don't get freed in long-running processes. Seems not to be easy to fix.

I know early JVM garabage collectors were also "conservative", but I don't recall JVMs running into these sorts of problems. Maybe folks are just using more of the 4G address space these days than back in the day of conservative GC JVMs?

Re: Go: Severe memory problems on 32bit Linux

#22

Post by moilolita »

eternalban wrote: post #15
moilolita wrote: post #12

Earlier quoted context omitted.

Unfortunately you are right. For me this is a very unpleasant surprise after I put aside D and rewrote my little framework in Go. Everybody said that the gc is not final, that there are some performance issues and they are working on it but I never imagined that such catastrophic bugs are not solved by now.

I also looked at D/Go -- I predict the reign of JVM is over (quote me). I wouldn't bother with the 32 bit hiccups. Go hits the sweet spot pretty well. You have chosen well. Hang in there.

D has a nice allocation management(i loved the manual+gc approach) and in my benchmarks, optimized D was slightly faster than Go at almost anything and consumed up to 70% less memory (I assumed that was because the Go gc kicked in later and was a bit lazy) but D was weak at threading/synchronization and the documentation of the standard lib was quite messy and lacking. So I decided that in the long run Go will be better (cheap goroutines, channels, good stdlib /documentation + support from google and the prospect of a better GC, all indicated a clear winner).I really hope they fix this cause I can't throw away my atom box and ARM is becoming more and more important.

Re: Go: Severe memory problems on 32bit Linux

#23

Post by eblume »

smanek wrote: post #19
jedbrown wrote: post #14

However, it does not address the base issue, which is that Go uses a conservative garbage collector, and more values look like pointers in a 32-bit world. The only real fix would be to improve the garbage collector's understanding of which values are pointers and which are something else (e.g., floating point numbers that happen to look like pointers). And that is not an easy fix. How does this GC work? Is it literal…

Yep, that's basically it. It sounds crazy (and it is!) - but it often works reasonably well in practice. SBCL (one of the most performant Common Lisp implementations) has an 'imprecise gc' that works the same way - and I've seen reasonably heavily stressed processes with uptimes in the weeks/months. Remember, even a few years ago (before fastthread, etc) a reasonably loaded Ruby on Rails app couldn't stay up for more…

That's really interesting - is there a place where I can read more about these early Rails performance issues?

Re: Go: Severe memory problems on 32bit Linux

#24

Post by scott_s »

_delirium wrote: post #10
sounds wrote: post #8

Earlier quoted context omitted.

Publicizing the bug may speed the fix along. Here's the real test: is hitting the front page of HN going to make a difference?

I don't think this one is lacking interest, but an obvious fix. It looks like Russ Cox did commit a fix for some portion of the cases. But a better fix would either need to significantly modify the way things are laid out to reduce the odds of false positives, or else move to a precise GC. (Precise GCs are possible in C-like languages, but trickier to implement. Here's a recent paper on one: http://www.cs.utah.edu/~r…

Brief HN discussion on that paper: http://news.ycombinator.com/item?id=586858

Re: Go: Severe memory problems on 32bit Linux

#25

Post by kevinpet »

r00tbeer wrote: post #21
_delirium wrote: post #3

This is the relevant bug: http://code.google.com/p/go/issues/detail?id=909 As the discussion there and in the thread says, the root problem is that Go uses a conservative garbage collector, and on 32-bit a lot more values look like pointers than on 64-bit, so many more things don't get freed in long-running processes. Seems not to be easy to fix.

I know early JVM garabage collectors were also "conservative", but I don't recall JVMs running into these sorts of problems. Maybe folks are just using more of the 4G address space these days than back in the day of conservative GC JVMs?

I've never heard of a JVM that uses this kind of GC. As far as I know, every Sun JVM has used explicit "this is a pointer" to identify things for GC.

Re: Go: Severe memory problems on 32bit Linux

#26

Post by rand_r »

smanek wrote: post #19
jedbrown wrote: post #14

However, it does not address the base issue, which is that Go uses a conservative garbage collector, and more values look like pointers in a 32-bit world. The only real fix would be to improve the garbage collector's understanding of which values are pointers and which are something else (e.g., floating point numbers that happen to look like pointers). And that is not an easy fix. How does this GC work? Is it literal…

Yep, that's basically it. It sounds crazy (and it is!) - but it often works reasonably well in practice. SBCL (one of the most performant Common Lisp implementations) has an 'imprecise gc' that works the same way - and I've seen reasonably heavily stressed processes with uptimes in the weeks/months. Remember, even a few years ago (before fastthread, etc) a reasonably loaded Ruby on Rails app couldn't stay up for more…

Would that gc scheme potentially cause the following (pseudo) code to break?

  x = malloc(1); // allocates block 'a' in memory
  int i = (int) x;
  x = 0;
  i = i - 1;
  // gc runs here and frees 'a'
  *( (int*)(i + 1) ) = 123; // failure

Re: Go: Severe memory problems on 32bit Linux

#27

Post by 0xABADC0DA »

It's amazing how nonchalant they are about this 'oh sure we'll fix it in a year or so' when the garbage collector is basically all there needs to be in a Google Go runtime

Some comments were asking why early Java's GC wasn't this bad even though it was conservative. The reason is that Java can't take references to fields of an object, so the data mistaken as a pointer has to actually point to an object header. In Google Go you can take a reference to a field, locking the whole object, so the faux pointer can point to any field as well (or in this probably any location in the object). Not exactly the wisest choice in semantics, as they are seeing now that it complicates the GC.

Re: Go: Severe memory problems on 32bit Linux

#28

Post by pcwalton »

rand_r wrote: post #26
smanek wrote: post #19

Earlier quoted context omitted.

Yep, that's basically it. It sounds crazy (and it is!) - but it often works reasonably well in practice. SBCL (one of the most performant Common Lisp implementations) has an 'imprecise gc' that works the same way - and I've seen reasonably heavily stressed processes with uptimes in the weeks/months. Remember, even a few years ago (before fastthread, etc) a reasonably loaded Ruby on Rails app couldn't stay up for more…

Would that gc scheme potentially cause the following (pseudo) code to break? x = malloc(1); // allocates block 'a' in memory int i = (int) x; x = 0; i = i - 1; // gc runs here and frees 'a' *( (int*)(i + 1) ) = 123; // failure

You'd have to insert a call to "malloc" to get the GC to run, but yes, it can break in that situation. (The classic example is an XOR-packed linked list being corrupted by the Boehm GC.)

But note that isn't really fair to conservative GC, as no GC, precise or not, can cope with hidden pointers like that. So your example isn't really a strike against Go.

Re: Go: Severe memory problems on 32bit Linux

#29

Post by smanek »

eblume wrote: post #23
smanek wrote: post #19

Earlier quoted context omitted.

Yep, that's basically it. It sounds crazy (and it is!) - but it often works reasonably well in practice. SBCL (one of the most performant Common Lisp implementations) has an 'imprecise gc' that works the same way - and I've seen reasonably heavily stressed processes with uptimes in the weeks/months. Remember, even a few years ago (before fastthread, etc) a reasonably loaded Ruby on Rails app couldn't stay up for more…

That's really interesting - is there a place where I can read more about these early Rails performance issues?

There was some back-and-forth between Zed Shaw and DHH that I remember being interested by at the time. Zed deleted all his posts - but here are some places to get started:

- Shaw's opening volley: http://web.archive.org/web/20080103072111/http://www.zedshaw... [lots of stupid personal flames - but his technical points are consistent with what I remember from the time]

- DHH's response: http://david.heinemeierhansson.com/posts/31-myth-2-rails-is-...

- I can't find a copy of shaw's actual response, but here's the relevant HN thread: http://news.ycombinator.com/item?id=364659

The salient point (quoting Zed):

"""

Now, DHH tells me that he’s got 400 restarts a mother fucking day. That’s 1 restart about ever 4 minutes bitches. These restarts went away after I exposed bugs in the GC and Threads which Mentalguy fixed with fastthread (like a Ninja, Mentalguy is awesome).

If anyone had known Rails was that unstable they would have laughed in his face. Think about it further, this means that the creator of Rails in his flagship products could not keep them running for longer than 4 minutes on average.

Repeat that to yourself. “He couldn’t keep his own servers running for longer than 4 minutes on average.”

"""

I've never been much of Ruby/Rails guy - but my understanding is that everyone had to restart Rails a few times an hour cause the memory leaks were so bad in those days.

Re: Go: Severe memory problems on 32bit Linux

#30

Post by smanek »

rand_r wrote: post #26
smanek wrote: post #19

Earlier quoted context omitted.

Yep, that's basically it. It sounds crazy (and it is!) - but it often works reasonably well in practice. SBCL (one of the most performant Common Lisp implementations) has an 'imprecise gc' that works the same way - and I've seen reasonably heavily stressed processes with uptimes in the weeks/months. Remember, even a few years ago (before fastthread, etc) a reasonably loaded Ruby on Rails app couldn't stay up for more…

Would that gc scheme potentially cause the following (pseudo) code to break? x = malloc(1); // allocates block 'a' in memory int i = (int) x; x = 0; i = i - 1; // gc runs here and frees 'a' *( (int*)(i + 1) ) = 123; // failure

I don't think any GC scheme could work with that. Languages that allow pointer arithmetic like that basically can't be GC'd.

The problem with an imprecise GC is that having an integer that looks like a pointer, could prevent an object from being freed.

Post reply on HN