Go: Severe memory problems on 32bit Linux • Hacker News Board

Re: Go: Severe memory problems on 32bit Linux

#151

Post by moilolita » Sat, Apr 07, 2012, 1:06 AM UTC

he_the_great wrote: post #137

moilolita wrote: post #47

Earlier quoted context omitted.

If Ceylon/Rust are not ready today it means they will be immature for at least 2 years from their release date. Without a large community behind them, they will slowly fade away like D did.In order to get community, they have to have a good/clean/pragmatic design and a library as comprehensive as possible. If they don't have a good library they better have excellent interop with other platforms or they will be doomed…

> Without a large community behind them, they will slowly fade away like D did. This is a rather strange assessment. The history of "successful" languages has been a mixture of "cool jump onboard" and "who can stay alive the longest to get a community." D is in the latter camp. It seems to me, unreasonable to expect a language to be coming out the gate with guns blazing. People expect Nukes now!

The market will decide. As someone said on HN: the best language in the world would be a mix of Go and D and would be called GoD. When there were few options, yes, it was enough to survive long enough to get a community. But when you have plenty of similar options, some of them backed by some major actors (at least in the early stages), i think no one will sacrifice his productivity (other than for hobby projects) for the sake of one language. I really hope D will develop into something, but looking to various sources (abandoned projects for D1 on dsource.org) on the net and google results, it seemed to me that D peaked somewhere around 2007-2008 with a slight revival in 2010 when D2 was released.

Re: Go: Severe memory problems on 32bit Linux

#152

Post by agentS » Sat, Apr 07, 2012, 4:05 AM UTC

jerf wrote: post #104

ghusbands wrote: post #87

Earlier quoted context omitted.

On 32-bit systems, the problem is that there's a very high chance of non-pointer data looking like pointers to existing data, which, in turn might have its own pointer-like data. This means that you can end up keeping an excessive amount of unreferenced data. However, a 64-bit address space is so much larger that you can't really suffer the same issue. Unlike on 32-bit systems, neither high entropy data nor text will…

Hmmmmm... I'm smelling memory-based DoS if I, say, upload carefully crafted files to a webserver. What accidentally happens in 32-bit may be deliberately triggered in 64-bit.

This is impossible. I don't like to make absolute statements, but I'll stand by this one.

In order to pull of an attack, you'd need to know what address range the program in question has been allocated, then figure out the smaller range that the runtime is actively using, then give it data with integers in that range. This is impractical.

If you think you can pull it off play.golang.org lets you upload text to a Go program on Appengine, then it compiles that program and runs it. This gives you 2 programs to attack, the playground binary, and the one compiled from your source. If you can do it, you'll have a way to kill machines inside Google.

Good luck.

Re: Go: Severe memory problems on 32bit Linux

#153

Post by agentS » Sat, Apr 07, 2012, 4:20 AM UTC

stiff wrote: post #93

dchest wrote: post #75

Earlier quoted context omitted.

There is an "official" response: the post references a bug # where Russ Cox said "the rest of the issue will have to wait until after Go 1". Note that Go 1 is a "language freeze", not the implementation freeze. Now that it's known there's a bug that will be worked on later, people proposed possible workarounds, the easiest of which is to switch to 64-bit platform. I agree that downplaying the issue is wrong, but only…

The very next sentence in this response from Russ Cox is: "Or maybe all the 32-bit systems will be replaced by 64-bit ones.". So, they do not admit this is a serious problem that needs attention, it is not clear whether they will fix it, when they are going to fix and whether anyone cares about fixing it at all. I couldn't find the adequate words to express this so far, but the reason I at all find this situation wor…

I don't think Russ denied it was a problem, so I don't think this is a "pattern of denial". I think you're incorrectly interpreting that statement as his "fix" for this bug.

The fix is well-known (a precise GC), just implementing it hasn't happened yet.

Re: Go: Severe memory problems on 32bit Linux

#154

Post by agentS » Sat, Apr 07, 2012, 4:31 AM UTC

0xABADC0DA wrote: post #144

burgerbrain wrote: post #143

Earlier quoted context omitted.

The "clearing up ambiguity" thing is bullshit. Nobody is going to fail to use context clues to figure out what is meant by "Go" in this discussion. Where such disambiguation is actually needed, "Go (language)" or simply "go-lang"/"golang" is preferred. You know, since they are not incorrect . "Frankly I fail to understand why this bothers you so much, as it clearly does." It is annoying because it is incorrect, becau…

For instance: http://www.cnn.com/2011/TECH/mobile/05/12/google.maps.androi... Or: http://en.wikipedia.org/wiki/Google_Maps > It is annoying because it is incorrect Maybe some English professor, technical writer, or journalist reading this can chime in and explain how so. It seems to be the standard practice and I intend to use the best grammar and construction that I am capable of, as bad as that may be. This isn't T…

An answer, by way of analogy.

When referring to a friend named "Edward" in a text to another friend planning Edward's surprise birthday party, you can probably refer to him as "Ed" or "Edward". You definitely don't need to refer to him as "Edward (Parent's SSN:12345...)" or "Edward (Philip's Son)". While these latter forms are less ambiguous (and more searchable, to boot), the context is more than sufficient to disambiguate.

Re: Go: Severe memory problems on 32bit Linux

#155

Post by ungerik » Sat, Apr 07, 2012, 2:12 PM UTC

After switching to 64 bit, we haven't had a single crash. So I would describe the 64 bit version as production ready. But the problems on 32 bit systems should have been documented in the release notes.

Beside this hiccup (and a week without sleep with the website on life support), Go has been a real joy to work with.

Re: Go: Severe memory problems on 32bit Linux

#156

Post by complexmath » Sat, Apr 07, 2012, 3:40 PM UTC

moilolita wrote: post #150

complexmath wrote: post #149

Earlier quoted context omitted.

Did you mean something like core.sync.barrier?

barrier is not a suitable primitive for a WaitGroup. The ideea is simple. You accept sockets in a loop and handle the connections in parallel threads.At one point you want to stop this loop and the main thread must wait for all the active threads to finish before exiting, otherwise some clients may receive "connection reset by peer". With a WaitGroup, every starting thread increments a counter, and every finishing th…

At the program level, this is built in (D has daemon and non-daemon threads like Java). Or you could use ThreadGroup in core.thread. It would be pretty trivial to do this at the user level with messaging as well.

Re: Go: Severe memory problems on 32bit Linux

#157

Post by FeepingCreature » Sat, Apr 07, 2012, 5:41 PM UTC

scott_s wrote: post #90

kevingadd wrote: post #61

Earlier quoted context omitted.

I didn't downvote it, but it mostly read like rambling nonsense to me, so that might be why. Maybe write it so it's clear to people who don't work on memory managers all day?

I've worked on memory allocators, and I can't make sense of it.

I think the idea is to differentiate between mutable and immutable data, ie. "worked on" and "final". I don't know what that would give you with regards to GC though.

Re: Go: Severe memory problems on 32bit Linux

#158

Post by el_muchacho » Sun, Apr 08, 2012, 8:57 AM UTC

moilolita wrote: post #151

he_the_great wrote: post #137

Earlier quoted context omitted.

> Without a large community behind them, they will slowly fade away like D did. This is a rather strange assessment. The history of "successful" languages has been a mixture of "cool jump onboard" and "who can stay alive the longest to get a community." D is in the latter camp. It seems to me, unreasonable to expect a language to be coming out the gate with guns blazing. People expect Nukes now!

The market will decide. As someone said on HN: the best language in the world would be a mix of Go and D and would be called GoD. When there were few options, yes, it was enough to survive long enough to get a community. But when you have plenty of similar options, some of them backed by some major actors (at least in the early stages), i think no one will sacrifice his productivity (other than for hobby projects) fo…

The drop after 2007/2008 probably corresponds to the Phobos/Tango debacle, and the fact that Walter decided to fork with D2.

One of the main problems was, he was almost the sole compiler developer, and could hardly keep up with the tasks of maintaining 2 parallel branches and developing new ideas at the same time. People complained that they couldn't get involved as much as they wanted. It's understandable that many people thought that D didn't have a solid future with such uncertainties.

Nowadays, these problems are mostly overcome with a much better organization: there are several committers for the compiler, and several committers for the standard library. Phobos is the standard library, it's maturing, D2 has shown its strengths over D1 and the community is united again, because not only it is deeply involved with the design of the language and standard lib(through the m-l), it is also involved with the implementation of essential parts of it. 2011 has been a very good year for D, and I think that more than ever, the whole project feels like it's going in the right direction.

edit: I guess another reason D isn't gaining as much traction as it could is, it has been removed from the Alioth computer language shootout. For a language which is aimed at raw speed (and was brilliant at that when it was still on the shootout), it's a severe blow.

Re: Go: Severe memory problems on 32bit Linux

#159

Post by el_muchacho » Sun, Apr 08, 2012, 9:08 AM UTC

sixbrx wrote: post #131

4ad wrote: post #108

Earlier quoted context omitted.

I've been exclusively using 64 bit computers and operating systems, both Windows and Linux, for about 7 years now, never had the reported driver problems, well, never had any issue, really. For me at least, 32 bit is only important for ARM. On the other hand the issue is very much blown out of proportion, most people haven't seen it, even if they run 32 bit servers. Most usual servers written in Go, like web servers,…

I do wish the desktop world would just move wholesale to 64 bit, but do note that Ubuntu still marks the 32 bit install the "recommended" one, so I would guess there must be some sort of problems that are lingering in the 64 bit versions. Maybe just flash support or something like that?

I don't think there are problems with 64 bit versions, but many computers are still 32 bits. The PC I use right now is 32 bits and I don't feel the urge to buy a new one. So I suppose 32 bit machines will stay around for another decade or so.