Live data from Hacker News

The State of Go: Where we are in February 2016

talks.golang.org

171–180 of 224 posts

Re: The State of Go: Where we are in February 2016

#171
post #150

Earlier quoted context omitted.

OK, the tradeoff they're making that I missed is that it's not a compacting collector. So eventually your heap can fragment to the point where allocation gets expensive or impossible. Unusual design choice.

Unlike Java Go has first class value types and memory layout can be controlled by developers. So it leads to much less objects on heap and compact layouts both will lead to far less fragmentation. As you can see here Go apps use quite less memory than Java. https://benchmarksgame.alioth.debian.org/u64q/go.html

Unfortunately it's impossible to reliably measure the memory usage of Java that way because the JVM will happily prefer to keep allocating memory from the OS rather than garbage collect. It makes a kind of sense: GC has a CPU cost that gets lower the more memory is given to the heap, so if you have spare memory lying around, may as well deploy it to make things run faster.

Of course that isn't always what you want (e.g. desktop apps) ... sometimes you'd rather spend the CPU and minimise the heap size. The latest Java versions on some platforms will keep track of total free system RAM and if some other program is allocating memory quickly, it'll GC harder to reduce its own usage and give back memory to the OS.

In the benchmarks game I suspect there aren't any other programs running at the same time, so Java will go ahead and use all the RAM it can get. Measuring it therefore won't give reasonable results as the heap will be full of garbage.

Value types don't have much to do with fragmentation, if anything they make it worse because embedding a value type into a larger container type results in needing larger allocations that are harder to satisfy when fragmentation gets serious. But ultimately a similar amount of data is going to end up in the heap no matter what. Yes, you can save some pointers and some object headers, so it'll be a bit less. But not so much that it solves fragmentation.

Re: The State of Go: Where we are in February 2016

#172

I develop a lot of command line tools for Linux using shell scripting. The scripts are getting huge and ugly, so I have been looking at Go and it seems I can do so many things by just using the standard library and in general a big improvement over using scripting. However, everytime Go is discussed at HN I see many posts criticizing the language for various reasons and this has put me off getting started learning Go…

Definitely take a look at Go. If criticism on HN is what you're concerned about, I can tell you that when I started writing Go ~3 years ago it got a lot more criticism on HN than it does now. I've used it as my primary language since then and it's a fun language.

I second taking a look at Go, but be aware that for replacing shell scripts Go has one big disadvantage: binaries tend to get rather large, because all libraries are statically linked.

I have one script that writes postgres, does a HTTP request and needs to read file. 100 locs resulted in 7.2M binary size.

Re: The State of Go: Where we are in February 2016

#173

Earlier quoted context omitted.

Can you speak to why you were behind latest on the JVM? I can think of less than a handful of breaking changes over the last 15 years. I'd say it was more stable than Go.

While I agree that Java has been one of the most stable and backwards-compatible languages out there (especially compared to things like Scala, which regularly breaks source and binary backwards compat), there are still rough edges. I am not the original poster, but I can speak for why JVM deployments take time on Hadoop (which I work on). * In order to upgrade the JVM, all the software you run has to work with the n…

The first issue is surely an issue with Go as well, if they ever do change the language or std lib in a way that isn't perfectly bug-for-bug compatible? As to upgrade Go I thought you have to recompile everything including all dependencies (I guess there's no stabilised ABI?), so it amounts to the same thing: if one of your dependencies doesn't play nice with the new Go, you can't upgrade.

The private APIs thing is indeed one of the most common issues, along with needed upgrades to bytecode rewriting libraries.

Re: The State of Go: Where we are in February 2016

#174

Earlier quoted context omitted.

Could you explain what essential things JVM does better than Go at this point ? Does it stop the world less ? Or does it do more things in parallel ? Thanks

With respect to garbage collection only and ignoring things like reliable debugging support, the primary thing it does is compaction. If your memory manager does not compact the heap (i.e. never moves anything), then this implies a couple of things: 1. You can run out of memory whilst still technically having enough bytes available for a requested allocation, if those bytes are not contiguous. Most allocators bucket…

While I agree that compaction is desirable in theory, empirically it's not really necessary. For example, there are no C/C++ malloc/free implementations that compact, because compaction would change the address of pointers, breaking the C language. Long-lived C and C++ applications seem to get by just fine without the ability to move objects in memory.

Java code also tends to make more allocations than Go code, simply because Java does not (yet) have value types, and Go does. This isn't really anything to do with the GC, but it does mean that Java _needs_ a more powerful GC just to handle the sometimes much greater volume of allocations. It also makes Java programmers sometimes have to resort to hacks like arrays of primitive types (I've done this before).

People like to talk about how important generational GC is, and how big a problem it is that Go doesn't have it. But I have also seen that if there is too high a volume of data in the young-gen in Java, short-lived objects get tenured anyway. In practice, the generational assumption isn't always true. If you use libraries like Protobuffers that create a ton of garbage, you can pretty easily exceed the GC's ability to keep up with short-lived garbage.

I'm really curious to see how Go's GC works out for big heaps in practice. I can say that my experience with Java heaps above 100 GB has not been good. (To be fair, most of my Java experience has been under CMS, not the new G1 collector.)

Re: The State of Go: Where we are in February 2016

#175
post #38
post #4

{{range . -}} {{.}} {{end -}} This seems like a bit of a hack to be honest. Would anything break if {{range .}} {{.}} {{end}} worked as expected?

The problem then is what whitespace do you keep and what do you ditch? The default behavior you want is to maintain everything inside. But I agree the - tag seems like a really inelegant hack.

The default behavior has to be backwards compatible, so there was only one serious option.

Re: The State of Go: Where we are in February 2016

#176

A Golang beginner's question: do these GC improvements make Golang a suitable language/ platform for writing games? EDIT: I realise this is a vague question. I suppose I was wondering if the order of magnitude GC performance in Go is likely to interfere with game loops you might find in reasonably CPU/ GPU intensive Indie games (i.e. NOT Crysis).

You have to ensure that the GC never makes you drop a frame. For 60Hz, that means staying below 16.7ms. Given that a Go 1.6 GC will still take about 4ms, you have 12.7ms to generate a frame, which can be too limiting for some CPU-intensive games, but is perfectly acceptable for many games. (In Go 1.5, a GC was much more likely to make you drop a frame, as it could easily average 40ms.) On the other hand, there are le…

Another weird thing is, the background collection is still using a core (plus somewhat slowing foreground code, with write barriers) when it runs. It's kinda like you have a varying amount of CPU power.

One approach is just to program as if you had less CPU, as if there were always a GC running. I suppose if you have some code that isn't smoothness critical (game AI, say), or CPU-affecting detail settings you can twiddle without looking too glitchy, maybe you can figure out some way to shed work when you start to fall behind.

It'd be really cool to see someone attempt a game or such in Go--boundary pushing's always fun, more so when the boundaries are recently expanded.

Re: The State of Go: Where we are in February 2016

#177

Earlier quoted context omitted.

While I agree that Java has been one of the most stable and backwards-compatible languages out there (especially compared to things like Scala, which regularly breaks source and binary backwards compat), there are still rough edges. I am not the original poster, but I can speak for why JVM deployments take time on Hadoop (which I work on). * In order to upgrade the JVM, all the software you run has to work with the n…

The first issue is surely an issue with Go as well, if they ever do change the language or std lib in a way that isn't perfectly bug-for-bug compatible? As to upgrade Go I thought you have to recompile everything including all dependencies (I guess there's no stabilised ABI?), so it amounts to the same thing: if one of your dependencies doesn't play nice with the new Go, you can't upgrade. The private APIs thing is i…

The difference is that you can upgrade each Go application to Go 1.6 separately, whereas with Java, once you upgrade the JVM, all of your applications get upgraded at once. While you could technically have two different JVMs installed side-by-side, in practice nobody actually does this because of the operational complexity and the way that Java handles dependencies. Perhaps this will become an issue for Go if people start using the new shared library feature. For Java, _everything_ is a shared library (sort of), so this is an omnipresent issue.

I think both Go and Java have been good about avoiding backwards incompatible changes in the standard library. Both languages have an explicit policy of avoiding these whenever it is at all possible.

Frankly, the Go standard library is a lot better written than the Java one. For example, I dare you to figure out how to call statvfs from Java, or figure out how many hardlinks there are to a specific file inode. Or make an asynchronous DNS lookup. Even simple things like creating a socket without doing a DNS lookup are very difficult to achieve in the Java standard library.

Re: The State of Go: Where we are in February 2016

#178
post #175
post #38

Earlier quoted context omitted.

The problem then is what whitespace do you keep and what do you ditch? The default behavior you want is to maintain everything inside. But I agree the - tag seems like a really inelegant hack.

The default behavior has to be backwards compatible, so there was only one serious option.

What? They could have done any number of things. They could have added a parameter "-trim" that's more clear than the "-" alone. They could have made it "8<" that looks like a little pair of scissors. Or they could have actually spent some time thinking of a good idea instead of those ones that are about as a bad as the one they went with.

Re: The State of Go: Where we are in February 2016

#179
post #153

Earlier quoted context omitted.

You're misinformed, I suspect because you are conflating .NET programs being distributed as bytecode with .NET programs being interpreted. It is true that .NET programs are traditionally distributed as CLR bytecode. This is similar to a .class file or a .pyc file. But the CLR does now, and always has, had a JIT. E.g., the first line or two of https://msdn.microsoft.com/en-us/library/ht8ecch6(v=vs.71).a... , which is…

> You're misinformed, I suspect because you are conflating .NET programs being distributed as bytecode with .NET programs being interpreted. No and no. > But the CLR does now, and always has, had a JIT. And Python has had a JIT[0] for as long as the framework has existed. [0] https://en.wikipedia.org/wiki/Psyco now replaced by the pypy project

Python doesn't have a JIT by default (CPython, the reference implementation). Does the same apply to .NET?

Re: The State of Go: Where we are in February 2016

#180
post #153

Earlier quoted context omitted.

You're misinformed, I suspect because you are conflating .NET programs being distributed as bytecode with .NET programs being interpreted. It is true that .NET programs are traditionally distributed as CLR bytecode. This is similar to a .class file or a .pyc file. But the CLR does now, and always has, had a JIT. E.g., the first line or two of https://msdn.microsoft.com/en-us/library/ht8ecch6(v=vs.71).a... , which is…

> You're misinformed, I suspect because you are conflating .NET programs being distributed as bytecode with .NET programs being interpreted. No and no. > But the CLR does now, and always has, had a JIT. And Python has had a JIT[0] for as long as the framework has existed. [0] https://en.wikipedia.org/wiki/Psyco now replaced by the pypy project

.NET is JIT'd in its most popular form, the .NET Framework.

Python is interpreted in what was its most popular form, but may not be now, CPython.

This is using the definition of a JIT as an execution engine which takes in some form of bytecode and, at runtime, emits architecture-specific assembly code to a page, marks that page executable, and changes the IP to that page.

If CPython executes in that manner then I'm mistaken about CPython's execution engine and would also consider CPython to be a JIT.

Post reply on HN