The State of Go: Where we are in February 2016 • Hacker News Board

Re: The State of Go: Where we are in February 2016

#201

Post by pjmlp » Thu, Feb 04, 2016, 7:59 AM UTC

heinrich5991 wrote: post #179

masklinn wrote: post #165

Earlier quoted context omitted.

> You're misinformed, I suspect because you are conflating .NET programs being distributed as bytecode with .NET programs being interpreted. No and no. > But the CLR does now, and always has, had a JIT. And Python has had a JIT[0] for as long as the framework has existed. [0] https://en.wikipedia.org/wiki/Psyco now replaced by the pypy project

Python doesn't have a JIT by default (CPython, the reference implementation). Does the same apply to .NET?

.NET as distributed by Microsoft:

- JIT and AOT compilation via NGEN up to .NET 4.5.2

- Starting with .NET 4.6, RyuJIT which uses the Visual C++ backend and exposes SIMD support to .NET languages

- When targeting Windows 8 and 8.1, AOT compilation to native code in a format called MDIL. Basically requires dynamic linking on device, everything else will be native code already

- When targeting Window 10 store applications onwards, AOT compilation to static executables

- .NET Compact Framework also always JITs

- .NET Micro Framework is the only one that does interpret MSIL

Also Microsoft .NET JIT compilers, with the exception of the .NET Micro Framework always jit the code, there is no threshold to trigger it like on most JVMs.

Re: The State of Go: Where we are in February 2016

#202

Post by pjmlp » Thu, Feb 04, 2016, 8:09 AM UTC

geodel wrote: post #108

pcwalton wrote: post #94

Earlier quoted context omitted.

Latency isn't the only concern; you also have to look at throughput. The JVM's GC has been carefully tuned to strike a balance here. In particular, Go's GC is not yet generational from the talks I've seen, which is a large throughput loss compared to the GC of the JVM.

If it is carefully tuned why it needs such a big GC tuning guide and 100s of JVM flags to tune runtime. Any Java product of consequence comes with custom GC settings meaning they do not find default ones suitable. https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gc...

Just like C and C++ compilers have lots of options to tune all generated code.

Sometime just -O2 or -O3 aren't not enough, regardless how much tuning has gone into them.

Re: The State of Go: Where we are in February 2016

#203

Post by pjmlp » Thu, Feb 04, 2016, 8:13 AM UTC

pjspycha wrote: post #193

jmgao wrote: post #182

Earlier quoted context omitted.

Does go even have any good command-line debuggers? All of the things I've seen either involve insanity like preprocessing your source to inject hooks in between each line of code, or don't actually work.

Delve is an open source debugger for go. And many editors can work with delve such as Visual Studio Code. There is an animated gif of the debugger on the vscode-go page. https://github.com/derekparker/delve https://github.com/Microsoft/vscode-go

Due the way Delve rewrites Go source code and relies on having it available, I wouldn't call it a debugger.

Re: The State of Go: Where we are in February 2016

#204

Post by pjmlp » Thu, Feb 04, 2016, 8:17 AM UTC

geodel wrote: post #157

blakeyrat wrote: post #139

Earlier quoted context omitted.

Great tooling? Does Go have any quality IDEs with integrated debuggers yet? I feel that open source developers mean something entirely different with the phrase "great tooling" than developers used to Visual Studio would mean. :)

Absolutely! They do not mean big GUI based IDE when they say tooling. Infact this is why Go will remain unviable option to .net developers. Go is likely to be much more popular among dynamic languages users and even some Java developers who are tired of enterprisey bloat.

> Infact this is why Go will remain unviable option to .net developers.

Why should we even bother with Go, if we have C#, F#, JIT, AOT compilation to dynamic binaries (NGEN), AOT compilation to static binaries (.NET Native), NuGET already available?

And yes, I do know Go and even tried some early contributions before the 1.0 release.

For me it is only a step forward for C developers.

Still a very important one, as we need more widespread use of safer compiled languages.

Re: The State of Go: Where we are in February 2016

#205

Post by nitwit005 » Thu, Feb 04, 2016, 9:34 AM UTC

colin_mccabe wrote: post #170

kasey_junk wrote: post #89

Earlier quoted context omitted.

Can you speak to why you were behind latest on the JVM? I can think of less than a handful of breaking changes over the last 15 years. I'd say it was more stable than Go.

While I agree that Java has been one of the most stable and backwards-compatible languages out there (especially compared to things like Scala, which regularly breaks source and binary backwards compat), there are still rough edges. I am not the original poster, but I can speak for why JVM deployments take time on Hadoop (which I work on). * In order to upgrade the JVM, all the software you run has to work with the n…

> This is not an issue with Go because everything is compiled to an x86_64 binary there

I don't think the situation is all that different. All the Java code ends up being binary as well, at least after running a while.

At the moment, I think most people's Go projects simply have fewer dependencies. If you poke about on some of the popular Go projects, you'll see some of the bad coding practices that will result in upgrade breakages like checking for exact error strings (unfortunately needed sometimes, I know), so I foresee the same issues over time.

Re: The State of Go: Where we are in February 2016

#206

Post by mike_hearn » Thu, Feb 04, 2016, 9:37 AM UTC

colin_mccabe wrote: post #177

mike_hearn wrote: post #173

Earlier quoted context omitted.

The first issue is surely an issue with Go as well, if they ever do change the language or std lib in a way that isn't perfectly bug-for-bug compatible? As to upgrade Go I thought you have to recompile everything including all dependencies (I guess there's no stabilised ABI?), so it amounts to the same thing: if one of your dependencies doesn't play nice with the new Go, you can't upgrade. The private APIs thing is i…

The difference is that you can upgrade each Go application to Go 1.6 separately, whereas with Java, once you upgrade the JVM, all of your applications get upgraded at once. While you could technically have two different JVMs installed side-by-side, in practice nobody actually does this because of the operational complexity and the way that Java handles dependencies. Perhaps this will become an issue for Go if people…

I really don't get this at all. The JVM is a program. It sits in a single directory. I routinely have several installed on my laptop. There are no operational complexities from having multiple different versions installed, if you want that.

I suspect this really boils down to operational complexities of Linux distros, not Java. If your package manager only lets you install one JVM then maybe this seems "complicated" relative to Go, but that's not a Java problem.

WRT the standard library, yes the Java standard library doesn't expose UNIX specific syscalls. It exposes stuff at a higher level instead because it tries to be portable. That's a different tradeoff to what Go makes but I wouldn't say that makes it badly written. To me badly written would mean buggy, confusingly designed, too small or too big etc. If you want to write non-portable software then that means you may have to link in an extra library or so (like JNA).

Creating sockets does not do DNS lookups in Java. You may be thinking of the URL class, which does, and there's a URI class that avoids that.

Re: The State of Go: Where we are in February 2016

#207

Post by mike_hearn » Thu, Feb 04, 2016, 9:43 AM UTC

colin_mccabe wrote: post #174

mike_hearn wrote: post #145

Earlier quoted context omitted.

With respect to garbage collection only and ignoring things like reliable debugging support, the primary thing it does is compaction. If your memory manager does not compact the heap (i.e. never moves anything), then this implies a couple of things: 1. You can run out of memory whilst still technically having enough bytes available for a requested allocation, if those bytes are not contiguous. Most allocators bucket…

While I agree that compaction is desirable in theory, empirically it's not really necessary. For example, there are no C/C++ malloc/free implementations that compact, because compaction would change the address of pointers, breaking the C language. Long-lived C and C++ applications seem to get by just fine without the ability to move objects in memory. Java code also tends to make more allocations than Go code, simpl…

Experience with C/C++ is exactly why people tend to value compaction. I've absolutely encountered servers and other long-lived apps written in C++ that suffer from heap fragmentation, and required serious attention from skilled developers to try and fix things (sometimes by adding or removing fields from structures). It can be a huge time sink because the code isn't actually buggy and the problem is often not easily localised to one section of code. It's not common that you encounter big firefighting efforts though, because often for a server it's easier to just restart it in this sort of situation.

As an example, Windows has a special malloc called the "low fragmentation heap" specifically to help fight this kind of problem - if fragmentation was never an issue in practice, such a feature would not exist.

CMS was never designed for 100GB+ heaps so I am not surprised your experience was poor. G1 can handle such heaps although the Intel/HBase presentation suggested aiming for more like 100msec pause times is reasonable there.

The main thing I'd guess you have to watch out for with huge Go heaps is how long it takes to complete a collection. If it's really scanning the entire heap in each collection then I'd guess you can outrun the GC quite easily if your allocation rate is high.

Re: The State of Go: Where we are in February 2016

#208

Post by nyan4 » Thu, Feb 04, 2016, 12:07 PM UTC

notacoward wrote: post #191

cyphar wrote: post #168

Earlier quoted context omitted.

The fact that you can create an online interface and an unrelated library (which you didn't write) can have structures that implement your interface automatically is pretty useful (and cool) IMHO.

Fair point, but much of that benefit is lost without decent support for those third-party implementations to be loaded dynamically. So Nim is missing one piece, but those other features I mentioned can get you very close to the same place. Go is missing the other piece, with no really good way to make up for it. That problem's not even solvable as long as Go's runtime makes no provision for interfacing to code that d…

> Go's runtime makes no provision for interfacing to code that doesn't play by its own

This alone makes Go really unfit for developing system libraries.

Re: The State of Go: Where we are in February 2016

#209

Post by vardump » Thu, Feb 04, 2016, 2:06 PM UTC

mike_hearn wrote: post #207

colin_mccabe wrote: post #174

Earlier quoted context omitted.

While I agree that compaction is desirable in theory, empirically it's not really necessary. For example, there are no C/C++ malloc/free implementations that compact, because compaction would change the address of pointers, breaking the C language. Long-lived C and C++ applications seem to get by just fine without the ability to move objects in memory. Java code also tends to make more allocations than Go code, simpl…

Experience with C/C++ is exactly why people tend to value compaction. I've absolutely encountered servers and other long-lived apps written in C++ that suffer from heap fragmentation, and required serious attention from skilled developers to try and fix things (sometimes by adding or removing fields from structures). It can be a huge time sink because the code isn't actually buggy and the problem is often not easily…

It's true heaps can be a pain with C/C++. 64-bit is pretty ok, it's rare to have any issues.

32-bit is painful and messy. If possible, one thing that may help is to allocate large (virtual memory wise) objects once in the beginning of a new process and have separate heaps for different threads / purposes. Not only heap fragmentation can be issue, but also virtual memory fragmentation. Latter is usually what turns out to be fatal. One way to mitigate issues with multiple large allocations is to change memory mapping as needed... Yeah, it can get messy.

64-bit systems are way easier. Large allocations can be handled by allocating page size blocks of memory from OS (VirtualAlloc / mmap). OS can move and compact physical memory just fine. At most you'll end up with holes in the virtual memory mappings, but it's not a real issue with 64 bit systems.

Small allocations with some allocator that is smart enough to group allocations by 2^n size (or do some other smarter tricks to practically eliminate fragmentation).

Other ways are to use arenas or multiple heaps. For example per thread or per object.

There are also compactible heaps. You just need to lock the memory object before use to get a pointer to it and unlock when you're done. The heap manager is free to move the memory block as it pleases, because no one is allowed to have a pointer to the block. Harder to use, yes, but hey, no fragmentation!

Yeah, Java is better in some ways for being able to compact memory always. That said, I've also cursed it to hell for ending up in practically infinite gc loop when used memory is nearing maximum heap size.

There's no free lunch in memory management.

Re: The State of Go: Where we are in February 2016

#210

Post by igouy » Thu, Feb 04, 2016, 4:17 PM UTC

geodel wrote: post #150

mike_hearn wrote: post #143

Earlier quoted context omitted.

OK, the tradeoff they're making that I missed is that it's not a compacting collector. So eventually your heap can fragment to the point where allocation gets expensive or impossible. Unusual design choice.

Unlike Java Go has first class value types and memory layout can be controlled by developers. So it leads to much less objects on heap and compact layouts both will lead to far less fragmentation. As you can see here Go apps use quite less memory than Java. https://benchmarksgame.alioth.debian.org/u64q/go.html

Those default memory use measurements are just a way to check if a particular 100 line toy benchmark program has been written to exploit time / space trade-off.