Live data from Hacker News

Building a Better Go Linker

golang.org

81–90 of 104 posts

Re: Building a Better Go Linker

#81

The article often mentions how the long-lived objects are straining the garbage collector. Wouldn't a generational GC solve this?

Potentially, but then the linker is just one use case for the GC and GCs are a game of trade offs so switching to a generational GC would impact other use cases.

Re: Building a Better Go Linker

#83
post #63
post #53

Earlier quoted context omitted.

What you vaguely label as poor design decisions are actually trade-offs and they make sense to a plethora of highly educated engineers.

The people who made golang are not language designers, and/or did not research established options in other language as per their admittance.

Ken Thompson wrote B which is the direct predecessor of C. He was and is certainly a language designer and implementer.

https://en.wikipedia.org/wiki/B_(programming_language)

Re: Building a Better Go Linker

#84

I don't get it. A linker's task should be straightforward. In essence, it looks up addresses from strings, whose number is bounded by the lines of code written by humans. I think that there must be a lot of incidental complexity if that task somehow becomes a bottleneck. And how can it be that a binary called "cmd/compile" has 170k symbols (that's like, global definitions, right?). Not that that's a huge number in te…

I don’t know how advanced the go linker is, but the distinction between compiler and linker gets blurred when one adds link-time optimization (https://llvm.org/docs/LinkTimeOptimization.html, https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html, https://en.m.wikipedia.org/wiki/Interprocedural_optimization)

Even if it doesn’t do that, fixing up various addresses can be a lot of work if the linker can make code shorter (e.g. by using short branches where possible), or if it tries to increase cache locality by changing function order.

Re: Building a Better Go Linker

#85

I don't get it. A linker's task should be straightforward. In essence, it looks up addresses from strings, whose number is bounded by the lines of code written by humans. I think that there must be a lot of incidental complexity if that task somehow becomes a bottleneck. And how can it be that a binary called "cmd/compile" has 170k symbols (that's like, global definitions, right?). Not that that's a huge number in te…

> I don't get it. A linker's task should be straightforward.

In Go the linker also generates DWARF, does deadcode elimination and a bunch of other stuff described in the associated document.

> And how can it be that a binary called "cmd/compile" has 170k symbols (that's like, global definitions, right?). Not that that's a huge number in terms of today's computing power, but how many millions of lines of source code does that correspond to?

As described in the link each global function actually ends up producing 4 symbols.

> Shouldn't 1 or 2 secs be sufficient?

On my system linking cmd/compile takes 1.3 seconds. The problem is that object files get cached so if you have a hot cache (let's say you made a few changes inside a single package and then recompile) compiling takes almost no time and linking accounts for nearly 100% of the build time.

Re: Building a Better Go Linker

#86
post #57
post #52

Earlier quoted context omitted.

The same people who created a language with poor design decisions where superior solutions existed in other languages they could have simply used?

Imagine what Ken Thompson could have accomplished if he hadn't made all those poor design decisions! And yet hundreds of thousands of working programmers around the world are productively using Go while still continuing to ignore the supposedly superior solutions.

>hundreds of thousands of working programmers around the world are productively using Go

Source needed. Even the Go surveys only have mere thousands of respondants. Just because you're stuck in a Go bubble does not mean the world outside is the same.

> still continuing to ignore the supposedly superior solutions.

Go was literally made because Google said their junior employees are not able to understand more powerful languages. Go is a dumbed down language so that their armies of developers can be efficient code monkeys.

Thousands of people are using excel macros and being very productive. That doesn't mean it's a good language either.

Re: Building a Better Go Linker

#87
post #26

What I would give for the developers of the Go toolchain to have spent the last decade improving GCC or LLVM instead of their own bespoke toolchain. In many ways Go seems like an excuse for Google to fund the continued development of Plan 9. Three of the five most influential people on the Go team (Ken Thompson, Rob Pike, and Russ Cox) were heavily involved in Plan 9. And it shows. Go's toolchain is a direct descenda…

I see lots of false assertions here. Here's a significant one:

> There are so many optimizations that will likely never be added to gc, like a register-based calling convention.

The calling conventions was marked as undefined as a first step to changing it [1]. The change was introduced by the very author of this post.

Also, a more agressive inlining mitigates the slow calling conventions.

More generally, I think it is a huge achievement for a PL to be self hosting. It makes its development easier as there is only one language to deeply know (plus assembly of course).

[1] https://github.com/golang/go/issues/27539

Re: Building a Better Go Linker

#88
post #77
post #62

Earlier quoted context omitted.

Appeal to authority fallacy. People still use C, what's your point? There are superior options, but people are either (1) forced to use something inferior, or (2) don't know any better (especially if they drank the kool aid).

> people are either (1) forced to use something inferior, or (2) don't know any better Here's another fallacy for you: false dilemma.

I agree with you and am nitpicking only for my own knowledge: isn’t it “False “Dichotomy”?

Re: Building a Better Go Linker

#89
post #85

I don't get it. A linker's task should be straightforward. In essence, it looks up addresses from strings, whose number is bounded by the lines of code written by humans. I think that there must be a lot of incidental complexity if that task somehow becomes a bottleneck. And how can it be that a binary called "cmd/compile" has 170k symbols (that's like, global definitions, right?). Not that that's a huge number in te…

> I don't get it. A linker's task should be straightforward. In Go the linker also generates DWARF, does deadcode elimination and a bunch of other stuff described in the associated document. > And how can it be that a binary called "cmd/compile" has 170k symbols (that's like, global definitions, right?). Not that that's a huge number in terms of today's computing power, but how many millions of lines of source code d…

One nice speed improvement for linking C++ (and I imagine C as well) with GCC and Clang is -gsplit-dwarf which does not link debug information but instead only references the original object files.

This of course only makes sense for development where the object files are still available for GDB to load the debug symbols from but it is a nice feature.

The MSVC linker has /DEBUG:FASTLINK as well.

Re: Building a Better Go Linker

#90
post #83
post #63

Earlier quoted context omitted.

The people who made golang are not language designers, and/or did not research established options in other language as per their admittance.

Ken Thompson wrote B which is the direct predecessor of C. He was and is certainly a language designer and implementer. https://en.wikipedia.org/wiki/B_(programming_language)

He falls under the "and/or did not research established options in other language as per their admittance" part of my previous statement. Not to mention that B nor C encompass lessons learned since their design.
Post reply on HN