Earlier quoted context omitted.
Which private OS calls. Can you site what they are?
I'm guessing GP refers to the fact that Go binaries use raw system calls even on operating systems where the syscall boundary is considered a private implementation detail (which is the case on Windows and macOS).
Building a Better Go Linker
41–50 of 104 posts
Re: Building a Better Go Linker
#42What I would give for the developers of the Go toolchain to have spent the last decade improving GCC or LLVM instead of their own bespoke toolchain. In many ways Go seems like an excuse for Google to fund the continued development of Plan 9. Three of the five most influential people on the Go team (Ken Thompson, Rob Pike, and Russ Cox) were heavily involved in Plan 9. And it shows. Go's toolchain is a direct descenda…
> the state of compilers would be much better off if the folks on the Go team had invested more in improving the compile and link times of LLVM or GCC It never would have happened. How do you motivate people whose principal frustration is the state of C++ to work on a large C++ codebase? Heterogeneity is a huge benefit to any ecosystem. Improving existing things is great, but building new things is also very importan…
I agree that enthusiasm is important! And indeed, for the Go creators, their particular leanings might have been such that they couldn't get excited about building an LLVM/GCC frontend, and adapting the Plan 9 toolchain is literally the only way those three could have Go gotten off the ground. As a member of the Go team, you'd certainly know better than I.
But Go is long past a personal passion project. Go is over ten years old. Go likely has over a million developers [0]. Go 1.0 has been stable for about seven years, and the first meaningful changes to the language are just now being talked about. In my opinion, it is several years past due for Google to start investing seriously in a Go toolchain based on a mature compiler stack.
I realize the audacity of this claim and I don't make it lightly. But if I had the money to spend on a team of developers, I would spend it making llvm-as and lld fast enough and stable enough to be Go's assembler and linker, and abandon the custom Plan 9 ones.
> It never would have happened. How do you motivate people whose principal frustration is the state of C++ to work on a large C++ codebase?
Well, for one, once the language gets off the ground, you can write the frontend in the new language. Rustc manages to be almost entirely Rust, for example.
> Heterogeneity is a huge benefit to any ecosystem. Improving existing things is great, but building new things is also very important.
I agree, and I think Go is an interesting contribution to the P/L landscape—essentially it proved that stripping away a good deal of complexity (generics, inheritance, etc.) results in a very useful, highly productive language. But I don't think Go's custom assembler and linker are meaningfully contributing to the ecosystem. They're useful presently in that they improve Go developers' productivity with ultra-fast builds, but they're not suitable for use by anything but Go. Improvements to Go's linker and assembler benefit only Go. Improvements to lld or gold can benefit practically everyone using a compiled language.
Re: Building a Better Go Linker
#43What I would give for the developers of the Go toolchain to have spent the last decade improving GCC or LLVM instead of their own bespoke toolchain. In many ways Go seems like an excuse for Google to fund the continued development of Plan 9. Three of the five most influential people on the Go team (Ken Thompson, Rob Pike, and Russ Cox) were heavily involved in Plan 9. And it shows. Go's toolchain is a direct descenda…
Given that Apple are the major sponsor of LLVM and they're not exactly cash poor, I think it's reasonable to conclude that being able to throw money and people at LLVM development doesn't explain why it's still much slower than the Go compiler toolchain. In hindsight the Go team made the right call to use their own toolchain. As enneff alludes, Ken Thompson's antipathy towards C++ is well documented: https://bryanpen…
Right, you need to specifically throw money and people at the problem of making LLVM faster, not just at LLVM in general. Neither Swift nor Objective-C have "fast compiles" as part of their pitch. Much of the work on LLVM goes into producing the highest quality object code possible, which is a goal often at odds with compiling quickly, and part of the reason the choose-your-optimization-level flag (-O) exists, though -O0 compiles are still not fast enough.
> In hindsight the Go team made the right call to use their own toolchain.
No, we don't have the benefit of hindsight yet. We don't know what could have been if the resources that had been spent on the Go toolchain had been spent on LLVM instead.
If five highly-qualified engineers spent five years trying to speed up gollvm compiles without success, we'd have strong evidence that something about LLVM prohibits the fast compiles that are possible with the gc toolchain. But that's not the situation.
Re: Building a Better Go Linker
#44Earlier quoted context omitted.
> the state of compilers would be much better off if the folks on the Go team had invested more in improving the compile and link times of LLVM or GCC It never would have happened. How do you motivate people whose principal frustration is the state of C++ to work on a large C++ codebase? Heterogeneity is a huge benefit to any ecosystem. Improving existing things is great, but building new things is also very importan…
Let me refine my point a bit. Apologies; my original post had gotten a bit long. I agree that enthusiasm is important! And indeed, for the Go creators, their particular leanings might have been such that they couldn't get excited about building an LLVM/GCC frontend, and adapting the Plan 9 toolchain is literally the only way those three could have Go gotten off the ground. As a member of the Go team, you'd certainly…
> Improvements to lld or gold can benefit practically everyone using a compiled language.
I do not see how a generic framework could be made as fast as custom build stack.
Re: Building a Better Go Linker
#45Earlier quoted context omitted.
I don't think that would be appropriate since Golang isn't (to my knowledge) under the GNU license. Maybe gocc?
I think you may misunderstand the nature of the post: https://golang.org/doc/install/gccgo It's a thing that exists, not a proposal.
I thought it was just spitballing an alternative name.
Re: Building a Better Go Linker
#46Earlier quoted context omitted.
Given that Apple are the major sponsor of LLVM and they're not exactly cash poor, I think it's reasonable to conclude that being able to throw money and people at LLVM development doesn't explain why it's still much slower than the Go compiler toolchain. In hindsight the Go team made the right call to use their own toolchain. As enneff alludes, Ken Thompson's antipathy towards C++ is well documented: https://bryanpen…
> being able to throw money and people at LLVM development doesn't explain why it's still much slower than the Go compiler toolchain. Right, you need to specifically throw money and people at the problem of making LLVM faster, not just at LLVM in general. Neither Swift nor Objective-C have "fast compiles" as part of their pitch. Much of the work on LLVM goes into producing the highest quality object code possible, wh…
At some point people are going to need to put the money where their mouth is. Or they can live with the software which open source developers developed on their own or employers' dime.
Re: Building a Better Go Linker
#47Earlier quoted context omitted.
Given that Apple are the major sponsor of LLVM and they're not exactly cash poor, I think it's reasonable to conclude that being able to throw money and people at LLVM development doesn't explain why it's still much slower than the Go compiler toolchain. In hindsight the Go team made the right call to use their own toolchain. As enneff alludes, Ken Thompson's antipathy towards C++ is well documented: https://bryanpen…
> being able to throw money and people at LLVM development doesn't explain why it's still much slower than the Go compiler toolchain. Right, you need to specifically throw money and people at the problem of making LLVM faster, not just at LLVM in general. Neither Swift nor Objective-C have "fast compiles" as part of their pitch. Much of the work on LLVM goes into producing the highest quality object code possible, wh…
Re: Building a Better Go Linker
#48Earlier quoted context omitted.
> being able to throw money and people at LLVM development doesn't explain why it's still much slower than the Go compiler toolchain. Right, you need to specifically throw money and people at the problem of making LLVM faster, not just at LLVM in general. Neither Swift nor Objective-C have "fast compiles" as part of their pitch. Much of the work on LLVM goes into producing the highest quality object code possible, wh…
> If five highly-qualified engineers spent five years trying to speed up gollvm compiles without success, we'd have strong evidence that something about LLVM prohibits the fast compiles that are possible with the gc toolchain. But that's not the situation. At some point people are going to need to put the money where their mouth is. Or they can live with the software which open source developers developed on their ow…
I did learn enough in those six weeks to feel comfortable asserting that there is incredible potential in gccgo/gollvm, and that I think Google is making a mistake by continuing to invest so heavily in the gc toolchain.
[0]: http://meltware.com/2019/01/16/gccgo-benchmarks-2019.html
Re: Building a Better Go Linker
#49Earlier quoted context omitted.
Let me refine my point a bit. Apologies; my original post had gotten a bit long. I agree that enthusiasm is important! And indeed, for the Go creators, their particular leanings might have been such that they couldn't get excited about building an LLVM/GCC frontend, and adapting the Plan 9 toolchain is literally the only way those three could have Go gotten off the ground. As a member of the Go team, you'd certainly…
You agree and then again go completely sideways by unreasonably expecting that Go team to contribute to stack they neither claim to be expert on, nor particularly like it. > Improvements to lld or gold can benefit practically everyone using a compiled language. I do not see how a generic framework could be made as fast as custom build stack.
Sorry, but I think you’re still missing my point. It’s well documented that the original creators of Go did not want to build their language on top of LLVM, and did not like C++. That’s totally fine! Of course you have to work on things you like.
But Go is so stable, and has been for more than half a decade. That means Google could spin up a brand new team, entirely separate if they must, of folks who like both LLVM and Go, to build out an LLVM frontend for Go. It’s not like Go is a fast-moving target. There is a very stable spec.
Gccgo is proof that it is possible to have a GCC frontend for Go without too much work. Ian Taylor has been maintaining Gccgo for about ten years, working what seems to be half time. Imagine what could be done if there were a few more folks actively working on improving gccgo, rather than just keeping parity with gc.
> I do not see how a generic framework could be made as fast as custom build stack.
I don’t see why not. It’s mostly a matter of introducing ways of disabling the expensive bits when compiling/linking a simpler language like Go. Yes, it’s probably somewhat slower to develop, but everyone benefits from the work, and you also offload much of the burden of maintaining a compiler/linker for a half dozen different platforms to the LLVM team collectively.
Re: Building a Better Go Linker
#50What I would give for the developers of the Go toolchain to have spent the last decade improving GCC or LLVM instead of their own bespoke toolchain. In many ways Go seems like an excuse for Google to fund the continued development of Plan 9. Three of the five most influential people on the Go team (Ken Thompson, Rob Pike, and Russ Cox) were heavily involved in Plan 9. And it shows. Go's toolchain is a direct descenda…
Given that Apple are the major sponsor of LLVM and they're not exactly cash poor, I think it's reasonable to conclude that being able to throw money and people at LLVM development doesn't explain why it's still much slower than the Go compiler toolchain. In hindsight the Go team made the right call to use their own toolchain. As enneff alludes, Ken Thompson's antipathy towards C++ is well documented: https://bryanpen…
The kinds of optimizations LLVM does is way beyond anything golang does. Golang doesn't even optimize passing function parameters in registers, let alone the advanced optimization techniques LLVM and GCC do.