Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“This change deletes the C implementations of the Go compiler and assembler”

github.com

Post reply on HN
Print view View original

111–120 of 137 posts

Re: “This change deletes the C implementations of the Go compiler and assembler”

#111

Post by pjmlp »

arcticbull wrote: post #95
dsymonds wrote: post #94

Earlier quoted context omitted.

What's your counter-proposal? If you're building a new language, you need a new AST. You can't represent Go source code in a C++ AST. There are alternate compilers for Go, in the form of gccgo and llgo. But those are both very slow to build (compared to the Go tree that takes ~30s to build the compiler, linker, assembler and standard library). And the "gc" Go compiler runs a lot faster than gccgo (though it doesn't p…

I would never set out to build a language I wanted people to use and not build it as a front-end for LLVM. I don't want to write an optimizer or assembler. I don't doubt for one second that llgo takes a longer time to compile. And in exchange for slower compile times you benefit from many PHDs worth of optimizations in LLVM. And every single target architecture they support. It's easy to build something faster when i…

Then you will be stuck with the C view of world of what a linker is supposed to do.

Just look at Modula-2 and Object Pascal toolchains as examples of compile speeds and incremental compilation features that could run circles around contemporanean C compilers.

Or the lack of proper module system, which requires linker help.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#112

Post by RayDonnelly »

wbond wrote: post #108
arcticbull wrote: post #91

I still just don't understand why they insist on building their own toolchain. It just doesn't make sense to me. When you set out to build a programming language, what is your objective? To create a sweet new optimizer? To create a sweet new assembler? A sweet new intermediate representation? AST? Of course not. You set out to change the way programmers tell computers what to do. So why do this insist on duplicating:…

I was impressed by the toolchain when I first peaked at Go because it was dead simple to get up and running on any platform, especially Windows. For gcc you have to deal with MinGW. Isn't LLVM just now getting to the point where it can build native Windows applications? This is one area where I hope Rust makes progress. MinGW/Msys2 is just kind of gross stuff to deal with.

Care to explain what's gross about MSYS2/MinGW-w64? I'm genuinely interested in making it less gross.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#113

Post by bsdetector »

mseepgood wrote: post #107
arcticbull wrote: post #91

I still just don't understand why they insist on building their own toolchain. It just doesn't make sense to me. When you set out to build a programming language, what is your objective? To create a sweet new optimizer? To create a sweet new assembler? A sweet new intermediate representation? AST? Of course not. You set out to change the way programmers tell computers what to do. So why do this insist on duplicating:…

> I still just don't understand why they insist on building their own toolchain. It just doesn't make sense to me. To quote rsc from https://news.ycombinator.com/item?id=8817990 : "It's a small toolchain that we can keep in our heads and make arbitrary changes to, quickly and easily. Honestly, if we'd built on GCC or LLVM, we'd be moving so slowly I'd probably have left the project years ago." "For example, no standa…

Which is of course no answer at all.

Their own explanation for wasting hundreds of thousands of man-hours on a "quirky and flawed" separate compiler, linker, assembler, runtime, and tools is because they absolutely needed an implementation detail that is completely invisible to programs and which they are now replacing because it wasn't a good idea in the first place (segmented stacks). And it's worth writing out a 1000 word rationalization that doesn't bother even mention the reason that implementation was necessary in the first place, to better run on 32-bit machines. In 2010.

Or they say that they had to reinvent the entire wheel, axle, cart, and horse so that five years later they could start working on a decent garbage collector. Never mind that five years later other people did the 'too hard and too slow' work on LLVM that a decent garbage collector needs. What foresight, that.

That's not sense, that's people rationalizing away wasting years of their time doing something foolish and unnecessary.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#114

Post by bsdetector »

dsymonds wrote: post #96
arcticbull wrote: post #95

Earlier quoted context omitted.

I would never set out to build a language I wanted people to use and not build it as a front-end for LLVM. I don't want to write an optimizer or assembler. I don't doubt for one second that llgo takes a longer time to compile. And in exchange for slower compile times you benefit from many PHDs worth of optimizations in LLVM. And every single target architecture they support. It's easy to build something faster when i…

Yes, that's why there's both gc and gccgo (llgo came later). Apart from the rigour of having two independent compilers, they are seeking different tradeoffs. gc is very interested in running fast, and gccgo benefits from decades of work that have been put into gcc's various optimisations. Does that answer your original statement that you didn't understand why we build our own toolchain?

Well I still don't understand. Russ says it was for segmented stacks, but doesn't explain why those were necessary. You say it was for compile speed, yet gcc and llvm can crank out millions of lines a code a second at similar optimization levels as the Go compiler. Neither of these are convincing explanations.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#115

Post by crawshaw »

bsdetector wrote: post #113
mseepgood wrote: post #107

Earlier quoted context omitted.

> I still just don't understand why they insist on building their own toolchain. It just doesn't make sense to me. To quote rsc from https://news.ycombinator.com/item?id=8817990 : "It's a small toolchain that we can keep in our heads and make arbitrary changes to, quickly and easily. Honestly, if we'd built on GCC or LLVM, we'd be moving so slowly I'd probably have left the project years ago." "For example, no standa…

Which is of course no answer at all. Their own explanation for wasting hundreds of thousands of man-hours on a "quirky and flawed" separate compiler, linker, assembler, runtime, and tools is because they absolutely needed an implementation detail that is completely invisible to programs and which they are now replacing because it wasn't a good idea in the first place (segmented stacks). And it's worth writing out a 1…

The replacement to segmented stacks is copying stacks, which as far as my knowledge of LLVM takes me, would be very difficult to add. You need a stack map of pointers to successfully move pointed-to objects on the stack from the old region to the new.

There is a great deal of work going on in LLVM on this issue for precise GC of other languages, and (from the outside) it looks like more hours have been spent on it than on the entire Go toolchain. As Go developers don't have the resources or expertise to make such wide-ranging changes to LLVM, it would have blocked Go development.

GCC is similar. Those working on gccgo are trying to work out how to add precise GC and copying stacks. It is much more complex than it was on the gc toolchain.

There is great value in having a simple toolchain that is completely understood by the developers working on it. In fact, that very idea, that code you depend on should be readable and widely understandable, is one of the goals of Go. Applying the goal to the toolchain is a case of eating our own ideological dogfood.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#116

Post by bsdetector »

crawshaw wrote: post #115
bsdetector wrote: post #113

Earlier quoted context omitted.

Which is of course no answer at all. Their own explanation for wasting hundreds of thousands of man-hours on a "quirky and flawed" separate compiler, linker, assembler, runtime, and tools is because they absolutely needed an implementation detail that is completely invisible to programs and which they are now replacing because it wasn't a good idea in the first place (segmented stacks). And it's worth writing out a 1…

The replacement to segmented stacks is copying stacks, which as far as my knowledge of LLVM takes me, would be very difficult to add. You need a stack map of pointers to successfully move pointed-to objects on the stack from the old region to the new. There is a great deal of work going on in LLVM on this issue for precise GC of other languages, and (from the outside) it looks like more hours have been spent on it th…

> The replacement to segmented stacks is copying stacks ...

Which again is not an answer. Why are segmented stacks necessary? Why are copying stacks necessary?

This reasoning, which is their best apparently, amounts to saying that they had to implement their own compiler, linker, assembler, and runtime because they decided they had to implement their own compiler, linker, assembler, and runtime.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#117

Post by mseepgood »

bsdetector wrote: post #116
crawshaw wrote: post #115

Earlier quoted context omitted.

The replacement to segmented stacks is copying stacks, which as far as my knowledge of LLVM takes me, would be very difficult to add. You need a stack map of pointers to successfully move pointed-to objects on the stack from the old region to the new. There is a great deal of work going on in LLVM on this issue for precise GC of other languages, and (from the outside) it looks like more hours have been spent on it th…

> The replacement to segmented stacks is copying stacks ... Which again is not an answer. Why are segmented stacks necessary? Why are copying stacks necessary? This reasoning, which is their best apparently, amounts to saying that they had to implement their own compiler, linker, assembler, and runtime because they decided they had to implement their own compiler, linker, assembler, and runtime.

> Which again is not an answer. Why are segmented stacks necessary? Why are copying stacks necessary?

Because Go wants to provide very lightweight goroutines for highly concurrent and scalable services.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#118

Post by crawshaw »

bsdetector wrote: post #116
crawshaw wrote: post #115

Earlier quoted context omitted.

The replacement to segmented stacks is copying stacks, which as far as my knowledge of LLVM takes me, would be very difficult to add. You need a stack map of pointers to successfully move pointed-to objects on the stack from the old region to the new. There is a great deal of work going on in LLVM on this issue for precise GC of other languages, and (from the outside) it looks like more hours have been spent on it th…

> The replacement to segmented stacks is copying stacks ... Which again is not an answer. Why are segmented stacks necessary? Why are copying stacks necessary? This reasoning, which is their best apparently, amounts to saying that they had to implement their own compiler, linker, assembler, and runtime because they decided they had to implement their own compiler, linker, assembler, and runtime.

Lightweight goroutines depend on small stacks. General API design in Go depends on lightweight goroutines.

In particular, Go style is to never write asynchronous APIs. Always write synchronous blocking code, and when you need to work concurrently, create a goroutine.

You cannot do this in C with pthread, because OS threads are too heavyweight. So you end up in callback-based APIs that are harder to use and harder to debug (no useful stack traces).

This small feature has a surprisingly wide-ranging effect on the use of the language. It is a very big deal.

Go is very much about reinventing these low-level things.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#119

Post by mseepgood »

bsdetector wrote: post #113
mseepgood wrote: post #107

Earlier quoted context omitted.

> I still just don't understand why they insist on building their own toolchain. It just doesn't make sense to me. To quote rsc from https://news.ycombinator.com/item?id=8817990 : "It's a small toolchain that we can keep in our heads and make arbitrary changes to, quickly and easily. Honestly, if we'd built on GCC or LLVM, we'd be moving so slowly I'd probably have left the project years ago." "For example, no standa…

Which is of course no answer at all. Their own explanation for wasting hundreds of thousands of man-hours on a "quirky and flawed" separate compiler, linker, assembler, runtime, and tools is because they absolutely needed an implementation detail that is completely invisible to programs and which they are now replacing because it wasn't a good idea in the first place (segmented stacks). And it's worth writing out a 1…

Most of the toolchain already existed. When Ken Thompson started writing the Go compiler he based it on his Plan 9 C compiler implementation.

Re: “This change deletes the C implementations of the Go compiler and assembler”

#120

Post by bsdetector »

crawshaw wrote: post #118
bsdetector wrote: post #116

Earlier quoted context omitted.

> The replacement to segmented stacks is copying stacks ... Which again is not an answer. Why are segmented stacks necessary? Why are copying stacks necessary? This reasoning, which is their best apparently, amounts to saying that they had to implement their own compiler, linker, assembler, and runtime because they decided they had to implement their own compiler, linker, assembler, and runtime.

Lightweight goroutines depend on small stacks. General API design in Go depends on lightweight goroutines. In particular, Go style is to never write asynchronous APIs. Always write synchronous blocking code, and when you need to work concurrently, create a goroutine. You cannot do this in C with pthread, because OS threads are too heavyweight. So you end up in callback-based APIs that are harder to use and harder to…

Threads and stacks are orthogonal. You can have coroutines with contiguous stacks, and threads with non-contiguous stacks.

Furthermore it's extremely easy to use non-contiguous stacks in C just by knowing the stack amount used by functions, which the compiler already knows.

This is a totally absurd reason to reimplement an entire toolchain.

Post reply on HN