Rustc_codegen_GCC can now bootstrap rustc • Hacker News Board

Re: Rustc_codegen_GCC can now bootstrap rustc

#11

Post by da-x » Fri, Apr 01, 2022, 2:29 PM UTC

hardware2win wrote: post #6

goombacloud wrote: post #3

It's really great to be able to skip compiling LLVM when bootstrapping a whole OS/toolchain from source and only Rust happens to pull LLVM in. Besides the gcc work I'm actually also hoping for the rustc_codegen_cranelift work to land one day!

Compiling llvm is a nightmare

Compiling LLVM for what purpose? Hacking on it? Producing an optimized build? Installing a build on a user's machine? You should be more specific.

The default clean build works just fine. On my machine I can clone the Git repo, do `mkdir build && cmake ../llvm && make -j64` and end up with working binaries after 5 minutes.

The build take 37GB by default here, and the binaries under bin/ are immediately usable work. If you want a smaller build with shared library just add a parameter. Want to throw in sub-projects such as LLD or Clang? Just another parameter. It's all very well-documented.

Re: Rustc_codegen_GCC can now bootstrap rustc

#12

Post by FooBarWidget » Fri, Apr 01, 2022, 2:33 PM UTC

da-x wrote: post #11

hardware2win wrote: post #6

Earlier quoted context omitted.

Compiling llvm is a nightmare

Compiling LLVM for what purpose? Hacking on it? Producing an optimized build? Installing a build on a user's machine? You should be more specific. The default clean build works just fine. On my machine I can clone the Git repo, do `mkdir build && cmake ../llvm && make -j64` and end up with working binaries after 5 minutes. The build take 37GB by default here, and the binaries under bin/ are immediately usable work. I…

> `make -j64` and end up with working binaries after 5 minutes.

I think the problematic part for most people lies in the first two words in that quoted sentence.

Re: Rustc_codegen_GCC can now bootstrap rustc

#13

Post by meinersbur » Fri, Apr 01, 2022, 2:45 PM UTC

FooBarWidget wrote: post #12

da-x wrote: post #11

Earlier quoted context omitted.

Compiling LLVM for what purpose? Hacking on it? Producing an optimized build? Installing a build on a user's machine? You should be more specific. The default clean build works just fine. On my machine I can clone the Git repo, do `mkdir build && cmake ../llvm && make -j64` and end up with working binaries after 5 minutes. The build take 37GB by default here, and the binaries under bin/ are immediately usable work. I…

> `make -j64` and end up with working binaries after 5 minutes. I think the problematic part for most people lies in the first two words in that quoted sentence.

`mkdir build && cmake ../llvm -GNinja && ninja`

better?

Re: Rustc_codegen_GCC can now bootstrap rustc

#14

Post by OskarS » Fri, Apr 01, 2022, 2:54 PM UTC

da-x wrote: post #11

hardware2win wrote: post #6

Earlier quoted context omitted.

Compiling llvm is a nightmare

Compiling LLVM for what purpose? Hacking on it? Producing an optimized build? Installing a build on a user's machine? You should be more specific. The default clean build works just fine. On my machine I can clone the Git repo, do `mkdir build && cmake ../llvm && make -j64` and end up with working binaries after 5 minutes. The build take 37GB by default here, and the binaries under bin/ are immediately usable work. I…

64 cores, and it takes 5 minutes? That’s… not awesome. Most people do not have 64 (presumably high performance) cores lying around for the purpose of building dependencies.

Re: Rustc_codegen_GCC can now bootstrap rustc

#15

Post by lazypenguin » Fri, Apr 01, 2022, 2:55 PM UTC

meinersbur wrote: post #13

FooBarWidget wrote: post #12

Earlier quoted context omitted.

> `make -j64` and end up with working binaries after 5 minutes. I think the problematic part for most people lies in the first two words in that quoted sentence.

`mkdir build && cmake ../llvm -GNinja && ninja` better?

Most people don’t have 64 cores available to them to compile LLVM in 5 minutes. Switching to ninja doesn’t really solve that problem.

Re: Rustc_codegen_GCC can now bootstrap rustc

#16

Post by bregma » Fri, Apr 01, 2022, 2:56 PM UTC

meinersbur wrote: post #13

FooBarWidget wrote: post #12

Earlier quoted context omitted.

> `make -j64` and end up with working binaries after 5 minutes. I think the problematic part for most people lies in the first two words in that quoted sentence.

`mkdir build && cmake ../llvm -GNinja && ninja` better?

You missed the part where you upgrade your RAM to at least 128 GB and up the swap to 512 GB and use gold as your linker because the OOM killer will still raise it's head for ld under those extreme restrictions.

Re: Rustc_codegen_GCC can now bootstrap rustc

#17

Post by mhh__ » Fri, Apr 01, 2022, 3:05 PM UTC

roblabla wrote: post #4

VWWHFSfQ wrote: post #2

This is all very exciting. I'm guessing we'll see some pretty major performance improvements for rust code in addition to all the new supported architectures

I don't see why the GCC codegen would end up with major performance improvements - my understanding is that GCC and LLVM are basically on par with performance, with some workloads being a tiny bit better in one or the other. Is that wrong?

GCC is still basically better overall but it's basically chaotic.

LLVM seems to have better defaults for branch-y compiler code, but GCC has a lot of tricks up its sleeve that LLVM hasn't got.

FWIW though I think GCC is really playing with fire when it comes to it's future - the mailing list approach to development deters new contributors, the testing infrastructure is quite arcane, etc.

GCC should be an aspirational tool that people want to contribute to but aside from a few core contributors it feels like its basically on life support in some regards. And I say that as someone who actively tries to use it.

Its core competencies are pretty good but it's (mostly stallman, to be blunt) legacy has made it very brittle at its extremities e.g. LTO explodes on me way too often.

Re: Rustc_codegen_GCC can now bootstrap rustc

#18

Post by ______-_-______ » Fri, Apr 01, 2022, 3:06 PM UTC

roblabla wrote: post #4

VWWHFSfQ wrote: post #2

This is all very exciting. I'm guessing we'll see some pretty major performance improvements for rust code in addition to all the new supported architectures

I don't see why the GCC codegen would end up with major performance improvements - my understanding is that GCC and LLVM are basically on par with performance, with some workloads being a tiny bit better in one or the other. Is that wrong?

Up until now the rust compiler has been finely tuned for LLVM. I would expect the GCC backend to perform worse initially, then improve over time depending on how much use it gets.

Re: Rustc_codegen_GCC can now bootstrap rustc

#19

Post by shipp02 » Fri, Apr 01, 2022, 3:41 PM UTC

bregma wrote: post #16

meinersbur wrote: post #13

Earlier quoted context omitted.

`mkdir build && cmake ../llvm -GNinja && ninja` better?

You missed the part where you upgrade your RAM to at least 128 GB and up the swap to 512 GB and use gold as your linker because the OOM killer will still raise it's head for ld under those extreme restrictions.

Sorry, what are you talking about. I have built ponylang, build llvm with a couple of patches from source on my 2018 Mac book air. Just don't put -j64. The first build will take a couple of hours but that is par for the course for a project of llvm's size and complexity.

The subsequent buttons take a couple minutes.

Re: Rustc_codegen_GCC can now bootstrap rustc

#20

Post by mahkoh » Fri, Apr 01, 2022, 4:10 PM UTC

mhh__ wrote: post #17

roblabla wrote: post #4

Earlier quoted context omitted.

I don't see why the GCC codegen would end up with major performance improvements - my understanding is that GCC and LLVM are basically on par with performance, with some workloads being a tiny bit better in one or the other. Is that wrong?

GCC is still basically better overall but it's basically chaotic. LLVM seems to have better defaults for branch-y compiler code, but GCC has a lot of tricks up its sleeve that LLVM hasn't got. FWIW though I think GCC is really playing with fire when it comes to it's future - the mailing list approach to development deters new contributors, the testing infrastructure is quite arcane, etc. GCC should be an aspirational…

>FWIW though I think GCC is really playing with fire when it comes to it's future - the mailing list approach to development deters new contributors, the testing infrastructure is quite arcane, etc.

And LLVM is better? At the start of last year I found and fixed a small bug in LLVM[1]. I submitted the fix via their Phabricator instance and it was approved by project members after a few days. As per the contributor guide, I then asked for a project member to commit my patch to the LLVM repository. This is because in Phabricator there does not appear to be an automatic way to commit approved changes. A project member has to manually take the patch and apply it.

A few weeks ago I tried to see what happened to my patch and found that it had never been applied. So basically my work and that of the reviewers was simply lost because of their infrastructure.

[1] https://reviews.llvm.org/D96334