Show HN: My C compiler compiled itself • Hacker News Board

Re: Show HN: My C compiler compiled itself

#41

Post by [deleted] » Wed, Dec 04, 2024, 6:59 AM UTC

3836293648 wrote: post #38

aleph_minus_one wrote: post #19

Earlier quoted context omitted.

> The thing about Makefiles is that simples ones at least are really easy to write, and read. Much simpler and quicker than a cumbersome python script, that will most likely do less with much more boilerplate code (e.g. dependencies), and be much harder to read. Whether this is true or not depends a lot on from which programming culture/background you come.

It's is absolutely objectively true. Whether you already know it or not depends on your background

[deleted]

Re: Show HN: My C compiler compiled itself

#42

Post by saghm » Wed, Dec 04, 2024, 7:21 AM UTC

anyfoo wrote: post #26

unclad5968 wrote: post #23

Earlier quoted context omitted.

As someone that learned C# and python before C and C++, to this day I couldn't explain to you how make rules work. Make is so unlike build tools from other languages that it doesn't surprise me someone would rather use python to bootstrap their compiler.

It is so trivial, it takes you 10 minutes to learn. Less even, here is an attempt: foo.o: foo.c clang -c -o foo.o foo.c This builds a file called foo.o, if foo.c is newer than foo.o. Imagine that there is an identical bar.o as well, that builds it from bar.c. fooexec: foo.o bar.o clang -o fooexec foo.o bar.o This links together foo.o and bar.o into a file called fooexec, if at least one of foo.o or bar.o is newer (wh…

> Do $@ and $And yet, I still don't actually know what `$@` and `$The problem is that "simple" is not the same as "intuitive" or "discoverable". Yes, I could probably put in the effort to go and find out these things, but why would you expect someone to be motivated to do that by someone telling you that it's "trivial" and it "pains" them as they explain things in a way that doesn't actually clarify things in a way that helps me? If you actually want to try to make a strong case that this is something worth learning for people, repeatedly referring to it in ways like "very simple" and "so trivial" and "immediately obvious" is counterproductive.

As an aside, I think it's also a bit of leap to assume that they're "rebuilding everything all the time". It looks to me like the build.py script is only needed a single time to bootstrap from the original compiler to this one, and rebuilding every single C file in the new compiler rather than using the artifacts built with the original one is kind of the whole point. After that's done once, I don't see why the new compiler couldn't be used via the Makefile. If you're complaining about Python build scripts in general rather than this specific one, I don't know why you assume they can't be written to check the timestamps on the files to see if a rebuild is necessary before doing so. That's exactly what `make` does, and hey, it's a very simple concept!

Re: Show HN: My C compiler compiled itself

#43

Post by teo_zero » Wed, Dec 04, 2024, 7:37 AM UTC

Nice job!

But why 3 steps of compilation?

The first step merely shows that you wrote valid C code that gcc can compile. It doesn't prove that the program actually does what promised. For example, if you missed to implement 'for' loops, this step would still produce a compiler, which could still work for a subset of C.

Here comes the second step: it proves that it implements enough features to at least recompile itself. Now, this is still not a guarantee that all C constructs and features work, but we can reason that even if it only implements a subset of C, it's a large enough subset to build such a substantial app as a compiler. How likely is it that a compiler doesn't use a 'for' loop, not even once?

But what is the reason for the third step? It doesn't add anything, it doesn't trigger any code path that was excluded in the first one. After all, if you did miss 'for' loops, and the 2nd step hasn't detected it, it must be because there are no 'for' loops in the source, so you will never detect it this way, no matter how many steps of recompilation you run.

Re: Show HN: My C compiler compiled itself

#44

Post by keyvank » Wed, Dec 04, 2024, 7:44 AM UTC

teo_zero wrote: post #43

Nice job! But why 3 steps of compilation? The first step merely shows that you wrote valid C code that gcc can compile. It doesn't prove that the program actually does what promised. For example, if you missed to implement 'for' loops, this step would still produce a compiler, which could still work for a subset of C. Here comes the second step: it proves that it implements enough features to at least recompile itsel…

Thanks! The third step proves that the assembly code generated by the second-step compiler is valid even when compiling a big C project.

Re: Show HN: My C compiler compiled itself

#45

Post by Tor3 » Wed, Dec 04, 2024, 8:34 AM UTC

aleph_minus_one wrote: post #19

anyfoo wrote: post #16

Earlier quoted context omitted.

Or just take a few moments to learn the basics about Makefiles. The thing about Makefiles is that simples ones at least are really easy to write, and read. Much simpler and quicker than a cumbersome python script, that will most likely do less with much more boilerplate code (e.g. dependencies), and be much harder to read. Of course, you may hit a point where you stretch your Makefile so much beyond its common capabi…

> The thing about Makefiles is that simples ones at least are really easy to write, and read. Much simpler and quicker than a cumbersome python script, that will most likely do less with much more boilerplate code (e.g. dependencies), and be much harder to read. Whether this is true or not depends a lot on from which programming culture/background you come.

A Makefile dependency:

  Left : right

Left depends on the right. I don't think the ability to learn that concept is related to programming culture/background. What follows below that is what happens.

  Left : right
         cp right Left
         echo and so on. Left is now updated.

The only tricky part of the above, and something I guess nobody has found any good reason for, is that the whitespace in front of the statements ('cp' in this case) has to be an actual tab, just spaces won't do.

When it comes to the more "advanced" concepts (wildcards etc) there are slight differences between Make versions. And to bother with that is to get into the mindset which created the (argh) Automake and Autoconf systems (to begin with), so back in the day our company simply decided that we'll use GNU Make on every single system (we supported lots of various UNIX systems) and not bother with any of that (no SYSV Make, no BSD Make or anything), because GNU Make was and is available on anything and everything. Made life very simple back then.

Re: Show HN: My C compiler compiled itself

#46

Post by mturmon » Wed, Dec 04, 2024, 8:35 AM UTC

saghm wrote: post #42

anyfoo wrote: post #26

Earlier quoted context omitted.

It is so trivial, it takes you 10 minutes to learn. Less even, here is an attempt: foo.o: foo.c clang -c -o foo.o foo.c This builds a file called foo.o, if foo.c is newer than foo.o. Imagine that there is an identical bar.o as well, that builds it from bar.c. fooexec: foo.o bar.o clang -o fooexec foo.o bar.o This links together foo.o and bar.o into a file called fooexec, if at least one of foo.o or bar.o is newer (wh…

> Do $@ and $ And yet, I still don't actually know what `$@` and `$ The problem is that "simple" is not the same as "intuitive" or "discoverable". Yes, I could probably put in the effort to go and find out these things, but why would you expect someone to be motivated to do that by someone telling you that it's "trivial" and it "pains" them as they explain things in a way that doesn't actually clarify things in a way…

Since you’re asking, $@ is the “target” of the build, that is, what you are aiming to build (left of the colon in the rule). This is why it looks like a bullseye.

And $Hopefully this helps?

Re: Show HN: My C compiler compiled itself

#47

Post by CJefferson » Wed, Dec 04, 2024, 8:50 AM UTC

anyfoo wrote: post #33

CJefferson wrote: post #29

Earlier quoted context omitted.

Saying this is an 'incremental makefile' isn't really correct, as changes to header files aren't going to lead to rebuilding. So, either you need to manually keep references to which .h files you include in your Makefiles up to date, or start worrying about M / MM / MG / MP, and of course you'd like those to be re-run when you change your files, and suddenly your Makefile is an awful lot less simple. This is the main…

It is correct to call it an incremental Makefile, it merely doesn’t name the dependencies for you. A Makefile has exactly the dependencies you specify. It’s that simple. That is not necessarily the most useful . Especially as your project gets larger, you probably will not want to specify every header file as a manual dependency, or alternatively rebuild everything every time you change a header file. (Though for me…

Having 'incrementality', when it isn't actually going to incrementally rebuild things when files change, is (in my experience), worse that having no incrementality at all. Having to remember when I have to manually disable to incrementality is an annoying overhead, and easy to forget.

If you can remember exactly when you need to manually skip the incremental build, that's great for you, but I find Make has enough of these kinds of footguns I don't recommend it to people any more.

Re: Show HN: My C compiler compiled itself

#48

Post by anyfoo » Wed, Dec 04, 2024, 9:16 AM UTC

CJefferson wrote: post #47

anyfoo wrote: post #33

Earlier quoted context omitted.

It is correct to call it an incremental Makefile, it merely doesn’t name the dependencies for you. A Makefile has exactly the dependencies you specify. It’s that simple. That is not necessarily the most useful . Especially as your project gets larger, you probably will not want to specify every header file as a manual dependency, or alternatively rebuild everything every time you change a header file. (Though for me…

Having 'incrementality', when it isn't actually going to incrementally rebuild things when files change, is (in my experience), worse that having no incrementality at all. Having to remember when I have to manually disable to incrementality is an annoying overhead, and easy to forget. If you can remember exactly when you need to manually skip the incremental build, that's great for you, but I find Make has enough of…

Meh, it can practically never be perfect, without diminishing returns that at some point invalidate the advantage of any incremental builds anyway.

As I said, you can let the C compiler generate your header dependencies for you. But that's not enough, because what's with header and libraries that are external for your project? Keep in mind that they will have dependencies, too, and at some point your transitive closure will be really big.

To a lesser degree, what's with a compiler, linker, or any random other part of your toolchain that changes (at least those are supposed to mostly keep a stable interface, which isn't the case at all for external libraries, but in some cases the toolchain can cause the footguns you mentioned).

A lot of the times, you don't need to rebuild everything just because a system header or the toolchain changed, but your build system would have a hard time knowing that. Because at some point, the halting problem even gets in your way (i.e. you can't really reliably detect if a given change in a header file would need rebuilding of a source file, which you would need for "fully incremental" builds). So it's always a trade-off.

Personally, I've fared very well with generated header dependencies (sometimes manually declared ones or none at all for small projects), and many other projects have, too.

YMMV of course, but I don't observe this to be bad. Most people who program C and use make are, I think, aware of what header mismatches can cause, and how to avoid that.

Re: Show HN: My C compiler compiled itself

#49

Post by anyfoo » Wed, Dec 04, 2024, 9:19 AM UTC

CJefferson wrote: post #39

fuhsnn wrote: post #32

Earlier quoted context omitted.

Yes, this is quite important for C projects as it leads to subtle bugs. I also long for a way to let make detect a change of $CC (new build of the compiler itself) in the context of debugging a compiler against third party makefile projects.

I often use a Python program called 'fabricate', which basically checks every file a command opens, and re-runs the command if any touched file changes. I love it, and I hoped this would become the future of how build-systems are built -- you could automatically parallelise, rebuild, make clean, everything, just by tracking which files are read, and written. It does add a bit of overhead, but I'd be willing to pay th…

That is neat, but also super brittle, and full of abstraction leakiness, potentially leading to both false positives and negatives. Sounds like it would break down with ccache or distcc immediately, for example.

Re: Show HN: My C compiler compiled itself

#50

Post by anyfoo » Wed, Dec 04, 2024, 9:30 AM UTC

saghm wrote: post #42

anyfoo wrote: post #26

Earlier quoted context omitted.

It is so trivial, it takes you 10 minutes to learn. Less even, here is an attempt: foo.o: foo.c clang -c -o foo.o foo.c This builds a file called foo.o, if foo.c is newer than foo.o. Imagine that there is an identical bar.o as well, that builds it from bar.c. fooexec: foo.o bar.o clang -o fooexec foo.o bar.o This links together foo.o and bar.o into a file called fooexec, if at least one of foo.o or bar.o is newer (wh…

> Do $@ and $ And yet, I still don't actually know what `$@` and `$ The problem is that "simple" is not the same as "intuitive" or "discoverable". Yes, I could probably put in the effort to go and find out these things, but why would you expect someone to be motivated to do that by someone telling you that it's "trivial" and it "pains" them as they explain things in a way that doesn't actually clarify things in a way…

One might ask how you learned C (or python, or anything else) in the first place, if you can't be bothered to learn what the very simple $The rest is just... of course you can reimplement make, or a subset of it, in python, python is turing complete after all. But why? make isn't that arcane. I'm sorry it uses the weirdly looking $@ and $This is trivial in the sense that it's a part of any build system worth its salt. And I can almost guarantee you that OP for example, who wrote their own C compiler, understood it immediately.