Live data from Hacker News

Please – A cross-language build system

please.build

241–250 of 257 posts

Re: Please – A cross-language build system

#241

Earlier quoted context omitted.

The problem with Make is that it doesn't know what it's building, so it can't do anything smart. For incremental builds to work at all, you have to supply the smarts. Having worked on a number of make-based projects (I am most scarred by buildroot), I can tell you that people make mistakes with build rules. The project then devolves to doing a clean build for every change, turning what could be a few milliseconds of…

I am personally not convinced that any build system can be correct and general, but perhaps that’s my lack of experience speaking. On that 30 minute note though: so, how big does the project need to be in order for Make not to be enough? And at that size, why wouldn’t the project invest the extra week it takes to get the Makefile correct?

I completely agree with you. To give the build system the understanding it needs, you have to give up the generality.

For me, I've found that the results are best when you use a build system and you use it exactly the way the author intends. For example, Go's built-in build system is so good that you don't even notice it's there. It automatically updates its configuration as you write code. It does a perfect incremental build every time, including tests. The developer experience is basically perfect, because it has a complete understanding of the mapping between source files, dependencies, and outputs. But, it is not extendable, so you're screwed when your Go project also needs to webpack some frontend code, or you need to generate protocol buffers (which involves a C++ binary that runs a Go binary), etc. So, people bolt those features on, and the build system becomes more general, but not quite as good. (Then there's make, which is as good or as bad as you want it to be.)

I think super small projects often do get their Makefiles right. But you can manually build small projects with something like "gcc foo.c bar.c -o foobar -lbaz", and so you don't really benefit from any improvement over the bare essentials. (Nothing wrong with keeping your projects tiny in scope, of course!)

But, sometimes you don't have the luxury of a super small project, and the Makefiles become quickly unfixable. Like I said, I am most scarred by a buildroot project I worked on (that's embedded Linux, basically). It never built what I expected it to build, and to test anything reliably I either had to yolo my own incremental build or wait a while for a full build. My productivity was minimal. I could switch between client-side and server-side tasks on that project, and so I really only touched the client if it was absolutely necessary. I would never be productive enough to undertake a major project that truly added value with that kind of build system, so I let others that didn't have the server-side experience write the client-side stuff. In that case, the poor build system silently cost our team productivity in terms of artificially splitting the team between people who could tolerate a shitty developer experience and those who couldn't.

I don't think anyone has fixed the buildroot problem, either. If you want to build a Linux image today, you are stuck with these problems. Nothing else is general enough to build Linux and the associated random C binaries that you're going to want to run.

Re: Please – A cross-language build system

#242
post #167

Earlier quoted context omitted.

Make is only good at one thing: calculating how to execute a dependency graph to produce a specific target. Everything else that is needed for a modern build system is lacking: - expressive scripting language - platform-related configuration - feature-related configuration - handling hierarchical projects - properly handling rebuilds when the Makefile itself has been changed - dealing with dynamically generated parts…

Make is a programming language: it is only good at executing that language, and that language kind of sucks, but saying Make is somehow only good at executing static dependency graphs is a fundamental misunderstanding of Make (but AFAIK might be a good description of say, ninja). All of the Make scripts I have written for the past 15 years have dynamically generated configuration and autodetected files and calculated…

Anything that is (like make) turing complete can be made to do all the things you list.

At a very minor cost after all: your sanity.

Re: Please – A cross-language build system

#243

Earlier quoted context omitted.

What was/is the motivation for this build system that an existing one doesn't satisfy?

We wanted something like Blaze (Google's system); at the time we were using Buck which didn't satisfy us (e.g. only one output per genrule, no directories; and it was more or less impossible to write first-class support for a new language without modifying the core system). Subsequently Bazel got released but we still find that a bit lacking in some areas; e.g. the CLI, the JVM reliance and some of Starlark (e.g. it…

At first this wasn't a singular clear answer I was hoping for, but the many points being made here make it clear that it's a complex area and there's room for another advanced/high-perf build system to cover a broader range of usage.

Re: Please – A cross-language build system

#244
post #220
post #89

Earlier quoted context omitted.

I have use all of these tools and your take isn't accurate at all. Make doesn't obfuscate anything more than Plz or CMake or whatever. Here are some real reasons why Make isn't the end-all: - Make doesn't allow platform selection in a nice way. - Make doesn't work on Windows natively (no, NMake doesn't count). - Recursive make doesn't work well at all. - Make doesn't track byproducts or deleted artifacts. - Make does…

What tool would you pick over plz?

CMake still. I know there is something that will come out some day that is better, but until that day, CMake is still the best.

Re: Please – A cross-language build system

#245
post #77

Earlier quoted context omitted.

I almost always inspect these kinds of scripts before running, more out of curiosity than anything, but also so that I know its not going to do that so stupid that even I can see it's stupid. Usually you can just pipe to `cat`, which is super low effort to do. I've occasionally seen scripts that install some other application where it was not clear that it was a dependency and there was no heads up to the user that t…

cat to terminal is kinda not enough. I can own you very easily if you do just that and think you've seen all the code that will be executed. Save to file, and read the file in the editor.

It’s not really intended as a defense against being owned per se, it’s more about knowing what’s going on and getting an additional signal about the risk profile (not just from maliciousness) of the thing I’m about to run.

That said, I generally pipe to file and cat the file, yes, if only because it somehow feels wrong to download it twice.

Re: Please – A cross-language build system

#247
post #23

Earlier quoted context omitted.

Plz is great for c++, and has a better workflow (IMO) vs (e.g) Cmake. The skylark-like syntax is easier to reason through than cmake generative expressions. Also the use of “plz run” vs “cmake -g; make; run”

cmake generator only needs to be run once per project. After that you use make or ninja directly, and it regenerates the build files if necessary. I can't say it's a strong argument against cmake.

Sadly this doesn’t always work. I have lost loads of time trying to figure this out then doing a clean rebuild.

Re: Please – A cross-language build system

#248
post #160

Earlier quoted context omitted.

Incremental builds with the said tools (Please, Bazel) should be a non-issue due to emphasis on hermetic build from these tools.

Unfortunately, reality is messier than this. For example, Please currently has escape hatches that can be used that nullify these guarantees. What I have found is that if such a mechanism exists it will eventually be abused. My perspective here is specifically from that of a maintainer of large builds where the code base is always under active development. Murphy's law becomes your enemy at scale. This is not a criti…

I haven't had many issues with non-determinism when using Bazel/Blaze? Especially when you use remote execution or sandboxing it's pretty hard to avoid being hermetic.

Re: Please – A cross-language build system

#249
post #171

I personally despise people combining terms like "fast" or "lightweight" with languages like Go and Python. We are looking at a build system that, to my knowledge, doesn't take any advantage from being written in Go, other than "being easier to write and understand by a human". When I see someone giving such reasons for not using a more performant language, I immediately associate such project with janky, fast writte…

Are build systems typically CPU bound, even during heavy operations such as calculating dependency closure (or other things I'm unaware of)? If not, the I don't see a need to use a language that doesn't provide high level, somewhat costly, but easy to use and understand abstractions. And if it's faster than its competitors and it weighs less, then there's also nothing wrong with calling it "fast" and "lightweight" (t…

My full time job was to make Bazel more efficient for about 1.5 years. The answer is: it really depends.

If you are executing 100,000 novel parallel C++ actions, then compilation for that might be I/O bound? If these actions are all cached though you would likely be CPU bound instead, as the build system works hard to discover that all the work is already done.

Re: Please – A cross-language build system

#250
post #77

Earlier quoted context omitted.

cat to terminal is kinda not enough. I can own you very easily if you do just that and think you've seen all the code that will be executed. Save to file, and read the file in the editor.

It’s not really intended as a defense against being owned per se, it’s more about knowing what’s going on and getting an additional signal about the risk profile (not just from maliciousness) of the thing I’m about to run. That said, I generally pipe to file and cat the file, yes, if only because it somehow feels wrong to download it twice.

I understand, but cating the saved file or printing from pipe to the terminal directly has identical issues. Terminal ANSI escape sequences are interpreted either way.
Post reply on HN