Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redo: A recursive, general-purpose build system

redo.readthedocs.io

Post reply on HN
Print view View original

21–30 of 95 posts

Re: Redo: A recursive, general-purpose build system

#21

Post by Quekid5 »

lolski wrote: post #15
AceJohnny2 wrote: post #4

As an intensive (and reluctant) user of GNU Make, I keep looking for a more modern replacement. Redo is not it. Make is really a complicated combination of these components: - a dependency definition language with in-place evaluation and half-assed wildcards - A functional language with extremely limited datatypes (effectively just space-separated strings) - a clunky macro system - A "distributed" topological executi…

I second Bazel. People keep on mentioning how steep the learning curve is, but the conceptual model is really simple, elegant, and intuitive. What is steep is the technical know-hows: 1. When things don't work as expected. For example, while it worked flawlessly with languages that it natively support such as Java, that wasn't the case for other languages such as Javascript or Python. 2. When you have to do something…

The problem with Bazel has nothing to do with learning curve. It's a pretty simple model as far as build systems go.

It's that it requires a lot of boilerplate and has a very rigid nested structure... which compounds boilerplate, unless you venture into custom plugins/build rules.

Some of the basic ideas are absolutely right, e.g. separating the resolution of dependencies from the build, and purity, but it's just soooo much boilerplate.

Re: Redo: A recursive, general-purpose build system

#22

Post by seanstrom »

Please someone mention Tup and how it can be a reasonable build system. I’ve heard the Fuse dependency is not ideal, though I felt it had a nice UX experience with the Lua config.

Plus it’s worth considering that you can potentially use Fennel to configure the builds (since Fennel compiles to Lua).

Tup: https://github.com/gittup/tup

Fennel: https://fennel-lang.org/

Re: Redo: A recursive, general-purpose build system

#23

Post by throwaway894345 »

packetlost wrote: post #19
lolski wrote: post #15

Earlier quoted context omitted.

I second Bazel. People keep on mentioning how steep the learning curve is, but the conceptual model is really simple, elegant, and intuitive. What is steep is the technical know-hows: 1. When things don't work as expected. For example, while it worked flawlessly with languages that it natively support such as Java, that wasn't the case for other languages such as Javascript or Python. 2. When you have to do something…

Bazel is awful unless you're Google and have a team of devs supporting Bazel.

I suspect the experience varies pretty widely based on what languages you use and what you’re trying to do with it. If you’re just building Go binaries then I’m sure Bazel is great, but if you’re doing anything with Python 3 or if you’re doing something a little off the beaten path like code generation, it’s probably a frustrating experience.

Re: Redo: A recursive, general-purpose build system

#24

Post by fragmede »

codedokode wrote: post #20
AceJohnny2 wrote: post #4

As an intensive (and reluctant) user of GNU Make, I keep looking for a more modern replacement. Redo is not it. Make is really a complicated combination of these components: - a dependency definition language with in-place evaluation and half-assed wildcards - A functional language with extremely limited datatypes (effectively just space-separated strings) - a clunky macro system - A "distributed" topological executi…

I don't like using tabs in Makefile syntax. Tabs are in many cases (especially on print) indistinguishable from spaces. Also, any "make" program for C should automatically detect dependencies between files because this information is present in the code and it doesn't make sense to duplicate it.

GNUMake allows for custom, non-tab leader characters.

https://stackoverflow.com/questions/2131213/can-you-make-val...

Re: Redo: A recursive, general-purpose build system

#26

Post by thechao »

AceJohnny2 wrote: post #4

As an intensive (and reluctant) user of GNU Make, I keep looking for a more modern replacement. Redo is not it. Make is really a complicated combination of these components: - a dependency definition language with in-place evaluation and half-assed wildcards - A functional language with extremely limited datatypes (effectively just space-separated strings) - a clunky macro system - A "distributed" topological executi…

I skimmed the paper. I use Make, extensively, but in a way I generally don’t see in the wild (for C/++):

1. I convert all paths to abspaths;

2. All intermediate products go into a temp dir;

3. All recipes are functions;

4. The set of objects are defined by a “find . -name ‘…’” for C sources;

5. Every C file depends on every header file in the repo;

6. Use ccache; and,

7. Deterministic build.

My big work project is ~20mm loc, including ~65 FW images (embedded OSes). Incremental compilation is ~300ms. Clean rebuilds are 30-40s; scratch builds are 8m. (All of this on ~4 year old MBP.)

Obviously, there are some design tricks to get raw compilation speed up — naive C++ only compiles at ~3kloc/s; it’s possible to get that up to 100kloc/s.

Re: Redo: A recursive, general-purpose build system

#27

Post by Arch-TK »

AceJohnny2 wrote: post #4

As an intensive (and reluctant) user of GNU Make, I keep looking for a more modern replacement. Redo is not it. Make is really a complicated combination of these components: - a dependency definition language with in-place evaluation and half-assed wildcards - A functional language with extremely limited datatypes (effectively just space-separated strings) - a clunky macro system - A "distributed" topological executi…

I think not adding an arbitrary new declarative language is good design feature of redo. I think redo has other issues but personally redo with bash as the underlying glue works a lot better than make (in terms of maintaining more complete DAGs). Now I'm not saying it wouldn't be nice to get a declarative language which works WITH redo, but it would be important to keep it separate.

One thing redo is missing for me is support for building things in a different directory. It's a nice-to-have feature but it's difficult to envision how such a redo would work. Really overlays actually solve this problem, although a fuse based overlayfs would also be nice (so you can use it without needing to escalate privileges).

Re: Redo: A recursive, general-purpose build system

#28

Post by Arch-TK »

einpoklum wrote: post #17

Some obvious questions: 1. How does redo compare with lower-level build tools such as ninja? 2. Why is it important/worthwhile to create a simplified/elegant version of Make, when for a few decades already, it is customary to write something higher-level that generates Makefiles (or builds in ways other than through Make, like scons or again via ninja)? 3. Is there a redo generator for CMake, and does it have advanta…

0. I would recommend you actually read about redo yourself. Redo is not just the implementation posted here (which personally I don't like for multiple reasons). Redo is a very simple idea envisioned by DJB a while ago which was never implemented by him properly. You can read more about it here: http://cr.yp.to/redo.html it won't take long.

1. Ninja is not meant to be hand written, redo can be hand written. Ninja verbosely describes your build as flat file containing the description of a DAG and the rules required to traverse it. Redo is a recursive definition, intended to be mostly hand written, which describes your build process. Redo can and should be split up across multiple files.

2. It may be customary to you, but not everyone agrees that the correct solution to building a project is to have a gigantic software suite consume a confusing and abstract configuration file and spit out a bunch of buggy makefiles. There are in fact plenty of people who hand write makefiles, redo fits in that same space. It's easier to hand write, it's easier to fully describe the DAG of your project, it's easier to reason about the build process and it's easier to make sure it is correct.

3. No. It would make no sense to have a redo generator for CMake as it would be severely missing the point of redo. The only reason CMake has generators for anything but ninja is because some people worry about portability to other systems and as such having make is better than nothing.

Re: Redo: A recursive, general-purpose build system

#29

Post by w4rh4wk5 »

seanstrom wrote: post #22

Please someone mention Tup and how it can be a reasonable build system. I’ve heard the Fuse dependency is not ideal, though I felt it had a nice UX experience with the Lua config. Plus it’s worth considering that you can potentially use Fennel to configure the builds (since Fennel compiles to Lua). Tup: https://github.com/gittup/tup Fennel: https://fennel-lang.org/

I have encountered Tup in the past and could not figure out how to define a "generator". As in: a function that defines how some output can be generated from a certain input running multiple commands. I don't want to copy those commands for every input I need to process.

Edit: Generator is a term typically used in CMake and Meson for this, in Make I'd use a pattern rule mostly.

Re: Redo: A recursive, general-purpose build system

#30

Post by xyzzy_plugh »

throwaway894345 wrote: post #23
packetlost wrote: post #19

Earlier quoted context omitted.

Bazel is awful unless you're Google and have a team of devs supporting Bazel.

I suspect the experience varies pretty widely based on what languages you use and what you’re trying to do with it. If you’re just building Go binaries then I’m sure Bazel is great, but if you’re doing anything with Python 3 or if you’re doing something a little off the beaten path like code generation, it’s probably a frustrating experience.

> If you’re just building Go binaries then I’m sure Bazel is great

Compared to other things in Bazel maybe, sure. But if you're just building Go binaries you don't need Bazel. The standard Go tooling is beyond sufficient: it's hard to beat.

Post reply on HN