Buck: A high-performance build tool • Hacker News Board

Re: Buck: A high-performance build tool

#31

Post by ithkuil » Sun, Apr 21, 2019, 4:03 PM UTC

lhorie wrote: post #27

ek750 wrote: post #25

Earlier quoted context omitted.

Can you explain why? I’d be interested to learn more.

In principle, you want the output of a build to always be the same if the inputs are the same. In terms of correctness, you want "sameness" to be defined in terms of the contents of the input, not their timestamp, since timestamps can be easily be changed inadvertently by things like `touch`, 3rd party tools, etc. Also, relying on timestamps could pose problems for caching/checksums/etc if they are printed into any t…

Furthermore, timestamp precision is still problematic. Some filesystems still have one second granularity, but even if you use a filesystems capable of storing sub-millisecond timrstamps, that doesn't mean commands that get invoked as part of the build process (e.g. cp -p) invoke the right APIs that make use of such precision. e.g. see the doc for the .LOW_RESOLUTION_TIME directive of GNU make

Re: Buck: A high-performance build tool

#32

Post by entelechy » Sun, Apr 21, 2019, 4:18 PM UTC

dikei wrote: post #28

Can Buck handle Maven dependencies yet ? We don't want to vendor hundreds of jars into our repositories like before maven.

No but this will be covered by buckaroo [1]

[1] https://github.com/LoopPerfect/buckaroo/issues/314

Re: Buck: A high-performance build tool

#33

Post by nine_k » Sun, Apr 21, 2019, 4:19 PM UTC

blattimwind wrote: post #14

> Buck looks at the contents of your inputs, not their timestamps to figure out what needs to be built. As a result, incremental builds should always be correct, so there's no need to perform a clean build. At least one of these new-fangled tools gets at least one part right. The salient point is now of course whether Buck considers the correct set of dependencies as well as negative dependencies.

BTW take a look at tup: it tries to infer dependencies by watching which files each build step actually uses.

Re: Buck: A high-performance build tool

#34

Post by ithkuil » Sun, Apr 21, 2019, 4:30 PM UTC

ek750 wrote: post #25

blattimwind wrote: post #15

Earlier quoted context omitted.

Well many build systems don't even try to work properly, many new ones included. E.g. anything using timestamps is unequivocally an incorrect build system.

Can you explain why? I’d be interested to learn more.

Apart from issues pertaining timestamp granularity and clock skew, there is a problem that is technically not about timestamps but is related since it's often a consequence of the dependency data model employed by that class of build systems.

Consider this makefile rule:

foo.a: $(patsubst %.c,%.o,$(wildcard *.c))

Now, the foo.a target will be considered as stale if any of timestamps of it's dependencies is newer than foo.a

But what if you remove a .c file?

As build systems like make don't capture a fingerprint of the names of the inputs of a built (let alone the hash of their contents) it's very hard for them to deal with that case correctly.

Correctness is important otherwise your trust in the incremental builds quicky erodes and you start doing clean builds all the times you get an error, just in case.

Re: Buck: A high-performance build tool

#35

Post by antpls » Sun, Apr 21, 2019, 4:40 PM UTC

malkia wrote: post #30

Also (bazel already mentioned, so no need to), but: Pants - https://www.pantsbuild.org/index.html Please.Build - https://please.build/ Closely related, but functioning bit different: Gn - https://gn.googlesource.com/gn/ (targets ninja). Used by Chromium, Fuchsia, and others Soong - https://android.googlesource.com/platform/build/+/master/REA... (targeting Kati?)

Would it be interesting to list Infrastructure-as-Code tools and cloud pipeline tools in parallel ? At some point, build tools and infrastructure tools may converge :

Kubeflow Pipeline (for ML) - https://www.kubeflow.org/docs/pipelines/pipelines-overview/

Tekton Pipeline - https://github.com/tektoncd/pipeline

TF Extended - https://www.tensorflow.org/tfx

Re: Buck: A high-performance build tool

#36

Post by mgoblu3 » Sun, Apr 21, 2019, 4:55 PM UTC

lhorie wrote: post #23

duality wrote: post #8

Can anyone who's used both compare Buck to Blaze/Bazel?

Uber is migrating away from Buck to Bazel. One major reason is poor support for a variety of languages (Go being one big one, where fixing issues was historically slow due to a need to upstream fixes to Buck core). I haven't worked with Buck myself, but colleagues who evaluated it for JS have expressed concerns with lack of support/ecosystem there as well. In comparison, there are various Bazel rulesets for JS/Typesc…

Does this include mobile projects? I know Uber was/is pretty big on the Buck migration

Re: Buck: A high-performance build tool

#37

Post by kadaj » Sun, Apr 21, 2019, 5:28 PM UTC

Software development is all about not re-inventing the wheel. Yet there are countless build systems, tons of pointless frameworks. I wonder developers have nothing else to do. Every company releases their own version of everything.

Re: Buck: A high-performance build tool

#38

Post by gecko » Sun, Apr 21, 2019, 5:29 PM UTC

dikei wrote: post #28

Can Buck handle Maven dependencies yet ? We don't want to vendor hundreds of jars into our repositories like before maven.

As noted below, no, but Bazel now officially does, if you'd like something similar (this is new as of a couple weeks ago): https://github.com/bazelbuild/rules_jvm_external

Re: Buck: A high-performance build tool

#39

Post by amelius » Sun, Apr 21, 2019, 5:58 PM UTC

kadaj wrote: post #37

Software development is all about not re-inventing the wheel. Yet there are countless build systems, tons of pointless frameworks. I wonder developers have nothing else to do. Every company releases their own version of everything.

https://en.wikipedia.org/wiki/Not_invented_here

Re: Buck: A high-performance build tool

#40

Post by nimih » Sun, Apr 21, 2019, 5:59 PM UTC

kadaj wrote: post #37

Software development is all about not re-inventing the wheel. Yet there are countless build systems, tons of pointless frameworks. I wonder developers have nothing else to do. Every company releases their own version of everything.

My understand of Buck (and Pants) is that they are the result of ex-Google employees going to other companies (resp. Facebook, Twitter), realizing that Blaze was more or less the Right Way to do a build system (at least for their set of circumstances), and then being forced to re-implement the ideas from scratch (edit: or memory, or exfiltrated docs/code) because Blaze was not open source. Reusing extant software is great, but it requires that software to be available to you in the first place.