Live data from Hacker News

Buck – A build system developed and used by Facebook

buckbuild.com

71–80 of 131 posts

Re: Buck – A build system developed and used by Facebook

#71
post #19

Earlier quoted context omitted.

When Facebook wrote this, the tooling that already existed wasn't available to them -- ex-Google engineers at Facebook wanted something like Bazel/Blaze, but Google had yet to release that publicly. So this is a Hadoop situation where Google had this internal tool that someone else really wanted and because Google didn't open-source it they wrote their own version.

Isn't there a risk that reimplementing a previous employer's tool constitutes a legal/contractual violation of some kind?

It's almost certain that it is a legal violation and if a lawsuit was brought, I'd bet a million dollars in favour of Google; however, the tool itself isn't the final product and it's silly, counterproductive and just outright malicious to sabotage efforts at building good tooling in a competitor's company. Plus, it will set precedent and Google would have to vet every single new internal tool against what the previous employers of their employees were doing.

Re: Buck – A build system developed and used by Facebook

#72

Continuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems. We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them. Ma…

To be fair a majority of the big technology companies all write their own tooling for many things that already exist because they didn't quite fit the way they needed them to and they had the resources to re-invent whatever they want. Sure when they open source them it further fragments the market and you always get the rush of "Facebook has almost 2 billion users therefore their tools must be the best!" which furthe…

> To be fair a majority of the big technology companies all write their own tooling for many things that already exist because they didn't quite fit the way they needed them

Too many engineers syndrome comes to mind too.

Re: Buck – A build system developed and used by Facebook

#73
post #50

Earlier quoted context omitted.

Facebook doesn't have timetravel, so contributing to Meson would have been impossible.

Ah I didn't know that Buck was created before 2013. What about Waf though?

TBH, I never heard of Waf before. It looks interesting.

Buck, Waf, Meson, Bazel....apparently Python is the language for next-gen hash-based build tools :)

Re: Buck – A build system developed and used by Facebook

#74
post #35
post #8

I don't work in a shop where performance/speed is important, but I am looking for other ways to do things I would do in Make but...not in Make. For example, my use-case is similar to what Mike Bostock described in "Why Use Make" [0] when explaining how he uses Make to build out his data tranformation process. Most of my work is data transformation/small-scale ETL, but I just haven't been able to get into Make beyond…

These build systems become super handy once you're working on a large tree of many different software projects. For small projects not so much. It's probably easier to use the default build system of your programming language. When make fails me for personal stuff, I switch to ninja [1]. The Chromium people use it in their Bazel-like build system. A hidden gem I recently discovered is doit [2]. I found this one incre…

Thanks for the suggestions! That "doit" looks like it's simple enough for my fairly pedestrian needs. I wonder if my aversion to Make is that I don't spend enough time thinking/making Makefiles, but having something a little friendlier and simpler -- at the cost of ubiquity/portability -- might be the kind of training wheels I need.

Edit: FWIW, I posted doit to r/python and someone suggestion Snakemake, which also looks excellent and well-maintained: http://snakemake.readthedocs.io/en/stable/

Re: Buck – A build system developed and used by Facebook

#75
post #35
post #8

I don't work in a shop where performance/speed is important, but I am looking for other ways to do things I would do in Make but...not in Make. For example, my use-case is similar to what Mike Bostock described in "Why Use Make" [0] when explaining how he uses Make to build out his data tranformation process. Most of my work is data transformation/small-scale ETL, but I just haven't been able to get into Make beyond…

These build systems become super handy once you're working on a large tree of many different software projects. For small projects not so much. It's probably easier to use the default build system of your programming language. When make fails me for personal stuff, I switch to ninja [1]. The Chromium people use it in their Bazel-like build system. A hidden gem I recently discovered is doit [2]. I found this one incre…

> A hidden gem I recently discovered is doit [2]. I found this one incredibly helpful when I had a build with tons of 1:N, N:1, N:M dependencies.

> [2] http://pydoit.org/

Thanks for this! I need a build tool for a Python based data analysis project ("micro ETL"?). Doit looks like it will work nicely, with the bonus of keeping everything in Python.

Re: Buck – A build system developed and used by Facebook

#76

Continuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems. We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them. Ma…

I can't speak to the rest of Facebook's stuff, but I think the build tool problem is a special case. Per their docs: > Buck is designed for building multiple deliverables from a single repository (a monorepo) rather than across multiple repositories. It has been Facebook's experience that maintaining dependencies in the same repository makes it easier to ensure that all developers have the correct version of all of t…

This. Also, I think that the build system itself is just the tip of the iceberg. At least in Google's case it has recently been very nicely documented [1] that blaze is "just" one piece of how google keeps velocity high

[1] https://arxiv.org/abs/1702.01715

Re: Buck – A build system developed and used by Facebook

#77
post #55
post #51

Wonder if this can/will replace CMake for cross platform builds.

Doesn't support windows. You're better off with bazel or GN.

What about it doesn't support Windows? Not disagreeing since I haven't used it much and I'm on Linux, but they have "Quick Start" instructions for Windows.

https://buckbuild.com/setup/getting_started.html

Re: Buck – A build system developed and used by Facebook

#78

Continuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems. We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them. Ma…

That's not Facebook's fault, that's fanboyism's fault. I fail to see how the world gets worse from large companies releasing internal tools as open source.

It is very simple. Instead of improving , they started yet another brand new one.

From my years and years at large companies, these things start because they don't want to share anything in the first place, and then it's just pricey to maintain.

Re: Buck – A build system developed and used by Facebook

#79
> "Buck is a build system developed and used by Facebook."

I have really, really grown to resent this culture of proud and unabashed cargo-culting that we've arrived at in the open source world. Why is this the first sentence describing a new project? Why do we need a Facebook™-approved build system? Does that somehow make it better than the others? And why does Facebook need their own build system? Was the existing ecosystem technically insufficient for them, or was the issue a legal one?

Whenever I make this point in dev circles, someone will reply, "They serve X amount of visitors a day, so they must know something!" Well, they also have a firehose of ad money pointed at them 24/7.

Re: Buck – A build system developed and used by Facebook

#80
post #8

I don't work in a shop where performance/speed is important, but I am looking for other ways to do things I would do in Make but...not in Make. For example, my use-case is similar to what Mike Bostock described in "Why Use Make" [0] when explaining how he uses Make to build out his data tranformation process. Most of my work is data transformation/small-scale ETL, but I just haven't been able to get into Make beyond…

I have used Bazel (on Mac OS) to build a small C++ project. The nice thing about it is that it can be easily configured to download and build third-party dependencies from github. I have used this feature to add gflags, protobuf, and googletest dependencies to my project. That being said, it is still not perfect. For instance, I was not able to build GzipInputStream and GzipOutputStream (see e.g. https://github.com/google/protobuf/issues/2365).
Post reply on HN