Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Buck – A build system developed and used by Facebook

buckbuild.com

Post reply on HN
Print view View original

41–50 of 131 posts

Re: Buck – A build system developed and used by Facebook

#41

Post by asymmetric »

cwyers wrote: post #19
BinaryIdiot wrote: post #17

Earlier quoted context omitted.

To be fair a majority of the big technology companies all write their own tooling for many things that already exist because they didn't quite fit the way they needed them to and they had the resources to re-invent whatever they want. Sure when they open source them it further fragments the market and you always get the rush of "Facebook has almost 2 billion users therefore their tools must be the best!" which furthe…

When Facebook wrote this, the tooling that already existed wasn't available to them -- ex-Google engineers at Facebook wanted something like Bazel/Blaze, but Google had yet to release that publicly. So this is a Hadoop situation where Google had this internal tool that someone else really wanted and because Google didn't open-source it they wrote their own version.

Isn't there a risk that reimplementing a previous employer's tool constitutes a legal/contractual violation of some kind?

Re: Buck – A build system developed and used by Facebook

#42

Post by pjmlp »

LegNeato wrote: post #30
beefsack wrote: post #11

Continuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems. We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them. Ma…

When Buck was created (my team created it!) there wasn't anything that supported what Facebook needed in a build tool. There still really isn't as all the Blaze work-a-likes are focusing on various different needs. I would argue Buck actually helps to solve what you are concerned about. At Facebook, everything builds with Buck (and Google with Bazel I hear)...which means you learn it once and you know it for your Obj…

> At Facebook, everything builds with Buck (and Google with Bazel I hear).

Surely not on Android or Skia teams, unless they internally use something other than what AOSP and Skia use.

Re: Buck – A build system developed and used by Facebook

#43

Post by scrame »

beefsack wrote: post #11

Continuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems. We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them. Ma…

Nope. I totally agree. I don't trust a single thing from facebook. I mean, good on them that they open source their tools, but most everything that comes out is just a huge pain in the ass with very little benefit to adopt, other than we can say we're using open-source facebook technology.

The funny thing, though, is that keeping up with other web tech, you always have to deal with facebooks API's and sharing and whatnot, and its also, usually a huge pain in the ass.

So, like, what's the benefit of this, switching build systems, other than a tick on the resume for someone who wants to get a job at facebook?

Re: Buck – A build system developed and used by Facebook

#44

Post by jlg23 »

beefsack wrote: post #11

Continuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems. We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them. Ma…

I'm all for using existing tools, but not for the sake of itself. If there is a good reason (and everyone in the team agrees), all now obsolete tools should simply be replaced by the new one.

If this did not work in your office, this is most likely a problem of people in your office not communication properly about their choice of tools. Did you have people make a short presentation when they introduced a new tool? Did nobody ever point out that you already have various tools that can (easily/elegantly) accomplish what the next new thing(tm) is introduced for?

Re: Buck – A build system developed and used by Facebook

#45

Post by ProblemFactory »

m_mueller wrote: post #12
danso wrote: post #8

I don't work in a shop where performance/speed is important, but I am looking for other ways to do things I would do in Make but...not in Make. For example, my use-case is similar to what Mike Bostock described in "Why Use Make" [0] when explaining how he uses Make to build out his data tranformation process. Most of my work is data transformation/small-scale ETL, but I just haven't been able to get into Make beyond…

So far I've also always found `make` to be the best option for what I need, especially because it's already available everywhere (albeit sadly in very subtly incompatible versions). Anyways, one thing that has always bothered me about make is that it so much depends on file modification dates. Imagine if it would instead use a very optimised hashing algorithm over the input content. Content can be a file or any URI,…

I have briefly used https://github.com/Factual/drake for this.

It is a bit unpolished and still a work in progress, but has some features that are useful for data science workflows that most software build tools do not cover:

* Steps can produce multiple named outputs (for example full.csv and summary.csv), dependency graph can branch in both ways,

* Outputs that are directories with many files,

* Built-in input/output file sharding (for example by date or month) and HDFS support,

* Lots of control over which steps to execute: not just "rebuild target X and any required upstream dependencies". Can select a target and all that depend on it downstream (if you know that a remote file changed), skip rebuilding some steps, etc,

* Can put very short (text processing) scripts directly in the Drakefile in Python, Ruby.

Re: Buck – A build system developed and used by Facebook

#47

Post by paulddraper »

jhasse wrote: post #40
paulddraper wrote: post #27

Earlier quoted context omitted.

What tool would you have suggested Facebook improve? Saying that Facebook should have improved Make or Maven is like saying Linus should have improved CVS or SVN instead of competing with Git. They're in the same space, but they differ at a very fundamental level.

Waf or Meson

Facebook doesn't have timetravel, so contributing to Meson would have been impossible.

Re: Buck – A build system developed and used by Facebook

#48

Post by adriancooney »

danso wrote: post #8

I don't work in a shop where performance/speed is important, but I am looking for other ways to do things I would do in Make but...not in Make. For example, my use-case is similar to what Mike Bostock described in "Why Use Make" [0] when explaining how he uses Make to build out his data tranformation process. Most of my work is data transformation/small-scale ETL, but I just haven't been able to get into Make beyond…

I've often found myself in a similar situation as myself where I just can't seem to enjoy using Make and never make it past trivial things. For that reason, I'll shamelessly plug what I think is a viable alternative: the Taskfile.

https://hackernoon.com/introducing-the-taskfile-5ddfe7ed83bd

Re: Buck – A build system developed and used by Facebook

#49

Post by hultner »

paulddraper wrote: post #27
beefsack wrote: post #11

Continuing the trend of Facebook creating competing tools more often than persisting with and improving existing ones, which I feel dilutes effort and has fragmented a number of ecosystems. We've got a couple of Facebook fans at work, which has left us with a number of our systems using different tools to accomplish essentially the same tasks, and no strong case on either side for us to standardise on one of them. Ma…

What tool would you have suggested Facebook improve? Saying that Facebook should have improved Make or Maven is like saying Linus should have improved CVS or SVN instead of competing with Git. They're in the same space, but they differ at a very fundamental level.

Some of the tools seems more like improving existing tools such as yarn (instead of npm) and jest over jasmine. However I still applaud Facebook for releasing these tools since they often offer massive improvements over previous tools many regards. It might be more apparent if you're exposed to web development since the toolsets are changing constantly and quickly, some stability would be great in this space.

This doesn't mean that we should disencourage creating new tools fundamentally changing the ways we work with them. You mention Git and I see it as on great such example, I remember with anguish some of the SVN trunks, working with their pseudo tags and the mess which you'd often be introduced to.

Post reply on HN