Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bazel – Correct, reproducible, fast builds for everyone

bazel.io

Post reply on HN
Print view View original

131–140 of 185 posts

Re: Bazel – Correct, reproducible, fast builds for everyone

#131

Post by lelandbatey »

davidgerard wrote: post #119
jacquesm wrote: post #91

Earlier quoted context omitted.

Convince your employer to ship a half decent unix environment with its OS and it will run on windows too. It's mostly a choice by microsoft to ship a half-baked command line interface with its products, you can't blame google for that.

How's it go with Cygwin or mingw?

I just want to boost big time for Cygwin, it really is amazing how much it improves the experience of using a Windows computer. It gives me the tools I want/need (gcc and python) and the programs I write run without modification in Cygwin, Linux and OS X!

Also, the default terminal emulator that comes with Cygwin, called MinTTY, is fantastic. It doesn't have the flashy features of many native Linux clients, but it has all the basic features one could want (full color support, nice fonts, very durable, no 'gotchas').

The only problem is that support outside of C-based languages is touch-and-go. Mainline Python works flawlessly since it's implemented in C, and all it's tools also work flawlessly (virtualenv, numpy, scipy, matplotlib, django, flask, etc), but other languages like Go or Rust don't really support Cygwin. I'm unaware how well other languages are supported (e.g. Ruby).

So Cygwin will definitely "give you that Unix feeling" on a Windows machine, but it can't always replace a virtual machine.

Re: Bazel – Correct, reproducible, fast builds for everyone

#132

Post by beagle3 »

thechao wrote: post #42

I've been burned by so many build tools over the years. I've finally settled (for C/++/asm) on the combination of Make + ccache: I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes. For instance, every rule that compiles a C/++ file is invoked if _any_ header/inc/template file changes. I let ccache do the precise timestamp/check-sum based analysis. The result is that (for l…

> I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes.

Do you have some kind of way to verify that your makefile dependencies conform to your source dependencies? Is clang/gcc tracking sufficient for your use case? What about upgrading the compiler itself, does your makefile depend on that? If so, how?

Have you considered tup[0]? Or djb-redo[1]? Both seem infinitely better than Make if you are paranoid. tup even claims to work on Windows, although I have no idea how they do that (or what the slowdown is like). Personally, I'm in the old Unix camp of many-small-executables, non of which goes over 1M statically linked (modern "small"), so it's rarely more than 3 secs to rebuild an executable from scratch.

> (deterministic mode for ar)

Why do you care about ar determinism? Shouldn't it be ld determinism you are worried about?

[0] http://gittup.org/tup/

[1] https://github.com/apenwarr/redo

Re: Bazel – Correct, reproducible, fast builds for everyone

#133

Post by davidgerard »

lelandbatey wrote: post #131
davidgerard wrote: post #119

Earlier quoted context omitted.

How's it go with Cygwin or mingw?

I just want to boost big time for Cygwin, it really is amazing how much it improves the experience of using a Windows computer. It gives me the tools I want/need (gcc and python) and the programs I write run without modification in Cygwin, Linux and OS X! Also, the default terminal emulator that comes with Cygwin, called MinTTY, is fantastic. It doesn't have the flashy features of many native Linux clients, but it ha…

Whenever I'm stuck on Windows, Cygwin is essential. (I use ssh from the Cygwin terminal.)

It's also great in production - tolerable command-line remote administration, and great for e.g. Nagios plugins that are bash scripts (vastly better than attempting to find a Windows-native plugin that someone else wrote).

Re: Bazel – Correct, reproducible, fast builds for everyone

#134

Post by justicezyx »

thechao wrote: post #42

I've been burned by so many build tools over the years. I've finally settled (for C/++/asm) on the combination of Make + ccache: I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes. For instance, every rule that compiles a C/++ file is invoked if _any_ header/inc/template file changes. I let ccache do the precise timestamp/check-sum based analysis. The result is that (for l…

They you should try this out.

Re: Bazel – Correct, reproducible, fast builds for everyone

#135

Post by rictic »

ovidiup wrote: post #130
OneMoreGoogler wrote: post #104

Earlier quoted context omitted.

> Any engineer can build any Google product from source on any machine A little too optimistic :) You can't build Android, Chrome, ChromeOS, iOS apps, etc. via blaze.

When I worked at Google I built a Blaze extension to be able to build Android apps. It worked really well, though I'm not sure how well it was maintained after I left in 2010. Internally at Google, Blaze was extremely customizable, and I hope Bazel too, so one can easily add support for building iOS apps etc. EDIT #1: I see support for building Objective-C apps is already present in Bazel. EDIT #2: Bazel uses Skylark…

There's an extension language in bazel named Skylark, which will be familiar to you if you wrote build_defs internally: http://bazel.io/docs/skylark/concepts.html

Re: Bazel – Correct, reproducible, fast builds for everyone

#136

Post by daxelrod »

sarnowski wrote: post #121
adambatkin wrote: post #84

Earlier quoted context omitted.

This is why you should have repository manager like JFrog Artifactory or Sonatype Nexus which can transparently proxy third-party repositories (like Maven Central).

This is true for Maven. The Maven repository project is very impressive and outstanding in its kind anyways. But leaving this great repository you are often on your own. Look at what happens to Google Code. How much stuff will be lost when its shut down? Hopefully nothing that you depend on. Its nice to have the binaries in a backup maybe but without access to the code, maintenance will be a nightmare.

Both of those products also handle other kinds of package repositories as well. You're right that they don't cover the full spectrum, however.

Re: Bazel – Correct, reproducible, fast builds for everyone

#137

Post by ambrop7 »

CurtHagenlocher wrote: post #79
krschultz wrote: post #31

Earlier quoted context omitted.

s/not running Windows/not running Windows or refusing to install a free VM/ 5 or 6 years ago I had to have Windows to run CAD software, but I found it easier to have a virtualbox install w/ Ubuntu in it for software development than trying to write code on Windows. The performance was good enough an the usability was pretty good. I imagine it has only gotten better since then.

Using a VM would only work if the individual build tools themselves run under Unix. That said, no open source project owes anybody anything.

Not quite. You can map shared folders to the guest and edit the code in the VM with your favorite editor, but build it on Windows.

Re: Bazel – Correct, reproducible, fast builds for everyone

#138

Post by georgehm »

lberki wrote: post #49
yarapavan wrote: post #34

Surprisingly, significant parts of the code is not open source. According to this page, http://bazel.io/docs/governance.html , Is Bazel developed fully in the open? Unfortunately not. We have a significant amount of code that is not open source; in terms of rules, only ~10% of the rules are open source at this point. We did an experiment where we marked all changes that crossed the internal and external code bases ov…

Currently about 60% of our code (in terms of lines of Java code, excluding tests) is open sourced. The rest is glue logic to internal Google systems or build rules that we haven't open sourced. Some of these rules, we are planning to open source in the future, and some others are specific to Google, so they don't really make much sense in the open source tree.

What about skyframe? http://bazel.io/docs/skyframe.html looks like an overview without any examples. Couldn't find any references to it in the bazel code at github too.

Re: Bazel – Correct, reproducible, fast builds for everyone

#140

Post by w4tson »

It's another impressive feat from Google and reading the comments I've kind of established that

1. Binaries are checked in to source 2. It's more structured than Gradle 3. It's for very large code bases 5. It's nix only

But...

1. We've already had the "chuck it in a lib directory" approach. The distributed approach maven/ivy etc seems to be working for the millions of developers out there who just have to get through the end of the day without production going up in flames. I suppose it's like moving a portion maven central into your code base. Checked in. Feels very odd, and kinda against one of the pillars of JVM: Maven. Love it or hate it it's one of most mature build/repository types out there. npm, bower anyone?

2. Got to agree with astral303. This isn't really something to shout about. Better reproducibility? Gradle/SBT have had incremental builds for quite a while. We all know there's no silver bullet, if you don't declare your inputs and outputs to gradle/blaze tasks or seed with random values then you're only going to get unrepoduceable builds.

3. Very large, I get that.

4. Very large code bases tend to enterprise systems. Enterprise systems tend to have a plethora of platforms/OSs so it being nix only is a drawback. However I suppose that if in charge of 10MLOC code base then I could mandate nix only builds? However in my experience they also tend to gravitate towards standards that seem to have longevity.

I'm yet to give it a go so I'll reserve final judgement. However I will say that I do wonder how far we'd be if Googles through their brightest minds at and worked with Maven/Gradle/SBT etc to scale their builds. (Yes I realise it's multi-lang - so is gradle). Perhaps the whole community would benefit from performance benefits.

Anyway hats off Google guys. It looks impressive and no doubt I'll jumping all over it in 12 months. In the mean time I'm off to go read up on Angular 2.0, or Typescript or ES6 or ES7 or whatever else I need* to know to get me through the day.

Really I'm just jealous I don't have 10MLOC code base :D

Post reply on HN