Using Makefile(s) for Go

If you download some random tar file with go code in it, I agree that you should expect to be able to build it with only go installed. "go get" depends on this, and largely works well!

But the day to day act of developing a system that uses go involves more than just building a go binary. Your go program might depend on things like generated protocol buffers. You need some way to regenerate those when you edit the definitions. That then involves having the right version of protoc installed and also having the right version of protoc-gen-go. The go compiler can't help you there. go generate suffers from the same problem; it's not automatic, so you can pass it the wrong dependencies (generation tool flags, version of the generator, etc.).

People are using makefiles as a convenient place to write all these extra instructions. "What flags to I pass to protoc?" "What flags do I pass to docker build?" Why document it when you can "make protos" or "make container"?

Unfortunately, make isn't actually good at this. It doesn't version the generation tools (or itself), so you will end up with vastly different results on different machines. The result is things like a 300 line diff to a generated protobuffer because the second engineer to work on that file happened to have protoc 0.7.7 instead of protoc 0.6.42, or they installed protoc-gen-go@master instead of protoc-gen-go@v1.2.3. Make doesn't care. It exited with exit status 0, so it must have worked.

What started as a nice way to write down some instructions for hacking the code has now become a giant mess. Reasonable makefiles can only ever work for one person on one computer at one point in time. At that point, they might as well be a README.md. At least the README can mention the version numbers of the dependencies, and, most importantly, can wish the reader luck.

There are two long-term solutions. One is to only use go. Write a program that reads the protos at runtime. Write a program that runs your Typescript through a hand-written compiler whose source code lives in your project at runtime. Now you only need "go build". This is... impractical, though. It's a nice ideal, but you'll never get anything done in the real world.

So what you really need is a real build system that captures every dependency, knows about high level tasks ("make foo.proto available to a go program"), and knows every dependency between files like the go compiler does. With such a tool, you can get a working build on every computer with no instructions or manual setup. And since it is carefully written to understand what it's doing, you can get reliable incremental builds. (The full build and your incremental build should have the exact same md5sum of the resulting binary.)

Such a tool does not exist. bazel is close. If you have to build more than just go files, you probably want to look into it. It's crazy. It's a lot of work. Don't do it if you're the only developer on the project. But if you want 10 random people to be able to build a project that's written in more than one language, you have to invest in some sort of tooling. Make is good for a one person team. Make can be scaled to do crazy things poorly (hi, Buildroot!). But it's probably not what you want to be using. If a README isn't good enough, you need a real build system.

I don’t mind Makefiles, and usually use them for basic command configuration.

However, when the logic gets even a little complicated and I almost always reach for bash. Everybody knows a little bash, and it’s available in almost all systems.

rm -rf ${APP} is a code smell. If ${APP} is not a directory, -r should not be in this command. At best it is possibly confusing, and at worst if somehow ${APP} accidentally becomes a directory it will just remove it and you will have no idea that it was a directory, whereas just rm -f ${APP} will fail because it can't unlink a directory. Build success is an important factor in a CI/CD pipeline, therefore builds should fail immediately under unexpected behavior.

Also, on .PHONY on a single line:

  But for Makefiles which grow really big, this is not suggested as it
  could lead to ambiguity and unreadability, hence the preferred way is
  to explicitly set phony target right before the rule definition.

If your Makefile grows really big, it's going to become a nightmare to maintain. Either split up your codebase + builds into sub-directories, or figure out some other way to structure your builds so that it's not super complicated to reason about or maintain them.

I went through a phase of using Grunt, Gulp, npm scripts, etc. for web projects.

Recently, after reading this article on make[1], I decided to give it a try and I've been pleasantly surprised with how much easier using make has been vs. some of the FUD I've read about it.

That's not to say there aren't some issues, but that's true of something that's been around since the '70s, including Unix itself.

But that also means it can address almost any situation where files need to be generated and there are dependencies involved, including what I'm doing with generating a static website, compiling and minifying CSS files (or not minifying but including a sourcemap if it's a dev build) and deploying to a production or staging server.

I don't love the syntax but it’s not that bad once you get used to it. Actually, I prefer it to the seemingly endless nesting of JavaScript objects in a Grunt or Gulp configuration file. My next step is to take advantage of Vim’s built-in support for a make-based workflow.

[1]: https://www.olioapps.com/blog/the-lost-art-of-the-makefile/

This Makefile could as well have been a shell script. It doesn't track changes to dependencies even when it's obvious how to do so. For example, the build rule has an obvious dependency (main.go) and an obvious target ($(APP)). Instead of tracking these which IMO is the primary advantage of using Make, it deliberately destroys the existing build. docked-build always necessarily rebuilds the binary as well

Presumably, Go has some kind of build cache making such dependency tracking relatively useless anyway, maybe Docker has too, but if you aren't tracking dependencies and rebuilding only when necessary why use Make instead of a big switch in a shell script?

Personally I'd only use Make for Go if I introduce some task that takes significant time and isn't already handled by the go toolchain.

Another couple of notes: there are two docker-push rules. The first seems like it was meant to be docker-build. The other is that the docker build rule will tag the build with the HEAD hash, regardless of whether it's building from a clean checkout or a dirty repo.