Live data from Hacker News

Defence Against the Docker Arts

blog.heroku.com

71–80 of 103 posts

Re: Defence Against the Docker Arts

#71

Earlier quoted context omitted.

As well as Kaniko, Makisu, Orca and I've genuinely lost track. Though buildpacks differ from most of these by skipping Dockerfiles altogether.

Thanks for that list. If I'm not mistaken, Buildah and Bazel skip Dockerfiles as well; not sure of the three you just named.

I forgot Jib! It also skips Dockerfiles.

Re: Defence Against the Docker Arts

#72
post #70
post #59

Earlier quoted context omitted.

Your answer makes sense, but it actually makes me less excited about CNB. It sounds like CNB will break compatibility with the massive Dockerfile ecosystem, in exchange for... sometimes not downloading a layer? That is not appealing to me at all, because Dockerfiles are too embedded in my workflow, losing support for them is simply not an option. As for unprivileged builds, I don’t see any reason buildkit can’t suppo…

Image rebasing and layer reuse really matter at scale. Patching base images for many images simultaneously in a registry is a huge win if you are an organization running thousands of containers and there is a CVE in one of the OS packages in your base image. Optimizing data transfer similarly matters when you add up the gains across many containers. And devs like fast builds too :) I also don't really see how this pr…

The incompatibility I mentioned is with Dockerfiles.

You’re right that more flexible patching and optimizing transfers are valuable. But those problems are independent of build frontends: you could solve them once for both buildpacks and Dockerfiles. In fact buildkit is well on its way to doing exactly that.

Basically I would prefer if buildpacks and Dockerfiles could all be built with the same tooling. CNB seems like a wasted opportunity to do that, because it bundles two things that should be orthogonal: a new build format, and a new build implementation. Docker is going in the opposite direction by unbundling the format (Dockerfile) from the implementation (buildkit).

Re: Defence Against the Docker Arts

#74
post #45

I'm a novice docker user, but I found Dockerfiles to be probably the most direct, graspable, important part of docker. It's one readable text file used to recreate an entire environment. It's sort of a picture worth a thousand command lines. That said, I wish there was a way to get rid of all the && stuff, which is used to avoild writing a layer of the filesytem. Why not have something like: RUN foo RUN bar RUN bletc…

When I first encountered Cloud Foundry, here's the arduous journey I had to first undertake to learn how to use buildpacks: cf push Whereas with Dockerfiles I had to learn several commands and a lot of exciting gotchas.

I just like the philosophy of docker, although I am recognizing here the implementation has some warts.

also, if you're just a user of docker you might not care about dockerfiles. (like most people building software don't care about the Makefiles and prefer not to look at them)

Re: Defence Against the Docker Arts

#75
post #10

Here’s a little known fact: “docker build” can trivially be extended to build buildpacks or CNB. Now that the buildkit refactoring is complete, Dockerfiles are just the default frontend. There’s already a buildpack frontend in the community repo, and it works great. Writing your own frontend is real straightforward. Honestly after years of stagnation, the most exciting work on container building is now coming out of…

My favorite Docker BuildKit feature is SSH agent forwarding. Add "--mount=type=ssh" after RUN commands in Dockerfiles and the command will use your host machine's SSH agent. I've been able to greatly simplify a lot of Dockerfiles and CI build processes using it. There's a good introduction here: https://medium.com/@tonistiigi/build-secrets-and-ssh-forward...

This feature currently does not work with the OS X ssh agent:

https://github.com/docker/for-mac/issues/410

Re: Defence Against the Docker Arts

#76
post #23

You are setting up an entire operating system to install a single Microservice and just now noticed that you have redundancies? You could have the same issue by simply trying to rpmbuild your app. No really, you are just doing packaging. If you want more comfort, look into how redhat or Debian maintain their packages. They have similar problems and most likely they have mature solutions.

If you have a single service and will never want more - sure. Once you have multiple, you actually get benefits if you depend on a lot of native libraries and external binaries. If you're in that situation, not having to upgrade every service at the same time when moving to a new base os release is really convenient.

NixOS offers the same benefits(more actually). Without containers.

Re: Defence Against the Docker Arts

#77
post #19

Gitlab would benefit from this in the context of their autodevops featureset

Yes, this is something we're definitely investigating! We have an issue open at https://gitlab.com/gitlab-org/gitlab-ce/issues/55840, please join us in the conversation there if you have thoughts on how we can do this in a way that works well for your use cases.

Re: Defence Against the Docker Arts

#78

I love the fact Heroku have open-sourced their buildpacks. Dokku takes great use of these and provides a very similar platform to themselves that you can host yourself (DigitalOcean even provide a base-image that will pre-configure Dokku for you). Great for personal websites and the like. If you want to scale in a pinch then it's a case of making some tiny tweaks and pushing to Heroku instead.

I run some things at home using dokku and I switched them all to use Dockerfiles since the buildpacks are crazy slow. Not 100% sure if that is the fault of the dokku implementation or buildpacks in general though. It's all IO related, so I might not even have noticed it had I had faster disks.

They do seem quite slow. I don't think it's so much I/O related as I'm using a DigitalOcean VPS which has an SSD.

I'm not too fussed as I only deploy once in a blue moon if I need to quickly update my website or something. I can see it getting frustrating if you've got a larger product/app doing CI and deploys every merge to master.

Re: Defence Against the Docker Arts

#79
post #10

Here’s a little known fact: “docker build” can trivially be extended to build buildpacks or CNB. Now that the buildkit refactoring is complete, Dockerfiles are just the default frontend. There’s already a buildpack frontend in the community repo, and it works great. Writing your own frontend is real straightforward. Honestly after years of stagnation, the most exciting work on container building is now coming out of…

At the end of the day a docker image (or qemu, firecracker, etc...) is just a tarballed root filesystem. The funny thing to me is how seemingly overly complex the ecosystem has become for turning a blueprint for a Linux system into a tarball of files and folders. What am I missing?

Re: Defence Against the Docker Arts

#80
post #10

Here’s a little known fact: “docker build” can trivially be extended to build buildpacks or CNB. Now that the buildkit refactoring is complete, Dockerfiles are just the default frontend. There’s already a buildpack frontend in the community repo, and it works great. Writing your own frontend is real straightforward. Honestly after years of stagnation, the most exciting work on container building is now coming out of…

At the end of the day a docker image (or qemu, firecracker, etc...) is just a tarballed root filesystem. The funny thing to me is how seemingly overly complex the ecosystem has become for turning a blueprint for a Linux system into a tarball of files and folders. What am I missing?

Layers?
Post reply on HN