Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Defence Against the Docker Arts

blog.heroku.com

Post reply on HN
Print view View original

61–70 of 103 posts

Re: Defence Against the Docker Arts

#61

Post by jkaplowitz »

zapita wrote: post #59
sclevine wrote: post #47

Earlier quoted context omitted.

While I really appreciate the work tonistiigi did to create the Cloud Foundry buildpack frontend for buildkit, it uses a compatibility layer[1] (which I wrote myself and no longer maintain) that only works with deprecated Cloud Foundry buildpacks that depend on Ubuntu Trusty. It doesn't work with the new, modular Cloud Native Buildpacks, and the buildpacks that ship with it are outdated (and vulnerable to various CVE…

Your answer makes sense, but it actually makes me less excited about CNB. It sounds like CNB will break compatibility with the massive Dockerfile ecosystem, in exchange for... sometimes not downloading a layer? That is not appealing to me at all, because Dockerfiles are too embedded in my workflow, losing support for them is simply not an option. As for unprivileged builds, I don’t see any reason buildkit can’t suppo…

Meanwhile some of us were already looking for alternatives to Dockerfiles due to roughly the same reasons described in the blog post and are excited about having this option. No connection to Pivotal or Heroku myself; and indeed the blog post came from Heroku, not Pivotal.

Re: Defence Against the Docker Arts

#62

Post by jkaplowitz »

mychael wrote: post #58

> creating Cloud Native Buildpacks (CNB), a standard for turning source code into Docker images without the need for Dockerfile This is a solution in search of a problem. Please stop.

I understand and accept that you're fine with Dockerfiles, but your statement is overbroad: Heroku and Pivotal aren't the only ones in the industry frustrated at the shortcomings of Dockerfiles. I'm glad alternatives are growing in number. (Others include Buildah, to be included in RHEL 8; and the Docker image support for Bazel.)

Re: Defence Against the Docker Arts

#63

Post by jacques_chester »

zapita wrote: post #59
sclevine wrote: post #47

Earlier quoted context omitted.

While I really appreciate the work tonistiigi did to create the Cloud Foundry buildpack frontend for buildkit, it uses a compatibility layer[1] (which I wrote myself and no longer maintain) that only works with deprecated Cloud Foundry buildpacks that depend on Ubuntu Trusty. It doesn't work with the new, modular Cloud Native Buildpacks, and the buildpacks that ship with it are outdated (and vulnerable to various CVE…

Your answer makes sense, but it actually makes me less excited about CNB. It sounds like CNB will break compatibility with the massive Dockerfile ecosystem, in exchange for... sometimes not downloading a layer? That is not appealing to me at all, because Dockerfiles are too embedded in my workflow, losing support for them is simply not an option. As for unprivileged builds, I don’t see any reason buildkit can’t suppo…

I'm not Stephen, but we've worked together on a few projects, including this one.

If you like Dockerfiles, you like Dockerfiles. Lots of people do. I did, until I'd used them for a while.

I'm not sure what you mean by "break compatibility". CNBs produce OCI images. They'll run on containerd just fine.

As for ecosystem: you'll note that the domain is heroku.com.

Re: Defence Against the Docker Arts

#64

Post by jacques_chester »

mychael wrote: post #58

> creating Cloud Native Buildpacks (CNB), a standard for turning source code into Docker images without the need for Dockerfile This is a solution in search of a problem. Please stop.

I work for Pivotal. I beg to differ. We see problems that Dockerfiles create at massive scale in massive organisations, mostly around predictable upgrades, provenance, ease of CVE remediation, not having tens of thousands of running containers with mystery meat and so forth.

Right now for one of our standard large customers, remediating a critical CVE may take several hours, whether they are using current-generation buildpacks or Dockerfiles and build farms. CNBs will drive that figure down to minutes. Thousands of distinct applications, dozens of sites, several tens of thousands of containers, billions of requests per day, patched a few minutes after the buildpack releases from automation observing hundreds of distinct upstream dependencies across a dozen language ecosystems.

Anyone with enough money and people and patience can build and maintain this kind of a capability for themselves. But it's a lot cheaper and easier to pay someone else to maintain it for you.

Re: Defence Against the Docker Arts

#65

Post by jacques_chester »

nhumrich wrote: post #60
m463 wrote: post #45

I'm a novice docker user, but I found Dockerfiles to be probably the most direct, graspable, important part of docker. It's one readable text file used to recreate an entire environment. It's sort of a picture worth a thousand command lines. That said, I wish there was a way to get rid of all the && stuff, which is used to avoild writing a layer of the filesytem. Why not have something like: RUN foo RUN bar RUN bletc…

The && is just layer squashing and not really needed. You can just use run on every line. Docker even supports squashing images now, so the && doesn't matter if you squash.

&& doesn't layer squash, it's a trick to avoid the layer commit that RUN implies.

And squashing comes with a major drawback: you lose layer caching and any hope of a vaguely-efficient rebuilding process.

Re: Defence Against the Docker Arts

#66

Post by jacques_chester »

jkaplowitz wrote: post #62
mychael wrote: post #58

> creating Cloud Native Buildpacks (CNB), a standard for turning source code into Docker images without the need for Dockerfile This is a solution in search of a problem. Please stop.

I understand and accept that you're fine with Dockerfiles, but your statement is overbroad: Heroku and Pivotal aren't the only ones in the industry frustrated at the shortcomings of Dockerfiles. I'm glad alternatives are growing in number. (Others include Buildah, to be included in RHEL 8; and the Docker image support for Bazel.)

As well as Kaniko, Makisu, Orca and I've genuinely lost track.

Though buildpacks differ from most of these by skipping Dockerfiles altogether.

Re: Defence Against the Docker Arts

#67

Post by codefinger »

kevinsimper wrote: post #24

I like they way they highlight that combining Ruby and nodejs makes for a complicated Dockerfile, while their example after only includes Ruby and not nodejs. And do they propose a buildpack called ruby-nodejs, because in many cases you don't need nodejs in your Ruby app. OMG now buildpack's are a leaky abstraction!

The Ruby buildpack shown in the example installs Node.js for you. A developer doesn’t even need to be aware that it’s required to precompile assets.

Re: Defence Against the Docker Arts

#68

Post by jkaplowitz »

jacques_chester wrote: post #66
jkaplowitz wrote: post #62

Earlier quoted context omitted.

I understand and accept that you're fine with Dockerfiles, but your statement is overbroad: Heroku and Pivotal aren't the only ones in the industry frustrated at the shortcomings of Dockerfiles. I'm glad alternatives are growing in number. (Others include Buildah, to be included in RHEL 8; and the Docker image support for Bazel.)

As well as Kaniko, Makisu, Orca and I've genuinely lost track. Though buildpacks differ from most of these by skipping Dockerfiles altogether.

Thanks for that list. If I'm not mistaken, Buildah and Bazel skip Dockerfiles as well; not sure of the three you just named.

Re: Defence Against the Docker Arts

#69

Post by rtpg »

wmf wrote: post #40

I just saw a submission on the "demise" of Cloud Foundry (a Heroku-like PaaS) that's relevant to this discussion: https://medium.com/@krishnan/lessons-from-the-demise-of-clou... Opinionated Platforms Are Risky: The CloudFoundry platform was more opinionated than some competing platforms in the market. In fact, the biggest debate between CloudFoundry and its direct competitors was about whether customers need opiniona…

Why can't we have a buildpack for a Database but Docker would be fine?

Re: Defence Against the Docker Arts

#70

Post by ekcasey »

zapita wrote: post #59
sclevine wrote: post #47

Earlier quoted context omitted.

While I really appreciate the work tonistiigi did to create the Cloud Foundry buildpack frontend for buildkit, it uses a compatibility layer[1] (which I wrote myself and no longer maintain) that only works with deprecated Cloud Foundry buildpacks that depend on Ubuntu Trusty. It doesn't work with the new, modular Cloud Native Buildpacks, and the buildpacks that ship with it are outdated (and vulnerable to various CVE…

Your answer makes sense, but it actually makes me less excited about CNB. It sounds like CNB will break compatibility with the massive Dockerfile ecosystem, in exchange for... sometimes not downloading a layer? That is not appealing to me at all, because Dockerfiles are too embedded in my workflow, losing support for them is simply not an option. As for unprivileged builds, I don’t see any reason buildkit can’t suppo…

Image rebasing and layer reuse really matter at scale. Patching base images for many images simultaneously in a registry is a huge win if you are an organization running thousands of containers and there is a CVE in one of the OS packages in your base image. Optimizing data transfer similarly matters when you add up the gains across many containers.

And devs like fast builds too :)

I also don't really see how this project is "incompatible" with anything in the docker ecosystem. I hope Dockerfiles have a long and healthy life. CNBs are another method to build OCI compatible images that provides a lot of benefits for some types of users and use cases.

I expect that the pack cli will eventually build on top of and take advantage of Buildkit

Post reply on HN