Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker is deleting Open Source organisations - what you need to know

blog.alexellis.io

Post reply on HN
Print view View original

551–560 of 764 posts

Re: Docker is deleting Open Source organisations - what you need to know

#551

Post by tetha »

bravetraveler wrote: post #471
softfalcon wrote: post #355

Earlier quoted context omitted.

First of all, want to say, that sounds deeply frustrating. Secondly, if this is a serious worry. I would recommend creating your own private docker registry. https://docs.docker.com/registry/deploying/ Then I would download all current versions of the images you use within your org and push them up to said registry. It’s not a perfect solution, but you’ll be able to pull the images if they disappear and considering t…

As someone who maintains the registries we use globally at work, +1. I know people groan at running infrastructure, but the registry software is really well documented and flexible. If you don't need to 'push', but only pull - configuring them as pull through caches is nice for availability and reliability -- while also saving from nickle/diming. They will get things from a configurable upstream, proxy.remoteurl . Co…

That's good to hear. So I'll just have to spend an hour or so tomorrow night ensuring our private pull-through registry is used on everything prod and the biggest explosion is averted. Images built by the company land in internal registries already, so that's fine as well.

Means, it's mostly a question of, (a) checking for image squatting on the hub after orgs get deleted, which I don't know how to deal with just yet (could I just null-route the docker hub on my registry until evaluated and we just don't get new images?), and (b) ruffling through all of our container systems to see where people use what image to figure out which are verified, or paying, or obsoleted, and where they went, or what is going on. That'll be great fun.

Re: Docker is deleting Open Source organisations - what you need to know

#552

Post by zelphirkalt »

college_physics wrote: post #352
jon-wood wrote: post #13

Docker the tool has been a massive benefit to software development, every now and then I have a moan about the hassle of getting something bootstrapped to run on Docker, but it's still worlds better than the old ways of managing dependencies and making sure everyone on a project is aligned on what versions of things are installed. Unfortunately Docker the company appears to be dying, this is the latest in a long line…

Can't comment specifically on this or that "dying company", but it is a bit disappointing that after, how many, four decades of open source? and the obvious utility of that paradigm, it still seems a major challenge to build sustainable open source ecosystems. This means we can't really move on and imagine grander things that might build on top of each other. Its not clear if that is due to: i) competition from propr…

With lots of open source licenses, there is no copyleft. Without copyleft, for profit companies can simply take the hard work, add a little on top, make it proprietary, and sell it. Customer mentality is to use the most comfortable thing, without paying attention on whom they depend, often choosing the proprietary offer, because of feature X.

There are healthy ecosystems, even some partially replacing docker, some with more daily updates than I can process, but they have copyleft licenses in place and are free software, to ensure contributions flow back. Companies can still make profit, but not from adding a minimalistic thing and making it proprietary. They need to find other ways.

Re: Docker is deleting Open Source organisations - what you need to know

#553

Post by bravetraveler »

tetha wrote: post #551
bravetraveler wrote: post #471

Earlier quoted context omitted.

As someone who maintains the registries we use globally at work, +1. I know people groan at running infrastructure, but the registry software is really well documented and flexible. If you don't need to 'push', but only pull - configuring them as pull through caches is nice for availability and reliability -- while also saving from nickle/diming. They will get things from a configurable upstream, proxy.remoteurl . Co…

That's good to hear. So I'll just have to spend an hour or so tomorrow night ensuring our private pull-through registry is used on everything prod and the biggest explosion is averted. Images built by the company land in internal registries already, so that's fine as well. Means, it's mostly a question of, (a) checking for image squatting on the hub after orgs get deleted, which I don't know how to deal with just yet…

The typical Docker registry software when configured as a 'pull through' doesn't allow for pushes, if memory serves. That may be an important consideration while handling the situation

We run them in 'maintenance mode' just to be absolutely sure anything the upstream doesn't have (or had at one point) is permitted in!

Though, I don't think they'll allow pushes anyway with 'proxy.remoteurl' defined.

I'm not sure I followed your setup properly, but with the private registry defined as your 'proxy.remoteurl', you shouldn't have to worry about the Hub in particular - unless it's looking there, or people are pushing bad things into it

Re: Docker is deleting Open Source organisations - what you need to know

#554

Post by tetha »

BlueTemplar wrote: post #514
dahdum wrote: post #332

Earlier quoted context omitted.

Not every open source project wants to deal with donations / payments that could force incorporation, tax filings, bank accounts, credit/debit cards, and other paperwork. I certainly wouldn't want to deal with that for a side project.

If you are part of an organization, you already need to deal with most of those ?

You might need to deal with this on the receiving end.

Re: Docker is deleting Open Source organisations - what you need to know

#555

Post by tetha »

bravetraveler wrote: post #553
tetha wrote: post #551

Earlier quoted context omitted.

That's good to hear. So I'll just have to spend an hour or so tomorrow night ensuring our private pull-through registry is used on everything prod and the biggest explosion is averted. Images built by the company land in internal registries already, so that's fine as well. Means, it's mostly a question of, (a) checking for image squatting on the hub after orgs get deleted, which I don't know how to deal with just yet…

The typical Docker registry software when configured as a 'pull through' doesn't allow for pushes, if memory serves. That may be an important consideration while handling the situation We run them in 'maintenance mode' just to be absolutely sure anything the upstream doesn't have (or had at one point) is permitted in! Though, I don't think they'll allow pushes anyway with 'proxy.remoteurl' defined. I'm not sure I fol…

> I'm not sure I followed your setup properly, but with the private registry defined as your 'proxy.remoteurl', you shouldn't have to worry about the Hub in particular - unless it's looking there, or people are pushing bad things into it

That is exactly the thing I am worried about, as we have a pull-through mirror for the docker hub.

What happens if some goofus container from that chaotic team pulls in knownOSS/component, but knownOSS got deleted and - after 30 days of available recon by _all_ malicious teams on the planet - got squatted instantly afterwards with rather vile malware? Spend some pennies to make a dollar by getting into a lot of systems.

Obviously, you can throw a million shoulds at me, shouldn't do that, should rename + vendor and such (though how would you validate the image you mirror.), but that's a messy thing to deal with and I am wondering about a centralized way to block it without needing anyone but the registry/mirror admins.

Re: Docker is deleting Open Source organisations - what you need to know

#556

Post by planetafro »

ohgodplsno wrote: post #233
dbingham wrote: post #230

As an SRE Manager, this is causing me a hell of a headache this morning. In 30 days a bunch of images we depend on may just disappear. We mostly depend on images from relatively large organizations (`alpine`, `node`, `golang`, etc), so one would want to believe that we'll be fine - they're all either in the open source program or will pay. But I can't hang my hat on that. If those images disappear, we lose the abilit…

Or you could, you know, host a Docker registry and reupload those images to something you control. Worst case scenario, in 30 days, nothing is gone from Docker and you can just spin it down. Your job as an SRE is not to look at things and go "oh well, nothing we can do lol".

This exactly. We have pipelines designed specifically for this reason. We pull, patch, and perform minor edits to images we use. We then version lock all-the-things for consistency.

Not saying this is good news, but in the Enterprise, you have to plan for shit like this.

Re: Docker is deleting Open Source organisations - what you need to know

#557

Post by MarkSweep »

One annoyance with how docker images are specified is they include the location where they are stored. So if you want to change where you store you image you break everyone.

I wonder if what regsitry.k8s.io does could be generalized:

https://github.com/kubernetes/registry.k8s.io/blob/main/cmd/...

The idea is the depending on which cloud you are pulling the image from, they will use the closest blob store to service the request. This also has the effect that you could change the source of truth for the registry without breaking all Dockerfiles.

Re: Docker is deleting Open Source organisations - what you need to know

#558

Post by dralley »

yjftsjthsd-h wrote: post #507
dralley wrote: post #442

Earlier quoted context omitted.

>Or systemd, written in a memory unsafe language (yes, that is a problem for something so critical and was already exploited at least a couple of times) What memory safe language 1) existed in 2010 and 2) is thoroughly portable to every architecture people commonly run Linux on and 3) is suitable for software as low-level as the init? Rust is an option now but it wasn't back then. And Rust is being evaluated now, eve…

There's Ada.

Ada has no ecosystem, and a lot of the ecosystem that does exist is proprietary, and it brings us back to point #2.

Re: Docker is deleting Open Source organisations - what you need to know

#559

Post by wavesquid »

the_jeremy wrote: post #539
softfalcon wrote: post #355

Earlier quoted context omitted.

First of all, want to say, that sounds deeply frustrating. Secondly, if this is a serious worry. I would recommend creating your own private docker registry. https://docs.docker.com/registry/deploying/ Then I would download all current versions of the images you use within your org and push them up to said registry. It’s not a perfect solution, but you’ll be able to pull the images if they disappear and considering t…

I would not recommend doing it through Docker, though, especially after this change. We use AWS's ECR, and you can set it to do pull-through caching of public images, so images you've already used will stick around even if Docker blows up, and you don't have to pull the images yourself, you just point everything in your environment to ECR and ~~ECR will pull from docker hub~~ (EDIT: it only supports quay.io, not dock…

ECR pull through caching is only possible for other ECR repos or quay.io. you cannot use it for Docker hub.

Re: Docker is deleting Open Source organisations - what you need to know

#560

Post by dbingham »

jjav wrote: post #477
dbingham wrote: post #230

As an SRE Manager, this is causing me a hell of a headache this morning. In 30 days a bunch of images we depend on may just disappear. We mostly depend on images from relatively large organizations (`alpine`, `node`, `golang`, etc), so one would want to believe that we'll be fine - they're all either in the open source program or will pay. But I can't hang my hat on that. If those images disappear, we lose the abilit…

> If those images disappear, we lose the ability to release and that's not acceptable. This shines light on why it is so risky (from both availability and security perspectives) to be dependent on any third party for the build pipeline of a product. I have always insisted that all dependencies must be pulled from a local source even if the ultimate origin is upstream. I am continuously surprised how many groups simpl…

We can't go NIH for everything. If we do that we're back to baremetal in our own datacenters and that's expensive and (comparatively) low velocity. We have to pick and choose our dependencies and take the trade off of risk for velocity.

This is the tradeoff we made with the move to cloud. We run our workloads on AWS, GCP or Azure, use DataDog or New Relic for monitoring, use Github or GitLab for repos and pipelines, and so forth. Each speeds us up but is a risk. We hope they are relatively low risks and we work to ameliorate those risks as we can.

An organization like Docker should have been low risk. Clearly, it's not. So now it's a strong candidate for replacement with a local solution rather than a vendor to rely on.

Post reply on HN