Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker is deleting Open Source organisations - what you need to know

blog.alexellis.io

Post reply on HN
Print view View original

581–590 of 764 posts

Re: Docker is deleting Open Source organisations - what you need to know

#581

Post by phaedrus »

caeril wrote: post #572
dbingham wrote: post #230

As an SRE Manager, this is causing me a hell of a headache this morning. In 30 days a bunch of images we depend on may just disappear. We mostly depend on images from relatively large organizations (`alpine`, `node`, `golang`, etc), so one would want to believe that we'll be fine - they're all either in the open source program or will pay. But I can't hang my hat on that. If those images disappear, we lose the abilit…

This whole thing is so weird. Why do so many organizations depend on the internet to function? It wasn't too long ago that it was standard practice to vendor your dependencies; that is, dump your dependencies into a vendor/ directory and keep that directory updated and backed up. But now, you all think it's 100% acceptable to just throw your hands up if github is down, or a maven repository is down, or docker hub mak…

There are some places that still work the old way, such as where I work - and we're finding we're increasingly out-of-touch with younger developers who grew up in a connected world. We had a recent college grad engineer (developer) who didn't work out as a hire. Some examples of the disconnect:

Try as I might, I couldn't get him to understand the difference between "git" the tool and "Github" the website. He kept making me nervous because he'd slip up and use the two terms interchangeably. (We have sensitive data that shouldn't be uploaded to the cloud.)

He didn't seem to completely understand files/folders and the desktop metaphor. He didn't seem to understand the difference between personal devices and work devices.

The last straw for our boss to let him go was he turned in a project that used a free web service on the cloud to upload data and get back the rows sorted. (Refer back to what I said above about: sensitive data.)

It didn't appear he was being obstinate, it was a tech-cultural difference. "Radical semantic disconnect" as I've seen the term used in science fiction.

Re: Docker is deleting Open Source organisations - what you need to know

#582

Post by [deleted] »

chaxor wrote: post #495
screamingninja wrote: post #221

Earlier quoted context omitted.

BitTorrent will solve the distribution problem but not magically provide more storage. Someone still has to foot the bill for storing gigabytes (or terabytes) worth of docker images.

This doesn't make sense as an argument at all. If there isn't anyone using the image, no one will have it on their computer... Sure - but that isn't as much of an issue if you have a build file that constructs the image up from more basic parts. Secondly, the popular files get way way faster with the more their used/downloaded. Torrent is a phenomenal *wonderful* system to distribute Machine Learning weights, docker…

[deleted]

Re: Docker is deleting Open Source organisations - what you need to know

#583

Post by web3-is-a-scam »

dbingham wrote: post #230

As an SRE Manager, this is causing me a hell of a headache this morning. In 30 days a bunch of images we depend on may just disappear. We mostly depend on images from relatively large organizations (`alpine`, `node`, `golang`, etc), so one would want to believe that we'll be fine - they're all either in the open source program or will pay. But I can't hang my hat on that. If those images disappear, we lose the abilit…

This is why our team vendors the images we depend on into AWS Elastic Container Registry.

Re: Docker is deleting Open Source organisations - what you need to know

#584

Post by justinclift »

yamtaddle wrote: post #294
justinclift wrote: post #172

Earlier quoted context omitted.

While that's true, for the amount of network traffic they're likely moving around, I wonder where they're placing their servers. eg something like AWS with massive data transfer costs, vs something else like carefully placed dedicated/colocation servers at places which don't charge for bandwidth

If it's AWS, they've surely got a huge discount. No way they're paying 8+x normal big-fish CDN rates for transfer. At their scale, it would have easily been worth the effort to move to something cheaper than AWS long ago, or else to negotiate a far lower rate.

> No way they're paying 8+x normal big-fish CDN rates for transfer.

While you're probably right, I've seen dumber things happen so I wouldn't completely rule out the possibility. :wink:

Re: Docker is deleting Open Source organisations - what you need to know

#585

Post by web3-is-a-scam »

cpitman wrote: post #412
dbingham wrote: post #230

As an SRE Manager, this is causing me a hell of a headache this morning. In 30 days a bunch of images we depend on may just disappear. We mostly depend on images from relatively large organizations (`alpine`, `node`, `golang`, etc), so one would want to believe that we'll be fine - they're all either in the open source program or will pay. But I can't hang my hat on that. If those images disappear, we lose the abilit…

Many of the responses here are talking about how to vendor/cache images instead of depending on an online registry, but remember that you also need access to a supply chain for these images. Base images will continue to be patched/updated, and you need those to keep your own images up to date. Unless the suggestion is to build all images, from the bottom up, from scratch.

Typically when you "cache" something, you're gonna expire it at some point...no? If the image is patched, it eventually gets refreshed in the mirror. If the image dissapears at least we still have it until we figure out where the heck it went.

Re: Docker is deleting Open Source organisations - what you need to know

#586

Post by web3-is-a-scam »

Woodi wrote: post #478
friendzis wrote: post #333

Earlier quoted context omitted.

> If those images disappear, we lose the ability to release and that's not acceptable. left-pad moment once again. > I mean, 30 days isn't enough time to find alternatives and migrate. Maybe take control of mission critical dependencies and self-host?

> Maybe take control of mission critical dependencies and self-host? Last few years prove that this option is a no-go - they just don't do such things ! Independence ? Self-sufficiency ? Security ? Local, fast access ? Obviousness ? No payment required ? Avoid at all costs !

taking responsibility for our supply chain and things we depend on that use mostly for free? absolutely preposterous, the business demands more features.

Re: Docker is deleting Open Source organisations - what you need to know

#587

Post by web3-is-a-scam »

ZiiS wrote: post #317
nine_k wrote: post #254

Earlier quoted context omitted.

Good thing is that setting up an image registry in AWS is so simple! (Ha-ha, only serious.)

Is https://aws.amazon.com/ecr/ no good?

It's great. We migrated our images to it months ago (not because of this, bandwidth issues mainly, we also vendor our base images on it) and it has given us exactly 0 problems.

Re: Docker is deleting Open Source organisations - what you need to know

#588

Post by ikiris »

bushbaba wrote: post #246
ohgodplsno wrote: post #233

Earlier quoted context omitted.

Or you could, you know, host a Docker registry and reupload those images to something you control. Worst case scenario, in 30 days, nothing is gone from Docker and you can just spin it down. Your job as an SRE is not to look at things and go "oh well, nothing we can do lol".

That’s unplanned work. There’s other work needing to be done as well.

I'm sure docker will happily hold off on this work until you can fit it into your OKR planning next quarter. /s

Re: Docker is deleting Open Source organisations - what you need to know

#589

Post by eecc »

dbingham wrote: post #230

As an SRE Manager, this is causing me a hell of a headache this morning. In 30 days a bunch of images we depend on may just disappear. We mostly depend on images from relatively large organizations (`alpine`, `node`, `golang`, etc), so one would want to believe that we'll be fine - they're all either in the open source program or will pay. But I can't hang my hat on that. If those images disappear, we lose the abilit…

> If those images disappear, we lose the ability to release and that's not acceptable

It’s your responsibility to ensure your own business continuity. You should review how your build pipeline depends on resources outside of your org perimeter, and deploy a private registry under your own control.

btw, you could also contribute some mirroring bandwidth to the community. You must’ve heard that the cloud is just someone else’s computer.

Re: Docker is deleting Open Source organisations - what you need to know

#590

Post by flyinghamster »

jjav wrote: post #477
dbingham wrote: post #230

As an SRE Manager, this is causing me a hell of a headache this morning. In 30 days a bunch of images we depend on may just disappear. We mostly depend on images from relatively large organizations (`alpine`, `node`, `golang`, etc), so one would want to believe that we'll be fine - they're all either in the open source program or will pay. But I can't hang my hat on that. If those images disappear, we lose the abilit…

> If those images disappear, we lose the ability to release and that's not acceptable. This shines light on why it is so risky (from both availability and security perspectives) to be dependent on any third party for the build pipeline of a product. I have always insisted that all dependencies must be pulled from a local source even if the ultimate origin is upstream. I am continuously surprised how many groups simpl…

Be sure to archive your development tools as well, just in case that rug gets pulled. You don't want to be in the position that you need v3.1415927 of FooWare X++ because version 4 dropped support for BazQuux™, only to find that it's no longer downloadable at any price.
Post reply on HN