Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Just say no to :latest

platformers.dev

Post reply on HN
Print view View original

81–90 of 135 posts

Re: Just say no to :latest

#81

Post by ipsocannibal »

bspammer wrote: post #52
peakaboo wrote: post #26

Earlier quoted context omitted.

I've also seen the same. As soon as people start locking versions, that code is no longer updated and nobody will change it, because it's extra work to do so. I personally think running latest is the best thing to do. And if something fails, you downgrade it temporarily until the latest work again. It's pretty much opposite to what is recommended, and it's just the best solution in my opinion.

My company locks versions, but dependabot is configured on all of our repos. It automatically creates PRs to bump versions, and if CI passes for minor/patch bumps they get automatically merged. This takes a lot of the hassle out of the problem. For major bumps, a manual approval is required, but they happen infrequently enough that it's not a lot of work.

A similar tool to dependabot written by Salesforce: https://github.com/salesforce/dockerfile-image-update

Re: Just say no to :latest

#82

Post by raffraffraff »

I somewhat agree. But it's not that simple in practice. For example, I pin every single version and the container fails a security scan because it's not using latest versions of everything. Or I pin to arbitrary versions that happened to be latest when pip/npm/terraform first created the precious word-of-god lock file, leaving future engineers afraid to upgrade those packages because "Chesterton's Fence".

Pinning versions creates work in the future. Using latest versions can result in ad-hoc breakages and security risk. So instead of saying "v1.22=good, latest=bad" and talking generally about the fact that lock files exist in most decent systems, I'd like an article that contains example strategies for paying back the tech debt of a thousand arbitrarily pinned versions across your code projects.

I like the idea of deleting the lock file on a regular cadence, accepting whatever new shit comes in, perhaps bumping a major version number, and then testing the crap out of everything and leaving this new artifact in integration/staging for a few extra days. If things go wrong between builds, then you have the lock files in git (and in your artifacts) to let you know what changed. If it turns out that you now have a real reason to pin a specific package (ie: it broke something), then you have to find some way to note that outside of the usual lock files, since they arbitrarily lock everything.

Re: Just say no to :latest

#83

Post by ievans »

saulr wrote: post #7

Worth noting that Hadolint[1] raises warnings the issues mentioned in the article. Some examples of warnings: - https://github.com/hadolint/hadolint/wiki/DL3007 : Using latest is prone to errors if the image will ever update. Pin the version explicitly to a release tag. - https://github.com/hadolint/hadolint/wiki/DL3013 : Pin versions in pip. - https://github.com/hadolint/hadolint/wiki/DL3018 : Pin versions in apk ad…

Hadolint is great! If you want to customize your lint logic beyond the checks in it, I recently wrote a Semgrep rule to require all our Dockerfiles to pin images with a sha256 hash that could be a good starting point: https://github.com/returntocorp/semgrep-rules/pull/1861/file...

Re: Just say no to :latest

#84

Post by Jhsto »

In a perfect one would use CRIU [0] to suspend the old container, then download the new one and start it. If the service comes up with the new container, then remove the suspended one and otherwise, fallback to the suspended container.

[0]: https://criu.org/Docker

Re: Just say no to :latest

#85

Post by mr_tristan »

raffraffraff wrote: post #82

I somewhat agree. But it's not that simple in practice. For example, I pin every single version and the container fails a security scan because it's not using latest versions of everything. Or I pin to arbitrary versions that happened to be latest when pip/npm/terraform first created the precious word-of-god lock file, leaving future engineers afraid to upgrade those packages because "Chesterton's Fence". Pinning ver…

Pinning can be far less painful with tooling to review and manage all the changes. One example would be the Maven versions plugin: you can use it to query "what's been updated", update to next release, etc. The list of goals is pretty explanatory, and a good starting point: https://www.mojohaus.org/versions-maven-plugin/

When evaluating new toolchains, I often ask myself "what is the versions plugin equivalent here". There's often commercial offerings that help this process out.

Dependency management is one of these topics that rarely gets attention until much later in the ecosystem's lifetime. So chances are you'll need to find or create tooling to fill gaps.

I mean, you can do the whole "damn the torpedos and see what breaks" approach, but, I'd prefer a way to review changes. And I'd prefer doing those updates _separate_ from other changes.

Re: Just say no to :latest

#87

Post by solatic »

oxfordmale wrote: post #65
solatic wrote: post #35

Forget for a second the security concerns - If you lose :latest, and the build breaks, you need to spend unexpected time fixing the build, for whatever reason that :latest broke it, in order to continue with the business objective that you were actually tasked to do. Essentially, you have InfoSec muscling in and demanding, by the most coercive way possible, that Thou Shalt Not Do Your Job Without Bowing Before Me. Wh…

My default workflow is not to the pin the version. The build will only break occasionally and is often trivial to fix. In the rare case it does require significant amount of work, and management is breathing down your neck, you do temporarily pin the version to get out the release, and create a backlog ticket to resolve this as soon as possible. This workflow ensures any work for minor version upgrade is part of the…

> This workflow ensures any work for minor version upgrade is part of the development workflow. PMs can't deprioritise this work against more exciting shiny product features. If you include this as separate development tickets, it just tends to get deprioritised until it becomes a big chunk of work.

Instead of your maintenance work being visible, you're hiding it. You're describing an adversarial culture, one where you need to do shadow work because you don't trust management to prioritize necessary maintenance, perhaps one where management doesn't trust engineering to place maintenance in its proper context. I don't think this is an ideal, I think this is sad.

Re: Just say no to :latest

#88

Post by 88913527 »

raffraffraff wrote: post #82

I somewhat agree. But it's not that simple in practice. For example, I pin every single version and the container fails a security scan because it's not using latest versions of everything. Or I pin to arbitrary versions that happened to be latest when pip/npm/terraform first created the precious word-of-god lock file, leaving future engineers afraid to upgrade those packages because "Chesterton's Fence". Pinning ver…

Needlessly upgrading all of your packages leaves you open to supply chain attacks. If one of those NPM packages got quietly compromised (and my lockfile likely has 4-to-5 digits worth of dependencies in it), you invested time in making what amounts to an unneeded change to expose yourself.

Re: Just say no to :latest

#89

Post by lbriner »

thenerdhead wrote: post #62

The age old question is "How quickly do you update?" Too slow - Exposed to known vulnerabilities. Too fast - Susceptible to active supply chain attacks. There's tradeoffs, and there's no solution here. It's a hard problem.

Exactly this. The OP reads like a nice perfect world scenario but...

1) Just because there is a vulnerability in a package of library doesn't mean that you are susceptible to that vulnerability 2) If you don't update immediately when will you? Most companies can't afford people to understand all posted vulnerabilities to work out whether updating is better than not updating

The truth is there are easier ways to reduce risk than pretending that a pinned version somehow solves a lot of problems: Reduce your codebase/dependencies and apply defence-in-depth.

Reproducability is something for some companies but, again, I doubt there would be many times that I would care about what exactly was deployed 6 months ago. If something is broken now, it gets fixed now.

Re: Just say no to :latest

#90

Post by isbvhodnvemrwvn »

cuteboy19 wrote: post #61
EdwardDiego wrote: post #37

Earlier quoted context omitted.

I've used Maven version ranges to great effect... ...only projects that actually observe semver. (Jackson, take a bow for doing so with your patch releases. I wouldn't use it for your 'minor' releases though.)

Maven versions are not present in VCS if you use ranges. Wouldn't this cause issues like not know what Library version is being used in production and builds not being reproducible?

Not to mention that Maven doesn't have any mechanism for locking in transitive versions, because typically everyone uses hardcoded patch versions and never update them.
Post reply on HN