Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Just say no to :latest

platformers.dev

Post reply on HN
Print view View original

21–30 of 135 posts

Re: Just say no to :latest

#21

Post by peakaboo »

hpaavola wrote: post #18

Use latest during development, but push to production using image in your private Docker registry which has proper names and tags. git push -> Docker build (bonus points for building only if needed [for example only if Dockerfile, Jenkinsfile or requirements.txt has changed], otherwise use latest from Artifactory) -> run all automated tests -> if pass, push the Docker image to Artifactory with reasonable name and tag…

So what actually always happens with this approach is that companies stay on ancient docker images, because there is never time for updates as long as they work.

It's the same with locking python pip packages to specific versions. Nobody ever looks at it again and you run code that is 5 years old in production. People only look at it when it breaks.

Re: Just say no to :latest

#22

Post by yardstick »

oxfordmale wrote: post #13
yardstick wrote: post #11

Earlier quoted context omitted.

Doesn’t the same logic apply to using :latest? If you are always pulling the latest, unvetted, code, shouldn’t you also have a vulnerability management process in place? You need vulnerability & risk management for any dependency, full stop.

Ideally you only use Docker official images,or their equivalent to avoid using unvetted code. It is always a trade off, however it is far more likely that a hacker will use a ten year old well exploited CVE, rather than a recent one

I wouldn’t describe Docker official images as more or less secure than anything else. Supply chain attacks can easily slip a vulnerability in.

Re: Just say no to :latest

#23

Post by laumars »

oxfordmale wrote: post #13
yardstick wrote: post #11

Earlier quoted context omitted.

Doesn’t the same logic apply to using :latest? If you are always pulling the latest, unvetted, code, shouldn’t you also have a vulnerability management process in place? You need vulnerability & risk management for any dependency, full stop.

Ideally you only use Docker official images,or their equivalent to avoid using unvetted code. It is always a trade off, however it is far more likely that a hacker will use a ten year old well exploited CVE, rather than a recent one

> Ideally you only use Docker official images,or their equivalent to avoid using unvetted code.

Docker images don’t ship every dependency in the average development project. They’re also not a security guarantee either.

> It is always a trade off, however it is far more likely that a hacker will use a ten year old well exploited CVE, rather than a recent one

In general that’s the case but in practice that’s still the wrong mindset. You pin to a minor version and still get patch updates. Or run CVE checks to prove you’re dependencies have no reported vulnerabilities. And that will also mitigate the risk of you again accidentally updating to to new vulnerabilities (though you’d need to pin against patch version too if you want to be certain there)

Re: Just say no to :latest

#24

Post by yardstick »

peakaboo wrote: post #21
hpaavola wrote: post #18

Use latest during development, but push to production using image in your private Docker registry which has proper names and tags. git push -> Docker build (bonus points for building only if needed [for example only if Dockerfile, Jenkinsfile or requirements.txt has changed], otherwise use latest from Artifactory) -> run all automated tests -> if pass, push the Docker image to Artifactory with reasonable name and tag…

So what actually always happens with this approach is that companies stay on ancient docker images, because there is never time for updates as long as they work. It's the same with locking python pip packages to specific versions. Nobody ever looks at it again and you run code that is 5 years old in production. People only look at it when it breaks.

At least they would have never upgraded to log4j 2…

Re: Just say no to :latest

#25

Post by hpaavola »

peakaboo wrote: post #21
hpaavola wrote: post #18

Use latest during development, but push to production using image in your private Docker registry which has proper names and tags. git push -> Docker build (bonus points for building only if needed [for example only if Dockerfile, Jenkinsfile or requirements.txt has changed], otherwise use latest from Artifactory) -> run all automated tests -> if pass, push the Docker image to Artifactory with reasonable name and tag…

So what actually always happens with this approach is that companies stay on ancient docker images, because there is never time for updates as long as they work. It's the same with locking python pip packages to specific versions. Nobody ever looks at it again and you run code that is 5 years old in production. People only look at it when it breaks.

I just said that use latest exactly to avoid this issue.

Re: Just say no to :latest

#26

Post by peakaboo »

oxfordmale wrote: post #9

This is always a balance. The moment you pin to a specific version you need to have a process in place to ensure you regularly upgrade to avoid introducing vulnerabilities in your production system. Throughout my career I have seen many cases where certain software still runs on ancient versions as the team originally maintaining it is no longer around (e.g. reorganisations or lay-offs). It is always hard to convince…

I've also seen the same. As soon as people start locking versions, that code is no longer updated and nobody will change it, because it's extra work to do so.

I personally think running latest is the best thing to do. And if something fails, you downgrade it temporarily until the latest work again. It's pretty much opposite to what is recommended, and it's just the best solution in my opinion.

Re: Just say no to :latest

#27

Post by Kwpolska »

bheadmaster wrote: post #14
oxfordmale wrote: post #9

This is always a balance. The moment you pin to a specific version you need to have a process in place to ensure you regularly upgrade to avoid introducing vulnerabilities in your production system. Throughout my career I have seen many cases where certain software still runs on ancient versions as the team originally maintaining it is no longer around (e.g. reorganisations or lay-offs). It is always hard to convince…

In that case, it's better to just pin the major version of the container: FROM python:3 will work as long as Python3 doesn't make any backwards incompatible changes (note: Python3 occasionally deprecates then removes a feature that was part of the official API[0], so Python3 is not technically semver-correct). If, however, one wants automatic updates WITH reproducible builds, then a CI/CD pipeline that automatically…

Python never claimed to be semver-compliant. Also, if you’re using 3rd-party packages that require compiling stuff (eg. numpy), pre-built binary packages (aka wheels) might not be available on PyPI in the first few days (or weeks, or sometimes even months) after Python 3.x.0 is released, so you don’t want the version to be randomly changed under you. Moreover, I pulled python:3 a year ago, so I get 3.9.

FROM python:3.10 should generally be fine though.

Re: Just say no to :latest

#28

Post by speed_spread »

oxfordmale wrote: post #9

This is always a balance. The moment you pin to a specific version you need to have a process in place to ensure you regularly upgrade to avoid introducing vulnerabilities in your production system. Throughout my career I have seen many cases where certain software still runs on ancient versions as the team originally maintaining it is no longer around (e.g. reorganisations or lay-offs). It is always hard to convince…

Covert maintenance is the name of the game. If the site goes down because of a security flaw you'll be blamed anyway. Might as well get fired for doing the right thing.

I'm only half joking... The right thing to do with dummy management is to get everything in writing to make it clear that if anything happens it'll be their fault.

Re: Just say no to :latest

#29

Post by oefrha »

bheadmaster wrote: post #14
oxfordmale wrote: post #9

This is always a balance. The moment you pin to a specific version you need to have a process in place to ensure you regularly upgrade to avoid introducing vulnerabilities in your production system. Throughout my career I have seen many cases where certain software still runs on ancient versions as the team originally maintaining it is no longer around (e.g. reorganisations or lay-offs). It is always hard to convince…

In that case, it's better to just pin the major version of the container: FROM python:3 will work as long as Python3 doesn't make any backwards incompatible changes (note: Python3 occasionally deprecates then removes a feature that was part of the official API[0], so Python3 is not technically semver-correct). If, however, one wants automatic updates WITH reproducible builds, then a CI/CD pipeline that automatically…

This is very bad advice. Lots of nontrivial things aren't automatically compatible with the latest Python minor version. PyTorch for instance doesn't support Python 3.10 and it's been five months since the stable release (ten months since the first beta). For anything nontrivial, you almost always want to specify a minor Python version.

Re: Just say no to :latest

#30

Post by peakaboo »

hpaavola wrote: post #25
peakaboo wrote: post #21

Earlier quoted context omitted.

So what actually always happens with this approach is that companies stay on ancient docker images, because there is never time for updates as long as they work. It's the same with locking python pip packages to specific versions. Nobody ever looks at it again and you run code that is 5 years old in production. People only look at it when it breaks.

I just said that use latest exactly to avoid this issue.

You said to use it for development. But I don't use Artifactory so maybe you meant to use the latest for production also, if it passes the build.
Post reply on HN