As others have noted here, this is why we use Nix
https://nixos.org/
Dive – A tool for exploring each layer in a Docker image
41–45 of 45 posts
Re: Dive – A tool for exploring each layer in a Docker image
#42Very, very useful. Thank you for sharing this!
Re: Dive – A tool for exploring each layer in a Docker image
#43Project idea: a Google for docker images. Search for files by hash within all public docker images. Find images that contain a certain piece of code. Reverse engineering Dockerfiles for those that were built without.
The latter can be easily done in a few hours by inspecting the image layer metadata. Every layer understands the command it was run in the Dockerfile to create itself. Just look at `docker history` and have a look at the "CREATED BY" field for human-readable output of the layer metadata, or depending on your graph driver have a look in /var/lib/docker/image/overlay2/imagedb/content/sha256. From there you can reverse-…
> Every layer understands the command it was run in the Dockerfile to create itself
How reliable is this? Can it be modified after creation by a malicious party?
That is, if I get a wild docker image, can I trust the results of `docker history`?
Re: Dive – A tool for exploring each layer in a Docker image
#44Earlier quoted context omitted.
Layers are delta transfers. Maybe not as smart as they could be, but there is a lot of under-appreciated complexity hiding here.
layers are not delta transfers in my opinion, they are just new copies or removed files. its like saying sftp is delta transfer because you don't have to send your entire disk image every time. I love layers and dockers very nice reusing of base layers and such but i feel like delta transfers would dramatically improve my workflow and upload times.
> i feel like delta transfers would dramatically improve my workflow and upload times
If you think delta transfers would help, you're saying you have a bunch of data that doesn't change between image builds, but you have to re-upload all that data every time.
This actually sounds like a perfect use-case for image layers. If you can rearrange your Dockerfile to put the steps where files change more rarely before steps that change frequently, docker will automatically cache the earlier steps and not reupload if that layer already exists.
Image layers are at their heart already an implementation of a cache + delta transfer system. (Compared to some other delta systems it trades storage efficiency for computational efficiency.) You can get small delta transfers, but you have to work with docker via your Dockerfile to make it happen.
For example, that could mean: moving steps up or down the Dockerfile so more frequently changed layers come later; splitting a step into two steps to exploit the previous; making a step deterministic so you don't have to force rebuild; use multi-stage builds to make all of these easier to implement.
Re: Dive – A tool for exploring each layer in a Docker image
#45Project idea: a Google for docker images. Search for files by hash within all public docker images. Find images that contain a certain piece of code. Reverse engineering Dockerfiles for those that were built without.
A GitHub for Docker images might be the better analogy.
Docker the company could, I imagine, add deep search and browsing as enhancements to their existing Docker Hub.