This is great! Docker feels imperative to me, and using this could help mitigate that. Also looking into Nix to help my container workflow more declarative.
But a Dockerfile is imperative. It just lists, in order, the steps needed to build an image.
Dive – A tool for exploring each layer in a Docker image
11–20 of 45 posts
Re: Dive – A tool for exploring each layer in a Docker image
#12Earlier quoted context omitted.
just curious, how do you intend to use this? I mean, what problem does exploring each layer in a Docker solve for you?
"why does this small change increase the size of my image by 500mb"
A few people pointed me to this dive tool after I wrote up these notes on shrinking Docker images: https://simonwillison.net/2018/Nov/19/smaller-python-docker-...
Re: Dive – A tool for exploring each layer in a Docker image
#13Is there a smart way to make more layers than splitting everything up into modules and ADDing each one? Why doesnt docker have delta transfers yet?
Layers are delta transfers. Maybe not as smart as they could be, but there is a lot of under-appreciated complexity hiding here.
Re: Dive – A tool for exploring each layer in a Docker image
#14This is amazing, thanks for sharing it! A very useful feature that I always missed but never actually tough about. I've had a hard time understanding the slices on the beginning will be very useful for troubleshooting bad images also while optimizing image sizes. Can probably help to understand problems like the one bug where the image size got almost a 2x increase after a CHOWN, because the slice is added again with…
Newer docker versions have a --chown flag for COPY/ADD exactly to address that.
Re: Dive – A tool for exploring each layer in a Docker image
#15This is great! Docker feels imperative to me, and using this could help mitigate that. Also looking into Nix to help my container workflow more declarative.
But a Dockerfile is imperative. It just lists, in order, the steps needed to build an image.
It's declarative in the sense that the instructions are set in text. But what goes inside a Dockerfile might be imperative, like running apt-get to install packages (i.e., running steps in order).
Re: Dive – A tool for exploring each layer in a Docker image
#16Earlier quoted context omitted.
But a Dockerfile is imperative. It just lists, in order, the steps needed to build an image.
It's declarative in the sense that the instructions are set in text. But what goes inside a Dockerfile might be imperative, like running apt-get to install packages (i.e., running steps in order).
But what goes inside a Dockerfile might be imperative, like running apt-get to install packages (i.e., running steps in order)
But...that too is done via the Dockerfile, via the RUN command to execute commands inside the container at run time. Or alternatively using the ENTRYPOINT command.
Re: Dive – A tool for exploring each layer in a Docker image
#17Project idea: a Google for docker images.
Search for files by hash within all public docker images.
Find images that contain a certain piece of code.
Reverse engineering Dockerfiles for those that were built without.
Re: Dive – A tool for exploring each layer in a Docker image
#18Is there a smart way to make more layers than splitting everything up into modules and ADDing each one? Why doesnt docker have delta transfers yet?
Layers are delta transfers. Maybe not as smart as they could be, but there is a lot of under-appreciated complexity hiding here.
layers are not delta transfers in my opinion, they are just new copies or removed files. its like saying sftp is delta transfer because you don't have to send your entire disk image every time. I love layers and dockers very nice reusing of base layers and such but i feel like delta transfers would dramatically improve my workflow and upload times.
Re: Dive – A tool for exploring each layer in a Docker image
#19Earlier quoted context omitted.
Layers are delta transfers. Maybe not as smart as they could be, but there is a lot of under-appreciated complexity hiding here.
layers are not delta transfers in my opinion, they are just new copies or removed files. its like saying sftp is delta transfer because you don't have to send your entire disk image every time. I love layers and dockers very nice reusing of base layers and such but i feel like delta transfers would dramatically improve my workflow and upload times.
My understanding is the on-disk storage might not be delta, as tar files, but the transfer on the wire is... could be wrong, but I think they intentionally made delta-transfer a docker registry network concern? I suppose if you wanted delta storage on-disk that again could be an implementation concern, knowing where to pull and reassemble the bits into a standard image? Oh and it uses gzip compression on the wire and possibly elsewhere, again a form of delta compression.
Re: Dive – A tool for exploring each layer in a Docker image
#20Earlier quoted context omitted.
Layers are delta transfers. Maybe not as smart as they could be, but there is a lot of under-appreciated complexity hiding here.
layers are not delta transfers in my opinion, they are just new copies or removed files. its like saying sftp is delta transfer because you don't have to send your entire disk image every time. I love layers and dockers very nice reusing of base layers and such but i feel like delta transfers would dramatically improve my workflow and upload times.
> its like saying sftp is delta transfer because you don't have to send your entire disk image every time
I see your point, but the comparison to sftp seems really off. Overlay2 [1] indeed works on whole file differences, just as sftp would do.
Additionally, I don't think making actual byte-level deltas would yield you much improvement on container size. It would also increase access time unless you keep a cache that doubles storage requirements per image. Dockerfiles primarily add files rather than change them, and actual changed files are often small.
[1] https://docs.docker.com/storage/storagedriver/overlayfs-driv...