51–60 of 62 posts
alias dive='docker run -ti --rm -v /var/run/docker.sock:/var/run/docker.sock wagoodman/dive'
Earlier quoted context omitted.
Is there any performance benefit to having fewer layers? My understanding is that there's no gain by merging layers as the size of the image remains constant.
If you've got a 50 layer image then each time you open a file, I believe the kernel has to look for that file in all 50 layers before it can fail with ENOENT.
A dumb question: Why are most of the container/infrastructure tools written in GoLang? Examples that come to my mind include Docker, Podman, nerdctl, Terraform and Kubernetes. Is there any obvious advantage that GoLang offers, making it so popular for building these tools?
I would argue system tools and utilities, CLIs and networking software are where Go shines the most. Rust is probably the only good modern alternative that is mature.
A dumb question: Why are most of the container/infrastructure tools written in GoLang? Examples that come to my mind include Docker, Podman, nerdctl, Terraform and Kubernetes. Is there any obvious advantage that GoLang offers, making it so popular for building these tools?
Kubernetes specifically is in go because google invented go and also invented Kubernetes. Their internal teams have a lot of go engineers due to the whole inventing it thing
It was originally written in Java.
In fact, most Go users are outside Google, internally it is all about Java, Kotlin, Dart, C++ and Python, and now Rust as well.
Kubernetes, gVisor and Android GPU debugger are probably the only major internal projects in Go.
Earlier quoted context omitted.
Is there any performance benefit to having fewer layers? My understanding is that there's no gain by merging layers as the size of the image remains constant.
Eventually, once zstd support gets fully supported, and tiny gzip compression windows are not a limitation, then compressing a full layer would almost certainly have a better ratio over several smaller layers https://github.com/opencontainers/image-spec/issues/803
Earlier quoted context omitted.
If you've got a 50 layer image then each time you open a file, I believe the kernel has to look for that file in all 50 layers before it can fail with ENOENT.
It depends on your OCI engine; but this isn’t the case with containers. Each layer is successively “unpacked” upon a “snapshot”, from which containers are created.
I build this Dockerfile into a container with 'podman build -t test .':
FROM registry.access.redhat.com/ubi9/ubi-minimal
RUN touch /file1
RUN touch /file2
RUN touch /file3
Then I can inspect the container's storage configuration:
$ podman inspect -l | jq '.[].GraphDriver'
{
"Name": "overlay",
"Data": {
"LowerDir": "/home/sam/.local/share/containers/storage/overlay/e0055211bf4b6a839f91759d9bbbb5b93c61f4e2969b25ec9d732428557c230f/diff:/home/sam/.local/share/containers/storage/overlay/efd666ad5dc6b7a92933043ed42a41341b4e25c80e981f6d01ab41927d8f8a89/diff:/home/sam/.local/share/containers/storage/overlay/1c108fb13c6708a93dcf237bea2637344cf65d3c9272693817aa0741b158fd7b/diff:/home/sam/.local/share/containers/storage/overlay/80c0d7946d02825018d47fbf34df054bec5dc433ae20f71ffabd8d3725448837/diff",
"MergedDir": "/home/sam/.local/share/containers/storage/overlay/f14a1506ed692e3c40b06d5eb1ba9c011bad159c82df6ddee632b49ba02a5bfc/merged",
"UpperDir": "/home/sam/.local/share/containers/storage/overlay/f14a1506ed692e3c40b06d5eb1ba9c011bad159c82df6ddee632b49ba02a5bfc/diff",
"WorkDir": "/home/sam/.local/share/containers/storage/overlay/f14a1506ed692e3c40b06d5eb1ba9c011bad159c82df6ddee632b49ba02a5bfc/work"
}
}
$ podman inspect -l | jq '.[].GraphDriver.Data.LowerDir' -r | tr : '\n' | xargs -rt -n1 ls
ls /home/sam/.local/share/containers/storage/overlay/e0055211bf4b6a839f91759d9bbbb5b93c61f4e2969b25ec9d732428557c230f/diff
file3 run/
ls /home/sam/.local/share/containers/storage/overlay/efd666ad5dc6b7a92933043ed42a41341b4e25c80e981f6d01ab41927d8f8a89/diff
file2 run/
ls /home/sam/.local/share/containers/storage/overlay/1c108fb13c6708a93dcf237bea2637344cf65d3c9272693817aa0741b158fd7b/diff
file1 run/
ls /home/sam/.local/share/containers/storage/overlay/80c0d7946d02825018d47fbf34df054bec5dc433ae20f71ffabd8d3725448837/diff
afs/ bin@ boot/ dev/ etc/ home/ lib@ lib64@ lost+found/ media/ mnt/ opt/ proc/ root/ run/ sbin/ srv/ sys/ tmp/ usr/ var/
# podman inspect systemd-oxidized | jq '.[].GraphDriver.Data.LowerDir' -r | tr ':' '\n' | xargs -rt -n 1 ls -F
ls -F /var/lib/containers/storage/overlay/062693a4d8fe24055645ee1c11bbc542507ba3fe101ed4e5b76b6457752d3ac5/diff
home/
ls -F /var/lib/containers/storage/overlay/9d9c6a906aff40160f924c4fec96692a6fc59f27316b8df5ee2616091008a2fe/diff
home/
ls -F /var/lib/containers/storage/overlay/e35e4cf6ce1057979baee991e609f82afb85360560dc8bf36ab069d8e8288f5f/diff
home/
ls -F /var/lib/containers/storage/overlay/da5189fd0e740204d7a30e4b59b012251e8d36a1db488bf47ede61247a948fff/diff
home/
ls -F /var/lib/containers/storage/overlay/702d48c0e8afce7cf5ce049939d8fdb596755e709a3735bbbf4900ec65c45529/diff
etc/ home/ var/
ls -F /var/lib/containers/storage/overlay/877031909d6cf6038e0e0295c34f14932b71903835b3eb13ae6bf23396d65daf/diff
etc/ usr/ var/
ls -F /var/lib/containers/storage/overlay/b90fc2ebe1d7c19e5ee43033b8e8be95210b4e3a0012bb54db327f6d52e89d12/diff
tmp/
ls -F /var/lib/containers/storage/overlay/aa2e131948d26d04f48f7c01c57aa98e2a308e4fc0d707c9532d30fa4ab89f5d/diff
root/ usr/ var/
ls -F /var/lib/containers/storage/overlay/6af6a203c83f697aa35b5007b5678b8c2b052ab482d9d7f7e5b322cd74682441/diff
root/ tmp/ usr/ var/
ls -F /var/lib/containers/storage/overlay/b237f20a22db5d282838c5a7c9d78f7d6e648d9b223a328fa8b8078e9c6b5bbe/diff
ls -F /var/lib/containers/storage/overlay/7d1bc5d46561795a4f7058e4098faa345e119617292761933e2ad7f185dbd45b/diff
ls -F /var/lib/containers/storage/overlay/88d52756a706920fbcb38cbbb6eaf43ea09be0045361ffe6920185ca7179a1d9/diff
tmp/
ls -F /var/lib/containers/storage/overlay/0d0dedc3e4f8d7f39b9a0c78d1cf380ad1161023a49bd74ab8b51626f6d21f19/diff
root/ usr/ var/
ls -F /var/lib/containers/storage/overlay/d74968407a57d753c2ae232ea26e0e2d7fed30683a67a993b814b71684817eaa/diff
root/ usr/ var/
ls -F /var/lib/containers/storage/overlay/885b215dc2a8304a58a6d35a5668c74d3d80f27dacf56b97a3b8b13c1a423b33/diff
root/ usr/ var/
ls -F /var/lib/containers/storage/overlay/a4edd3819e3655d0ca90ac898a4a2b91ec50c83cbf61862186e10ae1b44cbcc0/diff
etc/ usr/ var/
ls -F /var/lib/containers/storage/overlay/954dcca00292635597e4c5ee9ebc35bf5fcc15a50dfd8f176df7b4c15e695e10/diff
bd_build/ etc/ root/ run/ usr/ var/
ls -F /var/lib/containers/storage/overlay/fb670aa62e0f4f71382ef61a742e8ab1ad5d345b95267b0adc18c10e6c87f13b/diff
bd_build/
ls -F /var/lib/containers/storage/overlay/7f5cbd8cc787c8d628630756bcc7240e6c96b876c2882e6fc980a8b60cdfa274/diff
bin@ boot/ dev/ etc/ home/ lib@ lib32@ lib64@ libx32@ media/ mnt/ opt/ proc/ root/ run/ sbin@ srv/ sys/ tmp/ usr/ var/
I'd be interested to see what this looks like with docker?
Earlier quoted context omitted.
If you've got a 50 layer image then each time you open a file, I believe the kernel has to look for that file in all 50 layers before it can fail with ENOENT.
That seems ripe for optimization, if true, especially since those layers are all immutable.
Earlier quoted context omitted.
It really does sound amazing. Would have needed this when you guys (hn) and reddit helped me figure out what a rogue Raspberry Pi was doing in our server closet https://blog.haschek.at/2019/the-curious-case-of-the-RasPi-i...
2019! Can you post an update?
Dive also inspired us to make it easier to surface what is actually in your build context, on every build. So we shipped that as a feature in Depot a few weeks back.
Earlier quoted context omitted.
I would argue system tools and utilities, CLIs and networking software are where Go shines the most. Rust is probably the only good modern alternative that is mature.
I have a question for big companies building software in Rust. How are they able to find the talent? Unlike most other common languages, it is magnitudes more difficult to ramp up entry level new hires in Rust for the simple reason that Rust requires more than normal understanding of Computer Science. Is it just the C++ devs that wanna transition or have already transitioned to Rust that those companies can recruit?
They have not found this to be true.