Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Docker 29 has changed its default image store for new installs

docs.docker.com

Post reply on HN
Print view View original

11–20 of 93 posts

Re: Docker 29 has changed its default image store for new installs

#11

Post by freedomben »

ElevenLathe wrote: post #5
Oxodao wrote: post #4

Docker already fills up my dev machines yet they decided for this insane solution: > The containerd image store uses more disk space than the legacy storage drivers for the same images. This is because containerd stores images in both compressed and uncompressed formats, while the legacy drivers stored only the uncompressed layers. Why ?

Sounds like a straightforward time-space tradeoff: if you have the compressed layers sitting around when you need them, you can avoid the expense and time of compressing them.

did you mean the first "compressed" to be "uncompressed" ?

Re: Docker 29 has changed its default image store for new installs

#12

Post by sschueller »

Oxodao wrote: post #4

Docker already fills up my dev machines yet they decided for this insane solution: > The containerd image store uses more disk space than the legacy storage drivers for the same images. This is because containerd stores images in both compressed and uncompressed formats, while the legacy drivers stored only the uncompressed layers. Why ?

[flagged]

Re: Docker 29 has changed its default image store for new installs

#13

Post by colechristensen »

ElevenLathe wrote: post #5
Oxodao wrote: post #4

Docker already fills up my dev machines yet they decided for this insane solution: > The containerd image store uses more disk space than the legacy storage drivers for the same images. This is because containerd stores images in both compressed and uncompressed formats, while the legacy drivers stored only the uncompressed layers. Why ?

Sounds like a straightforward time-space tradeoff: if you have the compressed layers sitting around when you need them, you can avoid the expense and time of compressing them.

I'm not sure about the fastest macbook disk access, but even with NVMe storage I've found lz4 to be faster than the disk. That is (it's hard to say this exactly correct) compressed content gets read/written FASTER than uncompressed content because fewer bytes need to transit the disk interface and the CPU is able to compress/decompress significantly faster than data is able to go through whatever disk bus you've got.

Re: Docker 29 has changed its default image store for new installs

#14

Post by IsTom »

fabian2k wrote: post #9

> This difference is particularly noticeable with multiple images sharing the same base layers. With legacy storage drivers, shared base layers were stored once locally, and reused images that depended on them. With containerd, each image stores its own compressed version of shared layers, even though the uncompressed layers are still de-duplicated through snapshotters. This seems like a really weird decision. If bas…

Docker is already hogging a lot of disk space and needs to be pruned regularly. I can't imagine what's it's going to be like now.

Re: Docker 29 has changed its default image store for new installs

#15

Post by mschuster91 »

sschueller wrote: post #12
Oxodao wrote: post #4

Docker already fills up my dev machines yet they decided for this insane solution: > The containerd image store uses more disk space than the legacy storage drivers for the same images. This is because containerd stores images in both compressed and uncompressed formats, while the legacy drivers stored only the uncompressed layers. Why ?

[flagged]

> It is shameful for apple to hard solder their disks. There is no benefit to the user

Actually, it is. The speed and latency difference does matter, that is how even an 8GB RAM MacBook feels snappier than many a 32GB Windows machine - it can use the disk as swap.

Re: Docker 29 has changed its default image store for new installs

#16

Post by epistasis »

fabian2k wrote: post #9

> This difference is particularly noticeable with multiple images sharing the same base layers. With legacy storage drivers, shared base layers were stored once locally, and reused images that depended on them. With containerd, each image stores its own compressed version of shared layers, even though the uncompressed layers are still de-duplicated through snapshotters. This seems like a really weird decision. If bas…

This is hell for a lot of ML containers, that have gigabytes of CUDA and PyTorch. Before at least you could keep your code contained to a layer. But if I understand this correctly every code revision duplicates gigabytes of the same damn bloated crap.

Re: Docker 29 has changed its default image store for new installs

#17

Post by newsoftheday »

mschuster91 wrote: post #15
sschueller wrote: post #12

Earlier quoted context omitted.

[flagged]

> It is shameful for apple to hard solder their disks. There is no benefit to the user Actually, it is. The speed and latency difference does matter, that is how even an 8GB RAM MacBook feels snappier than many a 32GB Windows machine - it can use the disk as swap.

I had to work on a Mac M3 for a year, it sucked, it did not feel snappier than any Windows or Linux machine (including this one) that I've ever used and that is going back to the 1980's.

Re: Docker 29 has changed its default image store for new installs

#20

Post by NewJazz »

Filligree wrote: post #10
ElevenLathe wrote: post #5

Earlier quoted context omitted.

Sounds like a straightforward time-space tradeoff: if you have the compressed layers sitting around when you need them, you can avoid the expense and time of compressing them.

Why would I need the compressed layers?

Pushing
Post reply on HN