Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We reduced a container image from 800GB to 2GB

sealos.io

Post reply on HN
Print view View original

21–30 of 88 posts

Re: We reduced a container image from 800GB to 2GB

#21

Post by hug »

SurceBeats wrote: post #9

Fascinating deep dive into OverlayFS CoW behavior. The 11GB btmp file getting copied 271 times is a perfect storm scenario. Did they consider mounting /var/log outside the image layers? Seems like that would prevent any log file from causing this amplification. Also interested in image-manip... Does it handle metadata differently than docker export/import?

Is it fascinating?

Do people not know that each layer comes with its own downsides?

Do people just do 272 layers and think that it’s normal?

This seems like people discovering that water is wet and fire is hot.

Re: We reduced a container image from 800GB to 2GB

#22

Post by progbits »

trenchpilgrim wrote: post #7

In the comments: People who didn't read the article assuming they were literally building 800GB images (the example in the article is an 11GB image that was amplified by copying behaviors)

In fairness the article is LLM vomit and could be two paragraphs, can't blame people for not reading it.

Re: We reduced a container image from 800GB to 2GB

#23

Post by BoredPositron »

adastra22 wrote: post #13
BoredPositron wrote: post #5

If your image is 800GB you are doing something wrong in the first place.

You didn't read the article.

I did and the image had problems to begin with. If it's a bad image or a bad configuration of your visor or in the image doesn't matter. If your images can bloat to over 800GB you are doing the basics wrong. Hint: Using commit to create your images...

Re: We reduced a container image from 800GB to 2GB

#24

Post by ndsipa_pomu »

perlgeek wrote: post #20

The real lesson they should learn is to not rely on running images and then using "docker commit" to turn it into an image, but instead to use proper image building tools. If you absolutely have to do it that way, be very deliberate about what you actually need. Don't run an SSH daemon, don't run cron, don't an SMTP daemon, don't run the suite of daemons that run on a typical Linux server. Only run precisely what you…

Initially I didn't understand how they were getting the log files into the image. I had no idea that people abuse "docker commit" - do they know nothing about containers? If you want persistent logs, then have a separate volume for them so they can't pollute any image (plus they are readable when the image restarts etc).

When I saw the HN title, I thought this was going to be something subtle like deleting package files (e.g. apt) in a separate layer, so you end up with a layer containing the files and then a subsequent layer that hides them.

Re: We reduced a container image from 800GB to 2GB

#25

Post by bmitch3020 »

This whole article could have been much better written as: learn to build images with a Dockerfile/ Containerfile or similar tooling, and store logs in a volume rather than the image filesystem. Everyone that builds a process around `docker commit` is simply in a race against time before they learn this lesson.

Re: We reduced a container image from 800GB to 2GB

#26

Post by guywhocodes »

trenchpilgrim wrote: post #7

In the comments: People who didn't read the article assuming they were literally building 800GB images (the example in the article is an 11GB image that was amplified by copying behaviors)

The TLDR: > We tackled critical container image bloat on our Sealos platform, fixing a severe disk space exhaustion issue by shrinking an 800GB, 272-layer image to just 2.05GB.

They say they made a 800GB container image, so your issue is about singular vs plural?

Regardless, I don't really get why anyone would self report like this. Is next article going to be about how they don't encrypt passwords and when they accidentally dropped prod DB they could restore account from logs because it had the passwords in clear text?

Re: We reduced a container image from 800GB to 2GB

#29

Post by gobip »

perlgeek wrote: post #20

The real lesson they should learn is to not rely on running images and then using "docker commit" to turn it into an image, but instead to use proper image building tools. If you absolutely have to do it that way, be very deliberate about what you actually need. Don't run an SSH daemon, don't run cron, don't an SMTP daemon, don't run the suite of daemons that run on a typical Linux server. Only run precisely what you…

What if I need cron in my docker container? And ssh? And a text editor? And a monitoring agent? :P

Thankfully LXD is here to serve this need: very lightweight containers for systems, where your app runs in a complete ecosystem, but very light on the ram usage.

Re: We reduced a container image from 800GB to 2GB

#30

Post by curt15 »

gobip wrote: post #29
perlgeek wrote: post #20

The real lesson they should learn is to not rely on running images and then using "docker commit" to turn it into an image, but instead to use proper image building tools. If you absolutely have to do it that way, be very deliberate about what you actually need. Don't run an SSH daemon, don't run cron, don't an SMTP daemon, don't run the suite of daemons that run on a typical Linux server. Only run precisely what you…

What if I need cron in my docker container? And ssh? And a text editor? And a monitoring agent? :P Thankfully LXD is here to serve this need: very lightweight containers for systems, where your app runs in a complete ecosystem, but very light on the ram usage.

>What if I need cron in my docker container? And ssh? And a text editor? And a monitoring agent? :P

How are you going to orchestrate all those daemons without systemd? :P

As you mentioned, a container running systemd and a suite of background services is the typical use case of LXD, not docker. But the difference seems to be cultural -- there's nothing preventing one from using systemd as the entry point of a docker container.

Post reply on HN