Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We reduced a container image from 800GB to 2GB

sealos.io

Post reply on HN
Print view View original

81–88 of 88 posts

Re: We reduced a container image from 800GB to 2GB

#81

Post by BobbyTables2 »

sally_glance wrote: post #55
curt15 wrote: post #30

Earlier quoted context omitted.

>What if I need cron in my docker container? And ssh? And a text editor? And a monitoring agent? :P How are you going to orchestrate all those daemons without systemd? :P As you mentioned, a container running systemd and a suite of background services is the typical use case of LXD, not docker. But the difference seems to be cultural -- there's nothing preventing one from using systemd as the entry point of a docker…

fwiw I recently bootstrapped a small Debian image for myself, originally intended to sandbox coding agents I was evaluating. Shortly after I got annoyed by baseline vim and added my tmux & nvim dotfiles, now I find myself working inside the container regularly. It definitely works and is actually not the worst experience if your workflow is cli-focused.

Even putting GUI apps in a container isnt too bad once one develops the right incantation for x11/wayland forwarding.

Re: We reduced a container image from 800GB to 2GB

#82

Post by mejari »

trenchpilgrim wrote: post #7

In the comments: People who didn't read the article assuming they were literally building 800GB images (the example in the article is an 11GB image that was amplified by copying behaviors)

>the example in the article is an 11GB image that was amplified by copying behaviors

It's not, it's an 800GB image caused by multiple full writes of an 11GB file into the image's layers. I read the article.

Re: We reduced a container image from 800GB to 2GB

#83

Post by Atreiden »

move-on-by wrote: post #69
stackskipton wrote: post #59

Earlier quoted context omitted.

This sounds like a case of "We are in growth mode and will accept any garbage the customer will throw at us" without calculating the tech debt costs. As someone who is currently there, it's very frustrating place.

Oh just wait till it’s time for your company to stop the ‘growth mode’ shenanigans and get serious about acceptable levels of tech debt and feature bloat. It’s where we are. You can’t just flip a switch. There is no “Hey, that was fun, but it’s time to start designing these things with a purpose and vision”. Beyond the totally unreasonable expectations that have been set by Product and C-level- you still have the mou…

This is the job of tech leadership IMO. People respond to incentive changes. If these items are properly prioritized on the roadmap, and credit and recognition follows tech debt remediation efforts to a similar degree as feature delivery, the work can be done.

But this requires strong tech leadership who can interface well with the C Suite and get buy-in for delaying in feature delivery. In the absence of this buy-in, you pretty much need to control the narrative and create a rogue skunkworks initiative to wrap these improvements _into_ the feature delivery.

Many companies don't have strong tech leadership though, and will perpetually churn VPs and Directors, forever chasing A Change without addressing the culture and incentive system that created that culture.

Re: We reduced a container image from 800GB to 2GB

#84

Post by never_inline »

hiddew wrote: post #17
trenchpilgrim wrote: post #8

Earlier quoted context omitted.

If you are adding security keys and git repos to your final shipped image you are doing things very wrong - a container image is literally a tarball and some metadata about how to run the executables inside. Even if you need that data to build your application you should use a multi-stage build to include only the final artifacts in the image you ship. For stuff like security keys you should typically add them as bui…

> For stuff like security keys you should typically add them as build args, not as content in the image. Do not use build arguments for anything secret. The values are committed into the image layers.

Yep. The only valid usecase I think of is using the secret for something else, eg connecting to an internal package registry, in which case the secret mounts may help.

Re: We reduced a container image from 800GB to 2GB

#85

Post by pjmlp »

gobip wrote: post #29
perlgeek wrote: post #20

The real lesson they should learn is to not rely on running images and then using "docker commit" to turn it into an image, but instead to use proper image building tools. If you absolutely have to do it that way, be very deliberate about what you actually need. Don't run an SSH daemon, don't run cron, don't an SMTP daemon, don't run the suite of daemons that run on a typical Linux server. Only run precisely what you…

What if I need cron in my docker container? And ssh? And a text editor? And a monitoring agent? :P Thankfully LXD is here to serve this need: very lightweight containers for systems, where your app runs in a complete ecosystem, but very light on the ram usage.

The answer is naturally kubernetes, alongside rootless and noshell containers.

Re: We reduced a container image from 800GB to 2GB

#86

Post by pjmlp »

mystifyingpoi wrote: post #74
gkfasdfasdf wrote: post #58

> image-manip squash: This is the key to reclaiming disk space and the core of our strategy to squash the image layers. The tool creates a temporary container, applies all 272 layers in sequence to an empty root filesystem, and then exports the final, merged filesystem as a single new layer. This flattens the image's bloated history into a lean, optimized final state. Wouldn't a multistage Dockerfile have accomplishe…

I think yep, pretty much. Maybe they didn't know this existed?

Given that I was using multi-layers in 2020, when I finally got involved in projects with Docker, five years is already some time to learn about this stuff, and I bet is is much older, not bothering to look it up.

Re: We reduced a container image from 800GB to 2GB

#87

Post by hsbauauvhabzb »

chuckadams wrote: post #51
pandemic_region wrote: post #10

Earlier quoted context omitted.

Right? Blog posts like these makes me question competence instead of attributing it.

Competence comes from experience built on lots of screw-ups. I like when people blog their mistakes.

But if your computer doesn’t work and you inspect it and turns out you forgot to turn it on, does that really qualify for a blog post?

Log rotation and disk consuming logs are a tale as old as time

Re: We reduced a container image from 800GB to 2GB

#88

Post by chuckadams »

hsbauauvhabzb wrote: post #87
chuckadams wrote: post #51

Earlier quoted context omitted.

Competence comes from experience built on lots of screw-ups. I like when people blog their mistakes.

But if your computer doesn’t work and you inspect it and turns out you forgot to turn it on, does that really qualify for a blog post? Log rotation and disk consuming logs are a tale as old as time

If you had an automated process that was resulting in computers somehow not being turned on when they should, perhaps.

I guess some people just need to shit on everything.

Post reply on HN