We reduced a container image from 800GB to 2GB
1–10 of 88 posts
Re: We reduced a container image from 800GB to 2GB
#2Our platform is designed to solve a very specific workflow, and the DevBox is only the first step in that process.
Our users need to connect their local VS Code, Cursor, or JetBrains IDEs to the cloud environment. The industry-standard extensions for this only speak the SSH protocol. So, to give our users the tools they love, the container must run an SSHD to act as the host.
We aren't just a CDE like Coder or Codespaces. We're trying to provide a fully integrated, end-to-end application lifecycle in one place.
The idea is that a developer on Sealos can:
1. Spin up their DevBox instantly. 2. Code and test their feature in that environment (using their local IDE). 3. Then, from that same platform, package their application into a production-ready, versioned image. 4. And finally, deploy that image directly to a production Kubernetes environment with one click.
That "release" feature was how we let a developer "snapshot" their entire working environment into a deployable image without ever having to write a Dockerfile.
Re: We reduced a container image from 800GB to 2GB
#3Our platform is designed to solve a very specific workflow, and the DevBox is only the first step in that process. Our users need to connect their local VS Code, Cursor, or JetBrains IDEs to the cloud environment. The industry-standard extensions for this only speak the SSH protocol. So, to give our users the tools they love, the container must run an SSHD to act as the host. We aren't just a CDE like Coder or Codesp…
[deleted]
Re: We reduced a container image from 800GB to 2GB
#4This seems very much like a ‘we mis configured our containers; then we realised, then we fixed it, then we blogged about it’ post of very little value.
Re: We reduced a container image from 800GB to 2GB
#5If your image is 800GB you are doing something wrong in the first place.
Re: We reduced a container image from 800GB to 2GB
#6Is it spooky that they said they looked inside a customer's image to fix this? A bunch of engineers just had access to their customer's intellectual property, security keys, git repos, ...
Re: We reduced a container image from 800GB to 2GB
#7In the comments: People who didn't read the article assuming they were literally building 800GB images (the example in the article is an 11GB image that was amplified by copying behaviors)
Re: We reduced a container image from 800GB to 2GB
#8Is it spooky that they said they looked inside a customer's image to fix this? A bunch of engineers just had access to their customer's intellectual property, security keys, git repos, ...
If you are adding security keys and git repos to your final shipped image you are doing things very wrong - a container image is literally a tarball and some metadata about how to run the executables inside. Even if you need that data to build your application you should use a multi-stage build to include only the final artifacts in the image you ship.
For stuff like security keys you should typically add them as build --args-- secrets, not as content in the image.
Re: We reduced a container image from 800GB to 2GB
#9Fascinating deep dive into OverlayFS CoW behavior. The 11GB btmp file getting copied 271 times is a perfect storm scenario. Did they consider mounting /var/log outside the image layers? Seems like that would prevent any log file from causing this amplification. Also interested in image-manip... Does it handle metadata differently than
docker export/import?
Re: We reduced a container image from 800GB to 2GB
#10This seems very much like a ‘we mis configured our containers; then we realised, then we fixed it, then we blogged about it’ post of very little value.
Right? Blog posts like these makes me question competence instead of attributing it.