Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fully in-browser container builds

ochagavia.nl

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Fully in-browser container builds

#21

Post by wofo »

mystifyingpoi wrote: post #4

> and we sometimes resign ourselves to the limitations of docker build and friends What are the limitations of `docker build`?

I'd say the limitations are relative to your use case: `docker build` works well enough in most cases, but sometimes it's not the right tool for the job (e.g., you cannot use `docker build` inside an AWS Lambda, you can get way better performance from specialized tools in some cases [1], etc)

[1]: see my sibling comment at https://news.ycombinator.com/item?id=48301041

Re: Fully in-browser container builds

#22

Post by mathfailure »

ImJasonH wrote: post #13

I love seeing folks learn that container images are just fancy tar files and JSON and are therefore buildable by normal tools. Along my own journey of demystification I made a few toy container image registries over the years that generated and served container images from nothing but the URL itself: https://ko.kontain.me builds a go application on demand and serves it atop a minimal base image https://apko.kontain.m…

I don't know any scenario where I'd need any of that, but I love such things regardless.

About git.kontain.me - what if I'd like to setup my own copy of such a service and would like it to work with a non-public git repo, how to pass credentials? Since you support only cloning over https (btw, why not over ssh as well?) - one could use an URL of format user:pass@host:port/path/to/repo, would that work?

Re: Fully in-browser container builds

#23

Post by a_t48 »

wofo wrote: post #16
thewisenerd wrote: post #6

> we brought down image creation time to mere seconds, even for images that were multiple GiB in size this sounds interesting; for e.g., was wondering the other day if we could build images without actually pulling base images.. everytime we compile, we copy artifact(s) onto a multi-hundred MB base image which definitely doesn't need to be pulled everytime.

There are many ways to achieve this. If you are just copying artifacts, and not using RUN, then there is in principle no need to download the base image at all. Feel free to message me if you'd like to chat (email is in my profile).

Does COPY —-link allow this with BuildKit? In principle it should, in practice I’d guess it ends up pulling the base image.

Re: Fully in-browser container builds

#24

Post by ImJasonH »

mathfailure wrote: post #22
ImJasonH wrote: post #13

I love seeing folks learn that container images are just fancy tar files and JSON and are therefore buildable by normal tools. Along my own journey of demystification I made a few toy container image registries over the years that generated and served container images from nothing but the URL itself: https://ko.kontain.me builds a go application on demand and serves it atop a minimal base image https://apko.kontain.m…

I don't know any scenario where I'd need any of that, but I love such things regardless. About git.kontain.me - what if I'd like to setup my own copy of such a service and would like it to work with a non-public git repo, how to pass credentials? Since you support only cloning over https (btw, why not over ssh as well?) - one could use an URL of format user:pass@host:port/path/to/repo, would that work?

Unfortunately due to the limitations of a valid image ref you can't include creds like that.

You can fork the registry and include your own creds in the backend when it pulls the repo though.

Re: Fully in-browser container builds

#25

Post by wofo »

a_t48 wrote: post #23
wofo wrote: post #16

Earlier quoted context omitted.

There are many ways to achieve this. If you are just copying artifacts, and not using RUN, then there is in principle no need to download the base image at all. Feel free to message me if you'd like to chat (email is in my profile).

Does COPY —-link allow this with BuildKit? In principle it should, in practice I’d guess it ends up pulling the base image.

I can't comment on BuildKit, unfortunately, since I haven't used it. My experience comes from building bespoke systems (an image builder and a custom registry) fully from scratch, because we needed to have full control in order to achieve the performance we were aiming at.

Re: Fully in-browser container builds

#26

Post by a_t48 »

wofo wrote: post #25
a_t48 wrote: post #23

Earlier quoted context omitted.

Does COPY —-link allow this with BuildKit? In principle it should, in practice I’d guess it ends up pulling the base image.

I can't comment on BuildKit, unfortunately, since I haven't used it. My experience comes from building bespoke systems (an image builder and a custom registry) fully from scratch, because we needed to have full control in order to achieve the performance we were aiming at.

I haven't gone full custom (yet), just forks (except for the pull client), but totally understand.

Re: Fully in-browser container builds

#27

Post by wofo »

a_t48 wrote: post #26
wofo wrote: post #25

Earlier quoted context omitted.

I can't comment on BuildKit, unfortunately, since I haven't used it. My experience comes from building bespoke systems (an image builder and a custom registry) fully from scratch, because we needed to have full control in order to achieve the performance we were aiming at.

I haven't gone full custom (yet), just forks (except for the pull client), but totally understand.

Controlling both the builder and the registry is super nice btw, because they can work together. If the builder knows some of the layers already exist in the registry, it merely has to create and push the remaining ones (without downloading any of the other layers, not even those from the base image). That gives you near-instant builds once the biggest layers are cached in the registry!

Since builds usually happen in CI, and pulls happen elsewhere (e.g., a kubernetes node), in the end layers are only downloaded when the resulting container image is actually used.

Re: Fully in-browser container builds

#28

Post by a_t48 »

wofo wrote: post #27
a_t48 wrote: post #26

Earlier quoted context omitted.

I haven't gone full custom (yet), just forks (except for the pull client), but totally understand.

Controlling both the builder and the registry is super nice btw, because they can work together. If the builder knows some of the layers already exist in the registry, it merely has to create and push the remaining ones (without downloading any of the other layers, not even those from the base image). That gives you near-instant builds once the biggest layers are cached in the registry! Since builds usually happen in…

Base image independent layers is something I’ve pondered about, but doesn’t feel compatible with things like apt. This conversation is giving me more reason to go implement lazy/fuse base layers though. My exports are already pretty fast due not not using tar+deduplication with similar layers, but pulling the base can still take several minutes.

Re: Fully in-browser container builds

#29

Post by apatheticonion »

lucideer wrote: post #3
phendrenad2 wrote: post #2

I joked once that the future was dockerizing every single react component and running them in a wasm port of k8s... I hope that premonition isn't coming true!

I can't imagine this happening in reality (i.e. people wanting to do this for normal production applications), but after reading your comment I must admit I am very tempted to try it for purely esoteric reasons.

I wouldn't underestimate people in the React ecosystem not to adopt insanity en mass.
Post reply on HN