Example of integration of Dagger in Kraken CI: https://lab.kraken.ci/runs/4393/jobs
Dagger: a new way to build CI/CD pipelines
261–267 of 267 posts
Re: Dagger: a new way to build CI/CD pipelines
#262Earlier quoted context omitted.
> This is similar to installing something under /usr/sbin/ As someone who's trying to get to grips with the Linux filesystem conventions, would you mind elaborating on a) why that's wrong, and b) what you would suggest instead? This reference[0] suggests that `/usr/sbin` is for "general system-wide binaries with superuser (root) privileges required" (and `/usr/bin` for those that don't require root privileges). I've…
Generally, `/usr` is for stuff packaged by your distribution, while `/usr/local` (so, `/usr/local/bin` and so on) is for your own custom stuff. Both the `/usr` and `/usr/local` equivalents will be on your $PATH by default in most distros. For stuff that isn't just a self-contained executable, consider installing it to the folder `/opt/$MY_APP` and either symlinking the main binary into `/usr/local/bin` or putting a w…
Thanks! This helps a lot!
Re: Dagger: a new way to build CI/CD pipelines
#263Earlier quoted context omitted.
> This is similar to installing something under /usr/sbin/ As someone who's trying to get to grips with the Linux filesystem conventions, would you mind elaborating on a) why that's wrong, and b) what you would suggest instead? This reference[0] suggests that `/usr/sbin` is for "general system-wide binaries with superuser (root) privileges required" (and `/usr/bin` for those that don't require root privileges). I've…
/usr/sbin is a legacy artifact that shouldn't be used. /usr/sbin is usually just symlinked to /usr/bin
This was not a helpful reply because it only told me what not to do, without providing a better alternative. The sibling comment is much more helpful.
Re: Dagger: a new way to build CI/CD pipelines
#264Earlier quoted context omitted.
I use scoop to install this sort of tool in Windows. iwr -useb get.scoop.sh | iex # install scoop scoop install coreutils vim nano [...] # yay
Nice! I'll try it out tomorrow finally.. after giving up on choco, and possibly on winget. But in my case I wanted to leave something small (and busybox.exe is that small) /portable - for others to use (without the requirement to install scoop).
So scoop is awesome so far - the only thing I'm missing (right now) is to be able to specify specific bucket for some actions (but found workarounds).
Re: Dagger: a new way to build CI/CD pipelines
#265Hi everyone, I'm one of the co-founders of Dagger (and before that founder of Docker). This is a big day for us, it's our first time sharing something new since, you know... Docker. If you have any questions, I'll be happy to answer them here!
The carbon footprint of the cloud has exceeded the footprint of air travel. A move away from monolithic statically compiled binaries to constellations of microservices (usually bloated docker containers) is a significant part of the problem. Docker's explosive growth is partly due to the convenience of the abstraction it provides, abstracting the entire linux userspace, putting even OS-wide package managers and langu…
“A move away from monolithic statically compiled binaries to constellations of microservices (usually bloated docker containers) is a significant part of the problem.”
Untrue, in every way.
Why did you say this?
Re: Dagger: a new way to build CI/CD pipelines
#266Earlier quoted context omitted.
> Asking a creator to somehow factor in environmental impact isn't going to work. That attitude doesn't leave a great impression for me. I take your points about how difficult it is, but I think we can all do better than throw our hands up in the air. For instance, you could talk about how easy containerization of CI/CD makes it easier to move your pipeline where impact is lowest. Or that you can control your own imp…
> For instance, you could talk about how easy containerization of CI/CD makes it easier to move your pipeline where impact is lowest. Or that you can control your own impact rather than leave it up to the whim of someone like CircleCI. Oh yeah -- I'm not at all saying Docker is all bad. * Like you mentioned, increasing interoperability allows the market be more efficient. * Docker continued the path that VMs started…
It was meant as genuine constructive criticism and a plea to do better. They have more traction than most people to morph docker into something that comes close to its predecessors in efficiency. Somehow replace their fragile layer caching strategy based around diffs of the entire filesystem, with something that understands the OS and language-specific packaging systems being used inside the containers, and can therefore at ~least cache the package downloads. We desperately need a better, more efficient package manager, and Docker has been a huge setback; it has become normal for CICD to rebuild your image every time you touch the code, and pull down practically an entire linux distribution every time. I know you can do better with docker caching if you put in enough consistent and dedicated engineering effort into how you structure things, and set up local apt and pypi mirrors, etc... but that's the default behavior, and none of (admittedly very small number organizations) I've worked at have had the organizational capacity to really get past it. I don't know if we need what they're building, but we absolutely need a much more efficient new version of Docker with an easy migration path. Indeed they may be the ~only people with traction to resolve this, since in the current climate as soon as someone introduces a new package manager with more efficient dependency resolution or caching (i.e. nix, poetry, cargo...), people just stick it in their Docker container and break the caching anyway.
Re: Dagger: a new way to build CI/CD pipelines
#267Earlier quoted context omitted.
The carbon footprint of the cloud has exceeded the footprint of air travel. A move away from monolithic statically compiled binaries to constellations of microservices (usually bloated docker containers) is a significant part of the problem. Docker's explosive growth is partly due to the convenience of the abstraction it provides, abstracting the entire linux userspace, putting even OS-wide package managers and langu…
“A move away from monolithic statically compiled binaries to constellations of microservices (usually bloated docker containers) is a significant part of the problem.” Untrue, in every way. Why did you say this?
It won't be true in every shop, but I do this professionally and it's been my firsthand experience. A native statically compiled binary containing just the functions that actually get called will usually be... 10-100 MB. Ungroomed Docker images are ~10GB-20GB, same as you'd have on the root partition if you sat down and brought up a linux workstation or server node manually, and this is not a coincidence. Sure, docker avoids duplicating the linux kernel, making it more efficient than an old school VM, but these days all the ~other software bloat dominates the kernel in size. Most companies do not have a 100 person team of engineers dedicated to optimizing their image build and management workflow, and pruning what goes into their containers.