Not to be confused with https://dagger.dev/
Also not to be confused with https://dagster.io/
Dagger: a new way to build CI/CD pipelines
101–110 of 267 posts
Re: Dagger: a new way to build CI/CD pipelines
#102Earlier quoted context omitted.
> Seems to assume that all CI/CD workflows work in a single container at a time pattern. Dagger runs your workflows as a DAG, where each node is an action running in its own container. The dependency graph is detected automatically, and all containers that can be parallelized (based on their dependencies) will be parallelized. If you specify 10 actions to run, and they don't depend on each other, they will all run in…
Thank you for the detailed response. I appreciate you taking the time. One last question/note. > Note: you won't need to configure caching though, because Dagger automatically caches all actions out of the box :) Is this strictly because it's using Docker underneath and layers can be reused? If so, unless those intermediary layers are somehow pushed/pulled by the dagger github action (or any associated CI/CD tool equ…
> > Note: you won't need to configure caching though, because Dagger automatically caches all actions out of the box :)
> Is this strictly because it's using Docker underneath and layers can be reused?
Not exactly: we use Buildkit under the hood, not Docker. When you run a Dagger action, it is compiled to a DAG, and run by buildkit. Each node in the DAG has content-addressed inputs. If the same node has been executed with the same inputs, buildkit will cache it. This is the same mechanism that powers caching in "docker build", but generalized to any operation.
The buildkit cache does need to be persisted between runs for this to work. It supports a variety of storage backends, including posix filesystem, a docker registry, or even proprietary key-value services like the Github storage API. If buildkit supports it, Dagger supports it.
Don't let the "docker registry" option confuse you: buildkit cache data isn't the same as docker images, so it doesn't carry the same garbage collection and tag pruning problems.
Re: Dagger: a new way to build CI/CD pipelines
#103After reading this entire post, I’m still left wondering what problem this solves for me, beyond fluffy promises of ‘everything is going to be better’. At the very least I’d want to see a comparison with what we have now, to show me how this is better. I get that I can try to explore more, but if I don’t get a compelling reason to do so after reading the introductory post, I’m not very motivated to do so Any motivati…
It mostly solves this problem: - write code - run tests - commit code - update CI - commit - CI broken - update CI - commit - CI broken - update CI - ... The workarounds for this are generally awful. For Jenkins, you stage your own instance locally and configure your webhooks to use that. It's exactly as terrible as it sounds, and I never recommend this approach. For Travis and Concourse (I think), you can use their…
Jenkins lets you replay a Pipeline, with changes, which is massively useful — removing the need to change things locally and commit.
Re: Dagger: a new way to build CI/CD pipelines
#104I'm getting "AmongUs" vibes from this site. Its called 'Dagger', and the illustrations are an anonymously clad astronaut running around a space station.
Re: Dagger: a new way to build CI/CD pipelines
#105There's so much more to CI/CD than the build definitions (e.g. dashboarding, log viewing/retention, access control, manual interventions, secret management, test reporting, etc.) and while some of your points resonate very strongly with me (e.g. local builds), I can't help but wonder what the endgame is here? You've raised $27m to what? Give away a free tool for engineers to define their builds in? While they continu…
Good question. No, we don't intend to replace the CI service. We think of CI as infrastructure, and there are plenty of great infrastructure providers out there. And you are absolutely correct that for Dagger to succeed, it has to remain agnostic to infrastructure providers, which means we cannot build an infrastructure provider business ourselves. If the experience of hosting Dagger becomes so bad that it affects th…
> Each time Dagger runs your pipeline, it can produce telemetry about every aspect of your supply chain.
To me, that still doesn’t seem to go to the core of what value it brings to table. I understand that Dagger can do all this, and that businesses would like to know what runs where and how everything interacts, but… it doesn’t explain to me the CI / build pipeline angle?
How does knowing the telemetry around my build pipelines translate to better software, or cost savings, or other improvements?
If there’s a registry to exchange “components” of a build pipeline, what does that bring me what a regular python / java / etc package can’t?
Don’t get me wrong, I think there are plenty of problems with CI (I have a ~200 job CI pipeline on fire in front of me), I just can’t seem to connect the dots here. :)
Re: Dagger: a new way to build CI/CD pipelines
#106I'm happy to see dagger,
because it is something that I have been working with since 2016,
and ended up creating
https://github.com/ysoftwareab/yplatform
It uses well-tested software (GNU Make, Bash, Homebrew, Docker) and there's not much to learn really.
* Incubated at Tobii (eyetracking, Sweden),
* it has been used for several projects of different sizes,
* tested on 12+ CIs !
* tested on macOS, Linux and Windows (WSL) !
* tested on 8+ Linux distros !
* comes with prebuilt Docker images !
Happy to help anyone integrate it.
Re: Dagger: a new way to build CI/CD pipelines
#107Come on guys...there is an extremely well known and widely used Java library called Dagger.
Call yours Cinquedea, Jambiya, Anelace, or Rondel if you really want a knifey name.
Re: Dagger: a new way to build CI/CD pipelines
#108Earlier quoted context omitted.
Thank you. It isn't hard to find, but my point was that if you say "using an intuitive declarative language" then a developer will get excited by a code snippet that shows that intuitive developer language. It wasn't there, and I think their post could be improved by having less fluffy language and more code, if they are targeting me, that is. This link does show the code, but it took a few clicks to get there from y…
The language is CUE, which I think will see mass adoption in config / DevOps in the coming years. So regardless what you think of the language today, it is likely to become important and part of your like in the not too distant future. https://cuelang.org | https://cuetorials.com Dagger builds on top of CUE and the (DAG) flow engine therein
Oh wow, the validating yaml example on the front page looks super useful. Might try and play with this over the next few weeks!
Re: Dagger: a new way to build CI/CD pipelines
#109Paul Graham just highlighted it on Twitter: https://twitter.com/paulg/status/1509209579414040588
Re: Dagger: a new way to build CI/CD pipelines
#110I’m highly jealous of teams that can containerize all their build tools. If you deal with proprietary toolchains that are tens of gigabytes (Windows WDK, Xilinx Vivado/Vitis) it’s just untenable, and that’s before even mentioning licensing. Even Azure doesn’t have a great solution for WDK development. It’s hard to feel like we’re not being left behind. Bind mounting the tools into the container is an option but at th…
You can add Apple toolchains to that list also