Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dagger: a new way to build CI/CD pipelines

dagger.io

Post reply on HN
Print view View original

61–70 of 267 posts

Re: Dagger: a new way to build CI/CD pipelines

#61

Post by shykes »

time0ut wrote: post #56

Congratulations! This looks amazing. I think I am the target audience and I cannot wait to try this. One very important thing for my use case is being able to run steps in parallel distributed across multiple agents. Is it capable of this?

Yes :) We rely entirely on buildkit (https://github.com/moby/buildkit) to run the pipelines (`dagger` itself acts as a "compiler" and buildkit client). So anything you can do with buildkit, works out of the box with Dagger. That includes clustering.

There is a lot of ongoing work in that area in the buildkit ecosystem. For example Netflix has a distributed buildkit farm, which I believe relies on containerd storage backends to share cache data. There are also integrations to run buildkit nodes on a kubernetes cluster.

Dagger itself is obviously quite new, but buildkit itself is very mature, thanks to its integration with 'docker build'.

Re: Dagger: a new way to build CI/CD pipelines

#63

Post by shykes »

seeekr wrote: post #58

This seems similar to what https://earthly.dev is doing?

Yes, I believe that is a fair comparison. Earhly is more focused on builds, whereas Dagger has a wider scope: build, test, deployment, any part of a CI/CD pipeline really. But the overall philosophy is the same: run everything in containers. The choice of buildkit as a runtime is also a key similarity. One big difference is that we use CUE as a configuration language, and Earthly uses YAML.

We have a lot of respect (and common friends!) with the Earthly developers, I am confident we can help each other build even better tools, and grow the buildkit ecosystem in the process.

Re: Dagger: a new way to build CI/CD pipelines

#64

Post by seeekr »

shykes wrote: post #63
seeekr wrote: post #58

This seems similar to what https://earthly.dev is doing?

Yes, I believe that is a fair comparison. Earhly is more focused on builds, whereas Dagger has a wider scope: build, test, deployment, any part of a CI/CD pipeline really. But the overall philosophy is the same: run everything in containers. The choice of buildkit as a runtime is also a key similarity. One big difference is that we use CUE as a configuration language, and Earthly uses YAML. We have a lot of respect (…

Thanks for the info, that is wonderful to hear! Best of luck to both you and Earthly, much needed work you're doing!

Re: Dagger: a new way to build CI/CD pipelines

#65

Post by shredder25 »

shykes wrote: post #4

Hi everyone, I'm one of the co-founders of Dagger (and before that founder of Docker). This is a big day for us, it's our first time sharing something new since, you know... Docker. If you have any questions, I'll be happy to answer them here!

Congratulations! I know exactly how this tool will benefit us DevOps Engineers as I knew when you did a demo of Docker at PyCon 2013, wishing you and your team the best!

Re: Dagger: a new way to build CI/CD pipelines

#66

Post by verdverm »

xrd wrote: post #34
dijksterhuis wrote: post #5

Earlier quoted context omitted.

> But, I wish there was some code to show me what makes it so radically different. Sorry to be that guy but maybe try the docs page...? https://docs.dagger.io/ https://docs.dagger.io/1205/container-images

Thank you. It isn't hard to find, but my point was that if you say "using an intuitive declarative language" then a developer will get excited by a code snippet that shows that intuitive developer language. It wasn't there, and I think their post could be improved by having less fluffy language and more code, if they are targeting me, that is. This link does show the code, but it took a few clicks to get there from y…

The language is CUE, which I think will see mass adoption in config / DevOps in the coming years. So regardless what you think of the language today, it is likely to become important and part of your like in the not too distant future.

https://cuelang.org | https://cuetorials.com

Dagger builds on top of CUE and the (DAG) flow engine therein

Re: Dagger: a new way to build CI/CD pipelines

#67

Post by mhitza »

shykes wrote: post #4

Hi everyone, I'm one of the co-founders of Dagger (and before that founder of Docker). This is a big day for us, it's our first time sharing something new since, you know... Docker. If you have any questions, I'll be happy to answer them here!

Hi! I've browsed the docs quickly, and I have a few questions.

Seems to assume that all CI/CD workflows work in a single container at a time pattern. How about testing when I need to spin up an associated database container for my e2e tests. Is it possible, and just omitted from the documentation?

Not familiar with cue, but can I import/define a common action that is used across multiple jobs? For example on GitHub I get to duplicate the dependency installation/caching/build across various jobs. (yes, I'm aware that now you can makeshift on GitHub a composite action to reuse)

Can you do conditional execution of actions based on passed in input value/env variable?

Any public roadmap of upcoming features?

Re: Dagger: a new way to build CI/CD pipelines

#68

Post by dopidopHN »

Aeolun wrote: post #16

After reading this entire post, I’m still left wondering what problem this solves for me, beyond fluffy promises of ‘everything is going to be better’. At the very least I’d want to see a comparison with what we have now, to show me how this is better. I get that I can try to explore more, but if I don’t get a compelling reason to do so after reading the introductory post, I’m not very motivated to do so Any motivati…

My main takeaway was the ability to debug a build or deploy pipeline locally.

That would come handy once or twice per month

Re: Dagger: a new way to build CI/CD pipelines

#69

Post by mhitza »

cebert wrote: post #32

Our team has had great success with GitHub Actions and Environments for CI/CD. One nice thing going that route is the build related code is contained within repositories. A large number of developers are already familiar wit GitHub, which makes onboarding new team members easier. I don’t see anything too compelling with dagger.io that is missing with GitHub. You can even use ACT to test workflow changes to builds loc…

How well does act work nowadays in practice? I was automating multiple PHP, Ansible, and Nodejs related projects last year and act failed (can't remember the exact errors now) for each project at some step.
Post reply on HN