Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dagger: a new way to build CI/CD pipelines

dagger.io

Post reply on HN
Print view View original

161–170 of 267 posts

Re: Dagger: a new way to build CI/CD pipelines

#161

Post by judge2020 »

jeremy_k wrote: post #156
jaaron wrote: post #131

Earlier quoted context omitted.

100% this. I feel like there's a ton of innovation in the cloud/container CI/CD space, but next to nothing elsewhere. In fact, some of the innovations in CI/CD make things _more_ difficult for those of us developing in other environments (such as game development). There's a lot of low hanging fruit for improving things elsewhere.

Do you work in game development? I would love to hear a little bit more about your experience. I play and follow Apex Legends and I'm always so curious about how bugs and regressions seem to make it into every one of their patches. As well as tons of new information that gets data-mined. To me, its like they don't have branches (tons of new code not accessible in the game is released in a patch) and they don't have u…

> As well as tons of new information that gets data-mined.

Client side encryption is basically impossible; the decryption key has to be in the executable you ship somewhere, or at least sent to the client after a connection with the server. Perhaps some encryption tiering system could work to keep unreleased locked content locked for longer but I don’t think anyone has gone through that trouble yet.

Re: Dagger: a new way to build CI/CD pipelines

#162

Post by chologrande »

shykes wrote: post #13
bovermyer wrote: post #8

Earlier quoted context omitted.

So basically, if I want to not write Jenkinsfiles but still use my company's existing Jenkins installation, I can use Dagger?

Yes :) You can write one last Jenkinsfile that runs Dagger, then do everything inside Dagger. Then you can run the exact same Dagger configuration on another CI system, or on your laptop. All you need is a Docker-compatible runtime (we run everything on buildkit under the hood).

> You can write one last Jenkinsfile that runs Dagger

I'm very confused by this sentiment. This approach loses the best of existing CI tooling does it not?

Jenkins users lose understanding of what's actually being executed, what stage they're in, and how long it took. It might provide convenience, and the a (great) benefit of running locally the same as in your CI environment, but it seems to me this would make it difficult for devs to easily understand where/why their build failed, since it just has one megastep.

Re: Dagger: a new way to build CI/CD pipelines

#163

Post by zepn »

shykes wrote: post #4

Hi everyone, I'm one of the co-founders of Dagger (and before that founder of Docker). This is a big day for us, it's our first time sharing something new since, you know... Docker. If you have any questions, I'll be happy to answer them here!

Thanks for answering Qs. Does this compete directly with Tekton ( https://tekton.dev/ ), or do you imagine a way the two could interoperate? Why choose Dagger over Tekton to power pipelines?

Re: Dagger: a new way to build CI/CD pipelines

#164

Post by ______-_-______ »

chologrande wrote: post #162
shykes wrote: post #13

Earlier quoted context omitted.

Yes :) You can write one last Jenkinsfile that runs Dagger, then do everything inside Dagger. Then you can run the exact same Dagger configuration on another CI system, or on your laptop. All you need is a Docker-compatible runtime (we run everything on buildkit under the hood).

> You can write one last Jenkinsfile that runs Dagger I'm very confused by this sentiment. This approach loses the best of existing CI tooling does it not? Jenkins users lose understanding of what's actually being executed, what stage they're in, and how long it took. It might provide convenience, and the a (great) benefit of running locally the same as in your CI environment, but it seems to me this would make it di…

I assume if the jenkins-step failed, you'd click a link to the dagger UI to see which dagger-step failed. Alternatively, never open jenkins at all and instead keep a tab open with the dagger UI.

Re: Dagger: a new way to build CI/CD pipelines

#165

Post by shykes »

chippiewill wrote: post #157
shykes wrote: post #102

Earlier quoted context omitted.

> > Note: you won't need to configure caching though, because Dagger automatically caches all actions out of the box :) > Is this strictly because it's using Docker underneath and layers can be reused? Not exactly: we use Buildkit under the hood, not Docker. When you run a Dagger action, it is compiled to a DAG, and run by buildkit. Each node in the DAG has content-addressed inputs. If the same node has been executed…

> Don't let the "docker registry" option confuse you: buildkit cache data isn't the same as docker images, so it doesn't carry the same garbage collection and tag pruning problems. IIRC doesn't buildkit store its cache data as fake layer blobs + manifest? I don't see how it can avoid the garbage collection and tag pruning problems since those are limitations of the registry implementation itself.

You still need to manage the size of your cache, since in theory it can grow infinitely. But it’s a different problem than managing regular Docker images, because there are no named references to worry about: just blobs that may or may not be reused in the future. The penalty for removing the “wrong” blob is a possible cache miss, not a broken image.

Dagger currently doesn’t help you remove blobs from your cache, but if/when it does, it will work the same way regardless of where the blobs are stored (except for the blob storage driver).

Re: Dagger: a new way to build CI/CD pipelines

#166

Post by throwaway787544 »

We've really jumped the shark with people not understanding the word DevOps, haven't we? When the Docker people don't get it, we should probably give up.

Fine. "DevOps" now means "some something development something something servers something something operations". Are you happy now, tech world? You've made Patrick Debois cry.

Re: Dagger: a new way to build CI/CD pipelines

#167

Post by charcircuit »

scubbo wrote: post #127
gregmac wrote: post #54

Your Windows instructions/process needs work. > curl https://dl.dagger.io/dagger/install.ps1 -OutFile install.ps1 This uses the `curl` alias, which is really `Invoke-WebRequest`. It also makes the incorrect assumption I haven't fixed this dumb Microsoft mistake to be an alias to actual curl.exe. > [Windows] We try to move the dagger binary under C:\Windows\System32 Ack, please don't do this! This is similar to instal…

> This is similar to installing something under /usr/sbin/ As someone who's trying to get to grips with the Linux filesystem conventions, would you mind elaborating on a) why that's wrong, and b) what you would suggest instead? This reference[0] suggests that `/usr/sbin` is for "general system-wide binaries with superuser (root) privileges required" (and `/usr/bin` for those that don't require root privileges). I've…

/usr/sbin is a legacy artifact that shouldn't be used. /usr/sbin is usually just symlinked to /usr/bin

Re: Dagger: a new way to build CI/CD pipelines

#168

Post by bovermyer »

chadcmulligan wrote: post #158
bovermyer wrote: post #11

Earlier quoted context omitted.

The name war continues unabated. I'm a "devops engineer." Besides pipeline code and maintaining/building out infrastructure, I also write CLI applications to handle complex infrastructure tasks. At what point is a "devops engineer" not a software engineer? [p.s., I prefer the term "platform engineer," personally]

So you spend all your time looking after dev infrastructure? Can I ask how many devs there are where you work?

Somewhere around 75, if I had to guess.

Re: Dagger: a new way to build CI/CD pipelines

#169

Post by malkia »

simulate-me wrote: post #155
nickysielicki wrote: post #44

I’m highly jealous of teams that can containerize all their build tools. If you deal with proprietary toolchains that are tens of gigabytes (Windows WDK, Xilinx Vivado/Vitis) it’s just untenable, and that’s before even mentioning licensing. Even Azure doesn’t have a great solution for WDK development. It’s hard to feel like we’re not being left behind. Bind mounting the tools into the container is an option but at th…

In my personal experience, Windows development is absolutely awful. Nothing about it is ergonomic, and the management of the development environment is just painful. Obviously you don't always have a choice, but creating a nice development experience is clearly not a priority for Microsoft. One really basic example: I was deploying files to a Windows Server box using scp, but my internet connection died midway throug…

I love unix tools (grep, sed, cut, etc.), and while there are some good sub-systems (msys2, cygwin), they might be bit heavey. For that the windows version of busybox - https://frippery.org/busybox/ - and then I make sure my scripts are not using too powerful features of said tools (grep especially), such that the version in busybox works. Great, and also possible to port some of that back to linux (but I mostly use it to build something, or extract some data but want to share the .bat file with others - one day when I get better in PowerShell I'll try there more).

Re: Dagger: a new way to build CI/CD pipelines

#170

Post by pid-1 »

throwaway787544 wrote: post #166

We've really jumped the shark with people not understanding the word DevOps, haven't we? When the Docker people don't get it, we should probably give up. Fine. "DevOps" now means " some something development something something servers something something operations". Are you happy now, tech world? You've made Patrick Debois cry.

It's simple.

Before we had developer and operations teams.

Now operations write yaml and we call them devops.

Post reply on HN