Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dagger: a new way to build CI/CD pipelines

dagger.io

Post reply on HN
Print view View original

181–190 of 267 posts

Re: Dagger: a new way to build CI/CD pipelines

#181

Post by __turbobrew__ »

candiddevmike wrote: post #123
epage wrote: post #48

> Dev/CI drift: instead of writing the same automation twice - once in a CI-specific configuration, and again in a Docker compose file, shell script or Makefile - simply write a Dagger plan once, then run it in all environments. Working on developer tooling, a lot of times I would hear from people that they wanted CI and dev to be 100% the same and wanted a simple "run all CI locally" command to pre-check before post…

I don't understand why folks can't write everything in a makefile and then call make targets in CI.

This is the way. Writing your CI in groovy is a fast track to hell.

Re: Dagger: a new way to build CI/CD pipelines

#182

Post by __turbobrew__ »

nunez wrote: post #80
Aeolun wrote: post #16

After reading this entire post, I’m still left wondering what problem this solves for me, beyond fluffy promises of ‘everything is going to be better’. At the very least I’d want to see a comparison with what we have now, to show me how this is better. I get that I can try to explore more, but if I don’t get a compelling reason to do so after reading the introductory post, I’m not very motivated to do so Any motivati…

It mostly solves this problem: - write code - run tests - commit code - update CI - commit - CI broken - update CI - commit - CI broken - update CI - ... The workarounds for this are generally awful. For Jenkins, you stage your own instance locally and configure your webhooks to use that. It's exactly as terrible as it sounds, and I never recommend this approach. For Travis and Concourse (I think), you can use their…

> every other CI product still relies on VMs for build work.

Gitlab CI has dockerized runners? Works great!

Re: Dagger: a new way to build CI/CD pipelines

#183

Post by awinter-py »

diggs wrote: post #93

There's so much more to CI/CD than the build definitions (e.g. dashboarding, log viewing/retention, access control, manual interventions, secret management, test reporting, etc.) and while some of your points resonate very strongly with me (e.g. local builds), I can't help but wonder what the endgame is here? You've raised $27m to what? Give away a free tool for engineers to define their builds in? While they continu…

IMO the best open source infra lock-in strategy is kube

took a while for E/AKS to catch up with G, things like ingress still vary in DX across clouds

'the same everywhere but it only works here'

(no comment on morality of this, quality of kube generally, or whether this team will do the same. docker IMO missed the chance to be a cloud host or a standard interface)

Re: Dagger: a new way to build CI/CD pipelines

#184

Post by ghostdog20 »

shykes wrote: post #179
shadycuz wrote: post #173

I really wish you would have stuck with a language that is higher in popularity. Python, JavaScript or such.

We did. Each action can be written in the language of your choice (I personally use bash) then wrapped in a declarative CUE config (typically quite short). Then you compose declaratively in cue. Here is an example of a package to deploy to netlify: https://github.com/dagger/dagger/tree/main/pkg/universe.dagg...

Have you heard of or explored https://github.com/aws/constructs (related: https://github.com/aws/jsii and https://github.com/aws/aws-cdk)?

This is what CDK uses for declarative modeling, but gives the opportunity to use languages/tooling that most devs are already familiar with. CDK8s already uses it as a replacement for yaml (technically, the yaml becomes an implementation detail rather than actually replaced)

Re: Dagger: a new way to build CI/CD pipelines

#185

Post by shitloadofbooks »

awinter-py wrote: post #183
diggs wrote: post #93

There's so much more to CI/CD than the build definitions (e.g. dashboarding, log viewing/retention, access control, manual interventions, secret management, test reporting, etc.) and while some of your points resonate very strongly with me (e.g. local builds), I can't help but wonder what the endgame is here? You've raised $27m to what? Give away a free tool for engineers to define their builds in? While they continu…

IMO the best open source infra lock-in strategy is kube took a while for E/AKS to catch up with G, things like ingress still vary in DX across clouds 'the same everywhere but it only works here' (no comment on morality of this, quality of kube generally, or whether this team will do the same. docker IMO missed the chance to be a cloud host or a standard interface)

Can you elaborate on your concerns? I've not seen them occur in practice at all.

Sure each provider does things differently, but they still work with primitive Kubernetes manifests at the end of the day.

A migration from one to the other is nothing more than changing some annotations or potentially transforming the "shape" of some lists or maps.

Re: Dagger: a new way to build CI/CD pipelines

#186

Post by francislavoie »

candiddevmike wrote: post #123
epage wrote: post #48

> Dev/CI drift: instead of writing the same automation twice - once in a CI-specific configuration, and again in a Docker compose file, shell script or Makefile - simply write a Dagger plan once, then run it in all environments. Working on developer tooling, a lot of times I would hear from people that they wanted CI and dev to be 100% the same and wanted a simple "run all CI locally" command to pre-check before post…

I don't understand why folks can't write everything in a makefile and then call make targets in CI.

I hate makefile syntax even more than I hate yaml

Re: Dagger: a new way to build CI/CD pipelines

#187

Post by abrahms »

laurentb wrote: post #118
shykes wrote: post #98

Earlier quoted context omitted.

Good question. No, we don't intend to replace the CI service. We think of CI as infrastructure, and there are plenty of great infrastructure providers out there. And you are absolutely correct that for Dagger to succeed, it has to remain agnostic to infrastructure providers, which means we cannot build an infrastructure provider business ourselves. If the experience of hosting Dagger becomes so bad that it affects th…

Hey Solomon, question coming from a sales guy among this crowd... Who does this impact the most and what outcomes does that help them achieve? At the end of the day, you need to pay the bills, for you and your team, who is going to sign the dotted line and tell you that yes "I need to invest $100k on this because it solves a major pain!"? Whilst I can see this as a nice to have, I'm having a hard time understanding w…

As someone who does a bunch of CI/CD work, the answer is: supply chain and the security therein is a major focus of enterprises right now. Security folks as well as ops folks are keen on this. Big question is can those two groups convince the developers that this is a good idea (though vice versa should be fairly easy at a certain company size).

Re: Dagger: a new way to build CI/CD pipelines

#188

Post by thundergolfer »

epage wrote: post #48

> Dev/CI drift: instead of writing the same automation twice - once in a CI-specific configuration, and again in a Docker compose file, shell script or Makefile - simply write a Dagger plan once, then run it in all environments. Working on developer tooling, a lot of times I would hear from people that they wanted CI and dev to be 100% the same and wanted a simple "run all CI locally" command to pre-check before post…

> I would hear from people that they wanted CI and dev to be 100% the same and wanted a simple "run all CI locally" command to pre-check before posting.

This is exactly what developers should want. It's the most efficient workflow for a dev, because we then don't have to think at all. This is a huge dev efficiency anti-pattern: thinking your code is good to merge, pushing the change, and then finding out 10-20mins later that CI isn't happy for some reason that wasn't natural to check locally.

The thesis in the following is the way: https://gregoryszorc.com/blog/2021/04/07/modern-ci-is-too-co...

Re: Dagger: a new way to build CI/CD pipelines

#189

Post by gregmac »

pc86 wrote: post #130
gregmac wrote: post #54

Your Windows instructions/process needs work. > curl https://dl.dagger.io/dagger/install.ps1 -OutFile install.ps1 This uses the `curl` alias, which is really `Invoke-WebRequest`. It also makes the incorrect assumption I haven't fixed this dumb Microsoft mistake to be an alias to actual curl.exe. > [Windows] We try to move the dagger binary under C:\Windows\System32 Ack, please don't do this! This is similar to instal…

OT but is Scoop competing with Chocolatey? I'm having trouble seeing the difference and I've never heard of Scoop before

There's a good comparison on the Scoop wiki [1].

Chocolatey is a mixed bag. Since it is basically just wrapper scripts around upstream installers, it very much depends on what the upstream installer does. To me, it acts less like a package manager (apt) and more like a rudimentary installer runner.

This causes all kinds of annoying usability issues. Chocolatey doesn't know if you uninstall via Add/Remove programs, or update via standalone or self-update mechanism, and will just show incorrect install and/or version info, and fail to upgrade properly. A lot of packages don't even install the version that Chocolatey says, but instead just install the latest at time of install.

Scoop is way different, and has none of these problems. Funny enough, they describe it as "not a package manager" but it feels way closer to apt to me. Everything is essentially a portable version of the app, and it puts them in ~/scoop/apps/[name]/[version], creates a junction (symlink) to ~/scoop/apps/[name]/current, and adds executable "shims" to ~/scoop/shims (which is in the user's path). There's no Windows "uninstall" entry, no versions to be desync'd, and no hidden garbage that can sneak in.

[1] https://github.com/ScoopInstaller/Scoop/wiki/Chocolatey-Comp...

Re: Dagger: a new way to build CI/CD pipelines

#190

Post by larusso »

jaaron wrote: post #131
nickysielicki wrote: post #44

I’m highly jealous of teams that can containerize all their build tools. If you deal with proprietary toolchains that are tens of gigabytes (Windows WDK, Xilinx Vivado/Vitis) it’s just untenable, and that’s before even mentioning licensing. Even Azure doesn’t have a great solution for WDK development. It’s hard to feel like we’re not being left behind. Bind mounting the tools into the container is an option but at th…

100% this. I feel like there's a ton of innovation in the cloud/container CI/CD space, but next to nothing elsewhere. In fact, some of the innovations in CI/CD make things _more_ difficult for those of us developing in other environments (such as game development). There's a lot of low hanging fruit for improving things elsewhere.

Exactly. I manage a CI infrastructure for a mobile game company (28 macs, a few Linux and windows VMs). I still can‘t use docker because our main development box is still macOS. We are slowly moving this to Linux. The reasons why we are still using macs as the main build machines are manyfolded. I know that android SDK and Unity run on Linux but our whole company came from a iOS first model and still uses macOS as the primary development box. But even docker would not help 100%. I have a strong emphasis that the tools run locally as well. And we have a mix of build and development tools. What I mean is that one and the same basic script should be used both during CI and differently with other parameters during development. Jenkins is our build executor. The whole build is setup with gradle as it had 5 years ago some very nice properties over other tools (self bootstrapping, robust plugin system, lots of libraries available). We build everything around this and only use Jenkins-pipelines to kick of said gradle jobs. But I would prefer a nicer solution. Ah we manage the machines with ansible (Mac, windows and Linux) I guess redhad had not thought about the fact that someone took the Multiplattform claims to the test. I can say it is kind of a nightmare to Code playbooks and roles against three different OS types.
Post reply on HN