Live data from Hacker News

Dagger: Define software delivery workflows and dev environments

dagger.io

71–78 of 78 posts

Re: Dagger: Define software delivery workflows and dev environments

#71
post #64
post #41

Earlier quoted context omitted.

They have SDKs in many languages, not just Go. I use the python one. And they use code, not a DSL.

Right, my point is that this: https://docs.dagger.io/cookbook/services?sdk=typescript Still looks like "a circa-2000s Java builder API" and doesn't look like pleasant / declarative / idiomatic TypeScript, which is what aws-cdk pulled off. Genuinely impressively (imo), aws-cdk intermixes "it's declarative" (you're setting up your desired state) but also "it's code" (you can use all the usual abstractions) in a way tha…

Could you share an example of aws-cdk code that you think Dagger should take inspiration from? Dagger and aws-cdk work very differently under the hood, so it's difficult to make an apples-to-apples comparison. If there's a way to make Dagger more TS-native without sacrificing other important properties of Dagger, I'm interested. Thanks.

Re: Dagger: Define software delivery workflows and dev environments

#72
post #34

Earlier quoted context omitted.

I used the old CUE-based version when it came out, and was really excited about it. I liked it, and enjoyed working with CUE, but the API was clunky and incomplete. Then they completely abandoned not just the CUE frontend, but CUE altogether (while strenuously denying that they were doing so) for a GraphQL-based rewrite that focused on letting people use popular general-purpose languages to construct their workflows.…

Do you have any examples of your Devenv workflow you can share? I took a look at Dagger and really like the concept, but I'm trying to figure out the limitations/why there's so much negativity in this thread. I currently manage my development environments via NixOS and Devenv, so if I could just keep that and achieve the same functionality, that sounds good to me.

At the time, Devenv was still pretty barebones compared to now (the rate of feature development has been great in the past few years), so it didn't have much in the way of support for workflows per se.

Instead I used the scripts module to write shell scripts that bring along all of their dependencies. Thus all of our GitLab jobs are just one-line invocations of our wrapper scripts. We currently use a flake.nix-based Devenv and expose the scripts from the Devenv module as flake outputs, then just run them with `nix run`.

Nowadays, though, Devenv actually directly supports some relevant functionality in the form of tasks¹, which can be written in any language, and support dependency relationships and run concurrently by default.

We use those in a few places as well, though mostly in place of older, cruder `enterShell` integration. There, we use them: - to seamlessly ensure correct state of Git extensions: - all Git submodules are checked out - Git-LFS is configured and all large files are correctly checked out - to ensure valid OpenTofu caches since we use Nix instead of OpenTofu for managing providers - if .terraform dir doesn't exist, run `tofu init` - if Terraform providers lock file is older than devenv lock, run `tofu init -upgrade` - manage Poetry stubs and virtualenvs (most of this is configured behind the scenes upstream, but we do some extra stuff) - manage Bundler dependencies and binstubs for Ruby projects so executables under development can be involed naively from PATH (much of this is bespoke but upstreamable)

So a lot of those Devenv tasks are things you might do in build or pre-build stages of a CI pipeline. You can define the requisite dependency relationships and then just run `devenv test` or `devenv tasks ...` as your CI job.

What we don't have is any abstraction over the various CI platforms we use (currently GitLab, GitHub, and AzDO, with Jenkins support in the backlog). Unfortunately, for the sake of a better DX for our (internal) customers, we still integrate fairly directly with each CI platform, so we do have piles of vendor-specific YAML.

Our tasks and scripts are currently just Bash, but they're relatively nice Bash because their dependencies are managed with resholve and they're linted by ShellCheck, thanks to some experimental changes I made to Devenv's scripts module.

For our security/quality scans, we do, however, have some logic nestled in those Bash scripts which checks for a CI environment via env vars, then performs differential rather than full scans if we're in merge request contexts. Where the upstream tools don't directly support this, we basically perform "before" and "after" scans and take a diff. This is the direction I'd like to go in the future— all in on this, surfacing jobs in a "native" way be damned.

Unfortunately I can't directly share these examples as they belong to my employer and live in proprietary projects. But what I can do is open a couple of WIP PRs for the upstreamable work (new or modified Devenv modules) so you can take a look at that at least.

Also fwiw I think Dagger is still worth checking out. I just feel a bit burned by adopting it too early in its lifecycle, and generally advice caution about adopting tools developed by VC-backed startups.

--

1: https://devenv.sh/tasks/

Re: Dagger: Define software delivery workflows and dev environments

#73
post #70
post #69

Earlier quoted context omitted.

Thanks for the response. I think multi-language support is a great feature, and I understand why you had to go for it. While I'm sure some people likely switched away from CUE once they had the chance because they weren't interested in working with a novel and perhaps quirky DSL, I'm also sure some stopped using the CUE SDK just because it was clear to them that it was being abandoned— I know that because I'm one of…

I understand. We really did try to port the CUE SDK over to the new APIs, but there were impedance mistmatches that made it difficult to do so without major breaking changes - basically we would have needed to design a new SDK from scratch. We asked for opinions on our discord, and it felt like there weren't enough people interested to justify that kind of effort. For a while there was activity on the #cue channel ab…

> It looks like you were in the minority that would have preferred to continue using the original CUE SDK - I'm sorry that we didn't find a way to continue supporting it.

I admit that when I went to the CUE documentation (because I was learning Dagger!) and read about the idea of thinking of validation as locating both schema and configuration on an infinite type/value lattice and sort of walking down from schema to concrete configuration, I thought "holy shit, it makes perfect sense". I'd never really thought about unification in configuration languages before, and it's a really cool idea, one of those things that's simple enough to be intuitive but also really powerful. First-class deep merge support is something that I really miss in hacky configuration languages like HCL, for example. So right away I felt "these CUE guys are onto something! let's see how it pans out".

It felt painful for me to abandon that investment, as I'm sure it did for many on your team, too.

While I'm fairly happy with what I've managed to do since moving on from using Dagger, the CI space is still a mess overall, and I think it needs tools like Dagger and more. So while it will likely be some time before I reevaluate Dagger for use at my current job, I do still wish you and your team present and continued success!

Thanks again for dropping by and listening.

Re: Dagger: Define software delivery workflows and dev environments

#74
post #18
post #11

Earlier quoted context omitted.

Do you have any more details on using Cue with GHA? I've also looked at Dagger and been quite disappointed with it (and their terrible documentation).

When I got started it was much more difficult as you had to do a lot of manual work to get things started, and you really had to believe the promises that CUE offered (which I did...), but nowadays they've made so many steps in the right direction that getting something going is far quicker! Here are a few links to whet your appetite: - https://cue.dev/docs/getting-started-with-github-actions-cue... - https://cue.dev…

Interesting approach! I can see promise in the vet tooling assuming the module keeps up with GitHub's schema well.

If you had to pick, what's the "killer" feature for you with this module?

Re: Dagger: Define software delivery workflows and dev environments

#75
post #21
post #18

Earlier quoted context omitted.

When I got started it was much more difficult as you had to do a lot of manual work to get things started, and you really had to believe the promises that CUE offered (which I did...), but nowadays they've made so many steps in the right direction that getting something going is far quicker! Here are a few links to whet your appetite: - https://cue.dev/docs/getting-started-with-github-actions-cue... - https://cue.dev…

Maybe it’s just me, but these sample workflows don’t look less complicated, just another kind of complex? If you’re already heavily using CUE in your project this lateral complexity shift might make sense, but I don’t see why I would start using it…

Like my guy 'diarrhea' already echoed: using CUE absolutely does not make sense at a small scale; just write your YAML and get on with your day. We were using it to generate dozens upon dozens of GitHub Actions workflows from what was essentially a single source of truth, and because CUE can export to JSON too then that single source of truth could then easily be leveraged to provide other input files to be used elsewhere.

Re: Dagger: Define software delivery workflows and dev environments

#77
post #71
post #64

Earlier quoted context omitted.

Right, my point is that this: https://docs.dagger.io/cookbook/services?sdk=typescript Still looks like "a circa-2000s Java builder API" and doesn't look like pleasant / declarative / idiomatic TypeScript, which is what aws-cdk pulled off. Genuinely impressively (imo), aws-cdk intermixes "it's declarative" (you're setting up your desired state) but also "it's code" (you can use all the usual abstractions) in a way tha…

Could you share an example of aws-cdk code that you think Dagger should take inspiration from? Dagger and aws-cdk work very differently under the hood, so it's difficult to make an apples-to-apples comparison. If there's a way to make Dagger more TS-native without sacrificing other important properties of Dagger, I'm interested. Thanks.

Hello! Yeah, I totally get Dagger is more "hey client please create a DAG via RPC calls", but just making something up in 30 seconds, like this is what I had in mind:

https://gist.github.com/stephenh/8c7823229dfffc0347c2e94a3c9...

Like I'm still building a DAG, but by creating objects with "kinda POJOs" (doesn't have to be literally POJOs) and then stitching them together, like the outputs of 1 construct (the build) can be used as inputs to the other constructs (tests & container).

Re: Dagger: Define software delivery workflows and dev environments

#78
post #66

Earlier quoted context omitted.

People aren’t going to confuse a physical knife with this. Android is the most popular operating system on earth. Names can overlap but you shouldn’t choose one where you can’t live up to owning it.

People likewise won't confuse a DI framework for Android with a GitHub Actions replacement. Complaints about overlapping names would make sense if both Daggers were DI frameworks or Github Actions replacements, but they're not. > can’t live up to owning it. ?

People absolutely might confuse a DI framework for Android with this, seeing as lots of teams use both? They are both software products, to start?

> > can’t live up to owning it.

> ?

Dagger, the DI framework, is very popular, supported by Google, and the primary way to do DI on Android (for many). It is not going anywhere. It is unlikely that this tool or product will create a bigger audience. Therefore, it might never "live up to the name," compared to the benchmark set by current Dagger. It's a simple concept.

Post reply on HN