Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dagger: a new way to build CI/CD pipelines

dagger.io

Post reply on HN
Print view View original

151–160 of 267 posts

Re: Dagger: a new way to build CI/CD pipelines

#151

Post by Nullabillity »

scubbo wrote: post #127
gregmac wrote: post #54

Your Windows instructions/process needs work. > curl https://dl.dagger.io/dagger/install.ps1 -OutFile install.ps1 This uses the `curl` alias, which is really `Invoke-WebRequest`. It also makes the incorrect assumption I haven't fixed this dumb Microsoft mistake to be an alias to actual curl.exe. > [Windows] We try to move the dagger binary under C:\Windows\System32 Ack, please don't do this! This is similar to instal…

> This is similar to installing something under /usr/sbin/ As someone who's trying to get to grips with the Linux filesystem conventions, would you mind elaborating on a) why that's wrong, and b) what you would suggest instead? This reference[0] suggests that `/usr/sbin` is for "general system-wide binaries with superuser (root) privileges required" (and `/usr/bin` for those that don't require root privileges). I've…

Generally, `/usr` is for stuff packaged by your distribution, while `/usr/local` (so, `/usr/local/bin` and so on) is for your own custom stuff. Both the `/usr` and `/usr/local` equivalents will be on your $PATH by default in most distros.

For stuff that isn't just a self-contained executable, consider installing it to the folder `/opt/$MY_APP` and either symlinking the main binary into `/usr/local/bin` or putting a wrapper script in there (if the thing doesn't like being symlinked outside of its primary install dir). The wrapper can be as simple as:

    #!/bin/sh
    exec /opt/my-app/bin/my-app "$@"

Re: Dagger: a new way to build CI/CD pipelines

#152

Post by shykes »

EddieRingle wrote: post #150
shykes wrote: post #133

Earlier quoted context omitted.

> > We will not rely on unusual licences to restrict competitors from running Dagger as a service. > Your "Trademark Guidelines" appear to contradict you: They do not. Software licenses and trademark guidelines are two different things. Some commercial open-source vendors have changed their licenses to restrict use of the software in various ways - typically to limit competition from large cloud providers. We don't d…

> > > We will not rely on unusual licences to restrict competitors from running Dagger as a service. > > Your "Trademark Guidelines" appear to contradict you: > They do not. Software licenses and trademark guidelines are two different things. Some commercial open-source vendors have changed their licenses to restrict use of the software in various ways - typically to limit competition from large cloud providers. We d…

> When you're threatening people with legal action you need to be clear, and right now the text on that page is not, according to what you're saying here.

That's good feedback, thank you. We can try and make it clearer as long as it remains legally correct and enforceable. Do you have specific feedback on which parts you found unclear, and why?

Re: Dagger: a new way to build CI/CD pipelines

#153

Post by shykes »

candiddevmike wrote: post #123
epage wrote: post #48

> Dev/CI drift: instead of writing the same automation twice - once in a CI-specific configuration, and again in a Docker compose file, shell script or Makefile - simply write a Dagger plan once, then run it in all environments. Working on developer tooling, a lot of times I would hear from people that they wanted CI and dev to be 100% the same and wanted a simple "run all CI locally" command to pre-check before post…

I don't understand why folks can't write everything in a makefile and then call make targets in CI.

That is a common pattern. Dagger is essentially a more modern iteration on that pattern.

Re: Dagger: a new way to build CI/CD pipelines

#154

Post by lazypenguin »

jaaron wrote: post #131
nickysielicki wrote: post #44

I’m highly jealous of teams that can containerize all their build tools. If you deal with proprietary toolchains that are tens of gigabytes (Windows WDK, Xilinx Vivado/Vitis) it’s just untenable, and that’s before even mentioning licensing. Even Azure doesn’t have a great solution for WDK development. It’s hard to feel like we’re not being left behind. Bind mounting the tools into the container is an option but at th…

100% this. I feel like there's a ton of innovation in the cloud/container CI/CD space, but next to nothing elsewhere. In fact, some of the innovations in CI/CD make things _more_ difficult for those of us developing in other environments (such as game development). There's a lot of low hanging fruit for improving things elsewhere.

Yes! I’m responsible for a game development CI/CD pipeline that needs to run on windows and I feel like it’s harder than it needs to be in 2022.

Re: Dagger: a new way to build CI/CD pipelines

#155

Post by simulate-me »

nickysielicki wrote: post #44

I’m highly jealous of teams that can containerize all their build tools. If you deal with proprietary toolchains that are tens of gigabytes (Windows WDK, Xilinx Vivado/Vitis) it’s just untenable, and that’s before even mentioning licensing. Even Azure doesn’t have a great solution for WDK development. It’s hard to feel like we’re not being left behind. Bind mounting the tools into the container is an option but at th…

In my personal experience, Windows development is absolutely awful. Nothing about it is ergonomic, and the management of the development environment is just painful. Obviously you don't always have a choice, but creating a nice development experience is clearly not a priority for Microsoft. One really basic example: I was deploying files to a Windows Server box using scp, but my internet connection died midway through the transfer. All subsequent deployments would fail until I could connect via RDP to kill the old SSH process holding an open file handle. Another example: I bought into the WSL hype, hoping it would solve all my problems. But you can't easily interact with files on the system outside of Linux subsystem, which made it more or less useless for my purposes. If I wanted to interact only with Linux files, I would have used a Linux-based server. Maybe I could get better at Powershell, but it felt overly verbose while also being less composable / powerful. The simple posix shell primitives were sorely missed. Running things on startup was also insane. I think I needed to add a triggered job that ran on user log in, and then configure the host to login a specific user when the host starts. Very crazy...

Re: Dagger: a new way to build CI/CD pipelines

#156

Post by jeremy_k »

jaaron wrote: post #131
nickysielicki wrote: post #44

I’m highly jealous of teams that can containerize all their build tools. If you deal with proprietary toolchains that are tens of gigabytes (Windows WDK, Xilinx Vivado/Vitis) it’s just untenable, and that’s before even mentioning licensing. Even Azure doesn’t have a great solution for WDK development. It’s hard to feel like we’re not being left behind. Bind mounting the tools into the container is an option but at th…

100% this. I feel like there's a ton of innovation in the cloud/container CI/CD space, but next to nothing elsewhere. In fact, some of the innovations in CI/CD make things _more_ difficult for those of us developing in other environments (such as game development). There's a lot of low hanging fruit for improving things elsewhere.

Do you work in game development? I would love to hear a little bit more about your experience. I play and follow Apex Legends and I'm always so curious about how bugs and regressions seem to make it into every one of their patches. As well as tons of new information that gets data-mined.

To me, its like they don't have branches (tons of new code not accessible in the game is released in a patch) and they don't have unit tests that catch bugs (a certain ability has 2 modes, with 2 different activation times. In the most recent patch, the activation times ended up the same) but it's possible that developing a game is far different than developing a SaaS product from a coding perspective. Or it could be that studio just has weird practices.

Re: Dagger: a new way to build CI/CD pipelines

#157

Post by chippiewill »

shykes wrote: post #102
mhitza wrote: post #89

Earlier quoted context omitted.

Thank you for the detailed response. I appreciate you taking the time. One last question/note. > Note: you won't need to configure caching though, because Dagger automatically caches all actions out of the box :) Is this strictly because it's using Docker underneath and layers can be reused? If so, unless those intermediary layers are somehow pushed/pulled by the dagger github action (or any associated CI/CD tool equ…

> > Note: you won't need to configure caching though, because Dagger automatically caches all actions out of the box :) > Is this strictly because it's using Docker underneath and layers can be reused? Not exactly: we use Buildkit under the hood, not Docker. When you run a Dagger action, it is compiled to a DAG, and run by buildkit. Each node in the DAG has content-addressed inputs. If the same node has been executed…

> Don't let the "docker registry" option confuse you: buildkit cache data isn't the same as docker images, so it doesn't carry the same garbage collection and tag pruning problems.

IIRC doesn't buildkit store its cache data as fake layer blobs + manifest?

I don't see how it can avoid the garbage collection and tag pruning problems since those are limitations of the registry implementation itself.

Re: Dagger: a new way to build CI/CD pipelines

#158

Post by chadcmulligan »

bovermyer wrote: post #11
Svoka wrote: post #7

So if it is a "devops engineer" why not call them just "ops" if they don't do software engineering?..

The name war continues unabated. I'm a "devops engineer." Besides pipeline code and maintaining/building out infrastructure, I also write CLI applications to handle complex infrastructure tasks. At what point is a "devops engineer" not a software engineer? [p.s., I prefer the term "platform engineer," personally]

So you spend all your time looking after dev infrastructure? Can I ask how many devs there are where you work?

Re: Dagger: a new way to build CI/CD pipelines

#159

Post by judge2020 »

simulate-me wrote: post #155
nickysielicki wrote: post #44

I’m highly jealous of teams that can containerize all their build tools. If you deal with proprietary toolchains that are tens of gigabytes (Windows WDK, Xilinx Vivado/Vitis) it’s just untenable, and that’s before even mentioning licensing. Even Azure doesn’t have a great solution for WDK development. It’s hard to feel like we’re not being left behind. Bind mounting the tools into the container is an option but at th…

In my personal experience, Windows development is absolutely awful. Nothing about it is ergonomic, and the management of the development environment is just painful. Obviously you don't always have a choice, but creating a nice development experience is clearly not a priority for Microsoft. One really basic example: I was deploying files to a Windows Server box using scp, but my internet connection died midway throug…

> But you can't easily interact with files on the system outside of Linux subsystem, which made it more or less useless for my purposes.

It’s possible in file explorer via \\wsl$, but that is not always supported by applications so it’s not 100%.

Post reply on HN