Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dagger Python SDK: Develop Your CI/CD Pipelines as Code

dagger.io

Post reply on HN
Print view View original

31–40 of 92 posts

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#31

Post by wmichelin »

diarrhea wrote: post #20

I've long wished to be just writing (ideally) Python to define and run CI/CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes. Round-trip times for GitHub Actions are too high: sometimes you're waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There's zero IDE support for almost anything beyond getting the YAML syntax itse…

I don't know if this is just scoped to my company, but drone seemingly supports starlark, which is just python

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#33

Post by JackMcMack »

helderco wrote: post #3

Hey, lead developer for the Python SDK here. I have a few examples in https://github.com/helderco/dagger-examples and I plan to add more. There's also reference documentation in https://dagger-io.readthedocs.io/ so you can get a birds eye view on what's possible.

The api looks very simple and easy, that's a very good sign. We've been doing similar stuff in python, mainly in combination with Pulumi automation.

You can do some very fancy things, and I'm sure Dagger would be a nice addition to that. But honestly the thing I'm missing most is a nice UI above all that. How to present realtime and historic logs (and state) for multiple concurrent actions is still not as easy as I think it could be.

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#34

Post by shykes »

diarrhea wrote: post #20

I've long wished to be just writing (ideally) Python to define and run CI/CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes. Round-trip times for GitHub Actions are too high: sometimes you're waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There's zero IDE support for almost anything beyond getting the YAML syntax itse…

> What's unclear to me from the article and video is how this can replace e.g. GitHub Actions. Their integration with e.g. PR status checks and the like is a must, of course.

You guessed correctly: Dagger does not replace Github Actions, they are complementary. The Dagger project itself uses Github Actions and Dagger together :)

> Would Dagger just run on top of a `ubuntu-latest` GHA runner?

Sure, you can do that. The only dependency to run Dagger is Docker or any OCI-compatible runtime. So, assuming the `ubuntu-latest` runner has docker installed, you can just execute your Dagger-enabled tool, and it should work out of the box.

For example here's our own github workflow for testing the Python SDK (you can look around for other workflows): https://github.com/dagger/dagger/blob/main/.github/workflows...

Note that the word "dagger" doesn't even appear, since Dagger is embedded as a library (in this case, using the Go SDK). As far as GHA is concerned, it's just executing a regular binary.

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#35

Post by bennine »

shykes wrote: post #2

Hi everyone, Dagger co-founder here. Happy to answer any questions. We also released an update to the Go SDK a few days ago: https://dagger.io/blog/go-sdk-0.4

Hey there, this looks interesting. What's the scope of this project? Is it meant to be closer to developers or do you see this being used in production?

How is progress of builds, observability, etc being tackled?

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#37

Post by lmm »

pydry wrote: post #22
diarrhea wrote: post #20

I've long wished to be just writing (ideally) Python to define and run CI/CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes. Round-trip times for GitHub Actions are too high: sometimes you're waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There's zero IDE support for almost anything beyond getting the YAML syntax itse…

>Are Turing-complete languages a bad choice for pipelines? I'd say that non turing complete languages are a bad fit for pipelines. Even mildly complex pipeline will eventually have loops and conditionals. Better python than some originally-a-config-YAML language turned into an imperative monstrosity with loops and conditionals bolted on.

Something like Dhall might be viable for pipelines. I wonder if there are any efforts in that direction.

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#39

Post by travisgriggs »

Years ago, I had inherited a complicated C code base with some clever Makefile jujitsu. None of the recent maintainers understood it at its core. They just put a little lipstick here, and a little there to keep things running.

I sat down and extracted the compiler/link flags, and then wrote a Python script to do the build. The code was smaller, and built faster.

Every “build” engine evolves from being a simple recipe processor to the software equivalent of a 5 axis CnC mill. Some things should not succumb to one size fits all.

Post reply on HN