Live data from Hacker News

Dagger Python SDK: Develop Your CI/CD Pipelines as Code

dagger.io

21–30 of 92 posts

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#21
post #2

Hi everyone, Dagger co-founder here. Happy to answer any questions. We also released an update to the Go SDK a few days ago: https://dagger.io/blog/go-sdk-0.4

A) Why async in the user code? Is it really necessary?

B) Can you mock pipeline events?

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#22

I've long wished to be just writing (ideally) Python to define and run CI/CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes. Round-trip times for GitHub Actions are too high: sometimes you're waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There's zero IDE support for almost anything beyond getting the YAML syntax itse…

>Are Turing-complete languages a bad choice for pipelines?

I'd say that non turing complete languages are a bad fit for pipelines. Even mildly complex pipeline will eventually have loops and conditionals.

Better python than some originally-a-config-YAML language turned into an imperative monstrosity with loops and conditionals bolted on.

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#23
post #2

Hi everyone, Dagger co-founder here. Happy to answer any questions. We also released an update to the Go SDK a few days ago: https://dagger.io/blog/go-sdk-0.4

It looks awesome! I have a clarification question on this one:

> Using the SDK, your program prepares API requests describing pipelines to run, then sends them to the engine. The wire protocol used to communicate with the engine is private and not yet documented, but this will change in the future. For now, the SDK is the only documented API available to your program.

Does it mean the sdk is making a round trip to the dagger API remotely somewhere, or is the round trip to a locally running docker container?

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#24
post #22

I've long wished to be just writing (ideally) Python to define and run CI/CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes. Round-trip times for GitHub Actions are too high: sometimes you're waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There's zero IDE support for almost anything beyond getting the YAML syntax itse…

>Are Turing-complete languages a bad choice for pipelines? I'd say that non turing complete languages are a bad fit for pipelines. Even mildly complex pipeline will eventually have loops and conditionals. Better python than some originally-a-config-YAML language turned into an imperative monstrosity with loops and conditionals bolted on.

Not a snarky question : what about having data and config in yml, and some high level tool like ansible or terraform and the occasional bash?

You loop and branch in ansible/terraform

I hate that setup but also I have a hard time thinking of something else

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#25

I've long wished to be just writing (ideally) Python to define and run CI/CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes. Round-trip times for GitHub Actions are too high: sometimes you're waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There's zero IDE support for almost anything beyond getting the YAML syntax itse…

Think about it from the perspective of the provider of the build platform (e.g. CircleCI, GitLab, GitHub). There are way fewer edge cases parsing YAML files than allowing Turing complete languages.

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#26
post #22

Earlier quoted context omitted.

>Are Turing-complete languages a bad choice for pipelines? I'd say that non turing complete languages are a bad fit for pipelines. Even mildly complex pipeline will eventually have loops and conditionals. Better python than some originally-a-config-YAML language turned into an imperative monstrosity with loops and conditionals bolted on.

Not a snarky question : what about having data and config in yml, and some high level tool like ansible or terraform and the occasional bash? You loop and branch in ansible/terraform I hate that setup but also I have a hard time thinking of something else

Looping in ansible/terraform is exactly the problem. Logic and control flow in YAML/hcl is a nightmare. Plus there's no debugging! You can't set breakpoints in a yaml file or HCL.

Adding more YAML to be parsed by other YAML is just terrible at scale.

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#27
post #23
post #2

Hi everyone, Dagger co-founder here. Happy to answer any questions. We also released an update to the Go SDK a few days ago: https://dagger.io/blog/go-sdk-0.4

It looks awesome! I have a clarification question on this one: > Using the SDK, your program prepares API requests describing pipelines to run, then sends them to the engine. The wire protocol used to communicate with the engine is private and not yet documented, but this will change in the future. For now, the SDK is the only documented API available to your program. Does it mean the sdk is making a round trip to th…

Hi, on the locally running engine.

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#28

I've long wished to be just writing (ideally) Python to define and run CI/CD pipelines. YAML is simply hell, `if` keys in such data description languages are cruel jokes. Round-trip times for GitHub Actions are too high: sometimes you're waiting for 10 minutes just to run into a dumb typo, empty string-evaluated variable or other mishap. There's zero IDE support for almost anything beyond getting the YAML syntax itse…

I can only agree, sitting with this right now and it's all so brittle. Of course, it's not just YAML, it's Javascript, Powershell and Bash embedded into strings plus various hacky methods of escaping, templating and sending values between steps. It's really a mess.

Re: Dagger Python SDK: Develop Your CI/CD Pipelines as Code

#29
post #22

Earlier quoted context omitted.

>Are Turing-complete languages a bad choice for pipelines? I'd say that non turing complete languages are a bad fit for pipelines. Even mildly complex pipeline will eventually have loops and conditionals. Better python than some originally-a-config-YAML language turned into an imperative monstrosity with loops and conditionals bolted on.

Not a snarky question : what about having data and config in yml, and some high level tool like ansible or terraform and the occasional bash? You loop and branch in ansible/terraform I hate that setup but also I have a hard time thinking of something else

I don't think there's a meaningful distinction between configuration and code in a CI pipeline. This is what people try to do, and it's frankly a massive waste.

The problem is the attempt to make a distinction when there cannot be one.

Post reply on HN