Feature flags in a CI pipeline
andydote.co.uk
Feature flags in a CI pipeline
1–10 of 28 posts
Re: Feature flags in a CI pipeline
#2I’ve been looking for a way to do integration tests for our pipelines. So far haven’t found anything I could use.
Re: Feature flags in a CI pipeline
#3I’ve been looking for a way to do integration tests for our pipelines. So far haven’t found anything I could use.
I use matrix tests with github actions to test my kt-search client with different versions of elastisearch and opensearch. Pretty easy to set up: https://github.com/jillesvangurp/kt-search/blob/master/.gith...
Basically it fires up elasticsearch using docker-compose and then the integration tests run against that. You could use a similar strategy to test different feature flag combinations.
For some of our private projects, we use kts to generate the github action yaml files using this: https://github.com/krzema12/github-workflows-kt
Well worth checking out if you have more complex workflows. Yaml is just horrible in terms of copy paste reuse. Also nice to get some compile time safety and auto complete with our action files.
Re: Feature flags in a CI pipeline
#4Flagon's exit codes are as follows:
0 the flag queried is on (true) 1 the flag queried is off (false) 1 an error occurred querying the flag
I think that it would be nice that error and false had different exit codes.
Re: Feature flags in a CI pipeline
#5I’ve been looking for a way to do integration tests for our pipelines. So far haven’t found anything I could use.
I use matrix tests with github actions to test my kt-search client with different versions of elastisearch and opensearch. Pretty easy to set up: https://github.com/jillesvangurp/kt-search/blob/master/.gith... Basically it fires up elasticsearch using docker-compose and then the integration tests run against that. You could use a similar strategy to test different feature flag combinations. For some of our private pr…
Pretty sure the parent meant testing the pipelines themselves, end to end.
The tooling around pipelines is awful. A single typo in some variable’s name in a later stage can take minutes to catch. The feedback cycles are very long (cloud machines are much slower than local ones) and IDE tooling is bare-bones.
Just give me one large Python file with some library to manage common actions (building up the job DAG, accessing pull requests, easy shell access, …). We’d have refactoring, Turing completeness, type safety and so much more. A core downside would be managing the complexity of DevOps scripting going berserk. Personally I’d prefer that trade off.
Re: Feature flags in a CI pipeline
#6It looks like an additional tool and a new remote service to talk to. Are there any CIs that don't support editable variables on scopes larger than a build?
The article talks about GH Actions, they have org/repo/env variables.
Even Jenkins has variables usable from the build.
Re: Feature flags in a CI pipeline
#7Earlier quoted context omitted.
I use matrix tests with github actions to test my kt-search client with different versions of elastisearch and opensearch. Pretty easy to set up: https://github.com/jillesvangurp/kt-search/blob/master/.gith... Basically it fires up elasticsearch using docker-compose and then the integration tests run against that. You could use a similar strategy to test different feature flag combinations. For some of our private pr…
Pretty sure the parent meant testing the pipelines themselves, end to end. The tooling around pipelines is awful. A single typo in some variable’s name in a later stage can take minutes to catch. The feedback cycles are very long (cloud machines are much slower than local ones) and IDE tooling is bare-bones. Just give me one large Python file with some library to manage common actions (building up the job DAG, access…
Testing the pipelines is a thing as well. But pretty tricky to set up. You basically need to run dockerized actions locally for that. Not impossible and I've seen some attempts to do that. But I can't really justify spending a lot of time on this stuff.
The kotlin scripting support for github actions that I mentioned addresses things like typos, refactoring, and IDE tooling. Try it, it's pretty nice and easy to use. We actually have an integrity check as part of our build that runs the kts script to verify the yaml file stored in the repository is consistent with what the script generates.
Re: Feature flags in a CI pipeline
#8Earlier quoted context omitted.
I use matrix tests with github actions to test my kt-search client with different versions of elastisearch and opensearch. Pretty easy to set up: https://github.com/jillesvangurp/kt-search/blob/master/.gith... Basically it fires up elasticsearch using docker-compose and then the integration tests run against that. You could use a similar strategy to test different feature flag combinations. For some of our private pr…
Pretty sure the parent meant testing the pipelines themselves, end to end. The tooling around pipelines is awful. A single typo in some variable’s name in a later stage can take minutes to catch. The feedback cycles are very long (cloud machines are much slower than local ones) and IDE tooling is bare-bones. Just give me one large Python file with some library to manage common actions (building up the job DAG, access…
Exactly this. We are having quite complex pipelines and almost weekly basis we introduce some regression bugs there or change the rules so that they don't work anymore properly. It takes around 20 minutes to run the whole pipeline and of course some jobs are only triggered when merging to master etc. The development flow is painful and slow (although we have found some tooling to run some parts of our pipelines locally).
Re: Feature flags in a CI pipeline
#9I've done something like this with CircleCI and Github labels. The Continuation orb[1] allows you to dynamically generate your CI yml rather than using a static workflow. A Python script is executed which uses information about the PR (author, timestamp, labels applied to the PR, etc) and a CI template yml to add, remove, or skip jobs from the workflow. Some use cases are:
- Replacing a job with a newer job for testing purposes, as described in the article.
- Opting out of jobs, like "skip-deploy" or "skip-unit" labels, to speed up CI in some cases when these jobs aren't needed.
- Making some optional jobs required depending on the files that have been modified in the PR. You can get a list of modified files in a PR using the Github API. We use this for example if a new feature necessitates a new CI job but you want this job to be optional for branches that don't have this feature code yet.
1: https://circleci.com/developer/orbs/orb/circleci/continuatio...
Re: Feature flags in a CI pipeline
#10Earlier quoted context omitted.
I use matrix tests with github actions to test my kt-search client with different versions of elastisearch and opensearch. Pretty easy to set up: https://github.com/jillesvangurp/kt-search/blob/master/.gith... Basically it fires up elasticsearch using docker-compose and then the integration tests run against that. You could use a similar strategy to test different feature flag combinations. For some of our private pr…
Pretty sure the parent meant testing the pipelines themselves, end to end. The tooling around pipelines is awful. A single typo in some variable’s name in a later stage can take minutes to catch. The feedback cycles are very long (cloud machines are much slower than local ones) and IDE tooling is bare-bones. Just give me one large Python file with some library to manage common actions (building up the job DAG, access…
I've been building Jaypore CI for exactly this tradeoff.
The config is a single python file and I run jobs using a git hook on my own laptop.
I'd love some feedback on what else I could add to this project to make life easier for people.