Live data from Hacker News

Feature flags in a CI pipeline

andydote.co.uk

11–20 of 28 posts

Re: Feature flags in a CI pipeline

#11
post #4

Flagon's exit codes are as follows: 0 the flag queried is on (true) 1 the flag queried is off (false) 1 an error occurred querying the flag I think that it would be nice that error and false had different exit codes.

It actually does have different error codes; I think I just forgot to update the docs:

0 - flag on 1 - flag off 2 - error

I'll update the docs soon!

Re: Feature flags in a CI pipeline

#12

It looks like an additional tool and a new remote service to talk to. Are there any CIs that don't support editable variables on scopes larger than a build? The article talks about GH Actions, they have org/repo/env variables. Even Jenkins has variables usable from the build.

Sure you can use the environment variables, and if you have simple uses cases this is probably the best way to go.

In this case, we're using LaunchDarkly's targeting to serve different variations to groups of users, and also using the in built monitoring LaunchDarkly provides to see how often each is requested etc.

Re: Feature flags in a CI pipeline

#13
post #5

Earlier quoted context omitted.

I use matrix tests with github actions to test my kt-search client with different versions of elastisearch and opensearch. Pretty easy to set up: https://github.com/jillesvangurp/kt-search/blob/master/.gith... Basically it fires up elasticsearch using docker-compose and then the integration tests run against that. You could use a similar strategy to test different feature flag combinations. For some of our private pr…

Pretty sure the parent meant testing the pipelines themselves, end to end. The tooling around pipelines is awful. A single typo in some variable’s name in a later stage can take minutes to catch. The feedback cycles are very long (cloud machines are much slower than local ones) and IDE tooling is bare-bones. Just give me one large Python file with some library to manage common actions (building up the job DAG, access…

> The feedback cycles are very long (cloud machines are much slower than local ones) and IDE tooling is bare-bones.

I'm continuously amazed that none of the major CI providers offer standalone tooling to run and debug your CI pipelines locally. Seems like it'd be a killer feature for anyone working with complex pipelines.

Re: Feature flags in a CI pipeline

#14

It looks like an additional tool and a new remote service to talk to. Are there any CIs that don't support editable variables on scopes larger than a build? The article talks about GH Actions, they have org/repo/env variables. Even Jenkins has variables usable from the build.

Sure you can use the environment variables, and if you have simple uses cases this is probably the best way to go. In this case, we're using LaunchDarkly's targeting to serve different variations to groups of users, and also using the in built monitoring LaunchDarkly provides to see how often each is requested etc.

It makes sense if you already use a service like LaunchDarkly but adding a whole new service like LaunchDarkly to your stack seems overkill for most people using feature flags. It took me a long time to understand how to work with feature flags in projects that don't require all the monitoring and overhead these services provide because all videos and articles I could find on the subject always framed them in terms of using a service like that.

At the end of the day, a feature flag is just a boolean variable determined at run time that can be used to switch code paths on or off. For most purposes that can be as simple as having a list of strings on your user model and checking if it contains a specific value. Adding a service can be valuable if you need to be able to manage those flags more intelligently or monitor them or want to do more structured A/B testing or whatever, but always tying the concept to a commercial service just obfuscates what it actually is and does.

Re: Feature flags in a CI pipeline

#15
I've been thinking about a way to add feature flags to my project's CI pipeline. What makes me afraid of adding such thing is reliability: if someone breaks the feature flag service, now the CI pipeline is broken too. I could just give them a default value if the communication fails, but then we will have builds that are skipping checks potentially due to transient errors. I would be very curious to know the guidance for using this tool in my case.

Re: Feature flags in a CI pipeline

#16

I've been thinking about a way to add feature flags to my project's CI pipeline. What makes me afraid of adding such thing is reliability: if someone breaks the feature flag service, now the CI pipeline is broken too. I could just give them a default value if the communication fails, but then we will have builds that are skipping checks potentially due to transient errors. I would be very curious to know the guidance…

I feel like you can just add a table to your DB and be done with it for most systems. I feel like people with large enough systems where “stick it into the DB” is not reasonable also should probably have a bespoke feature flag system.

Re: Feature flags in a CI pipeline

#17
We add a "branch config" JSON file for each branch, which is later merged with some CUE. We put metadata and "feature flag" like values in there. This full config value is passed to all subjobs and makes it easy to change and grow our CI.

The main problem I see with real feature flags in CI is that it can make reproducing a build nearly impossible. What happens if someone flips a flag in the middle of a build? (where that flag value is checked in multiple places, so the flip happens in between the checks)

Re: Feature flags in a CI pipeline

#18

We add a "branch config" JSON file for each branch, which is later merged with some CUE. We put metadata and "feature flag" like values in there. This full config value is passed to all subjobs and makes it easy to change and grow our CI. The main problem I see with real feature flags in CI is that it can make reproducing a build nearly impossible. What happens if someone flips a flag in the middle of a build? (where…

Would logging feature flag values be enough? Then you could look at stdout to see what the value was during each job

Re: Feature flags in a CI pipeline

#19
post #14

Earlier quoted context omitted.

Sure you can use the environment variables, and if you have simple uses cases this is probably the best way to go. In this case, we're using LaunchDarkly's targeting to serve different variations to groups of users, and also using the in built monitoring LaunchDarkly provides to see how often each is requested etc.

It makes sense if you already use a service like LaunchDarkly but adding a whole new service like LaunchDarkly to your stack seems overkill for most people using feature flags. It took me a long time to understand how to work with feature flags in projects that don't require all the monitoring and overhead these services provide because all videos and articles I could find on the subject always framed them in terms o…

Bespoke feature flag systems can become a quagmire. You might think “we only need a boolean value”, but then you need a number, and then you need a string, and then you need a UI to manage it, and then you have a 2nd service and DB and have to expose feature flags via an API endpoint, and then you realize that getting feature flag values via HTTP calls is causing latency and reliability issues, and then and then and then.

I worked at a company that went far down that slippery slope and now there’s and horribly convoluted bespoke feature flag system that no one enjoys using and no one wants to maintain.

I’ve used LaunchDarkly and it’s really nice. The only downsides I’ve experienced are:

- UI is complicated.

- They have the concept of end users but not end “organizations” (like “set this flag to X for an entire customer organization”). There are hacky ways to do this but they feel icky.

Re: Feature flags in a CI pipeline

#20

We add a "branch config" JSON file for each branch, which is later merged with some CUE. We put metadata and "feature flag" like values in there. This full config value is passed to all subjobs and makes it easy to change and grow our CI. The main problem I see with real feature flags in CI is that it can make reproducing a build nearly impossible. What happens if someone flips a flag in the middle of a build? (where…

Would logging feature flag values be enough? Then you could look at stdout to see what the value was during each job

How would you ensure the same values after pushing a new commit or retriggering a build at the same commit? There are many times where external failures, like NPM returning a 50x for a package, that requires a build to be rerun at the same commit.

It seems impossible to ensure reproducible CI when some of that configuration lives outside of source control. Given the CI files are in source control, you can just change them there, rather than using feature flags. Using feature flags comes with a host of hard problems that you ought to just avoid by not using them.

Post reply on HN