Live data from Hacker News

Feature Flags: Theory vs. Reality

bpapillon.com

11–20 of 92 posts

Re: Feature Flags: Theory vs. Reality

#11
My biggest problem with 3rd party feature flag setups is that I have high expectations for them and it is technically difficult to meet all of them:

- local/static access: it should not have to call out to a 3rd party server to get basic runtime config

- unused-flag detection: flags should have three reported states: never used, recently used, not recently used. These will be different from the user-controlled states of active, inactive, etc.

- sticky a/b testing: should follow the logged in user until the flag is removed

- integration with logger: I should be able to use it with my logger out of the box to report only relevant feature flags. Alternatively can provide a packed value of all relevant flags, would probably have to do flag state versioning.

- integration with linter: should warn me if flag has not recently been used or I used a flag in the code that is not in our database (alternatively, will upsert the flag automatically if it doesn’t exist)

- hashed flag names on frontend build: prevent the leakage of information, not a perfect solution, but I would want to avoid writing “top-secret-feature” where we can.

I fully acknowledge that a lot of solutions come close, but I haven’t looked at the current state of things in the last few years so it may have improved.

Re: Feature Flags: Theory vs. Reality

#12
The zombie flags are a huge problem. Management is always pushing for feature completion and its done - behind a feature flag. The complication is now they never want to allow time to remove all the dead code paths later, which leaves you dependent on all sorts of potential things, imports, libraries maybe even connections. One day they inevitably find out something is "still in prod" and they get curious and don't understand why it's still there. Well, feature flags require more TCO, period. They don't want to give you more time though.

Re: Feature Flags: Theory vs. Reality

#13
post #11

My biggest problem with 3rd party feature flag setups is that I have high expectations for them and it is technically difficult to meet all of them: - local/static access: it should not have to call out to a 3rd party server to get basic runtime config - unused-flag detection: flags should have three reported states: never used, recently used, not recently used. These will be different from the user-controlled states…

I feel your pain.

This is what drove me to build an open source solution: https://featurevisor.com/

Re: Feature Flags: Theory vs. Reality

#14
post #9

I've definitely lived with the zombie flags problem. Teams ship experiments that double the size of a piece of code, but never go back to refactor out the unused code branches. In shared codebases this becomes a nightmare of thousands of lines of zombie code and unit tests. This is a social problem as much as a technical one: even if you have LaunchDarkly, DataDog etc making very clear that a flag isn't used, getting…

A softer solution is to name and shame with periodic "leaderboard" emails to the org showing how many experiments each team has failed to clean.

But isn't spam like that exactly the sort that will ultimately get completely ignored? I probably get a dozen or so such barely-relevant internal emails a day where I work, and have learned how to recognize them by the sender and subject line, and ignore them.

A "leaderboard" email would 100% be one that I ignore as a time-waster.

Re: Feature Flags: Theory vs. Reality

#15
Useful post outlining a lot of common pain points I have experienced myself in my career.

One of the reasons I went for an open source solution ( https://featurevisor.com ) that's Git based, and every change is done via Pull Requests.

Building blocks:

- Attributes for conditions: https://featurevisor.com/docs/attributes/

- Segments for targeting users: https://featurevisor.com/docs/segments/

- Features with variations and rules: https://featurevisor.com/docs/features/

Process:

- Merge PRs

- Trigger CI/CD pipeline: https://featurevisor.com/docs/deployment/

- Consume with SDK: https://featurevisor.com/docs/sdks/

Use cases:

- User entitlements: https://featurevisor.com/docs/use-cases/entitlements/

- Testing in production: https://featurevisor.com/docs/use-cases/testing-in-productio...

- A/B testing & experimentation: https://featurevisor.com/docs/use-cases/experiments/

- Remote configuration: https://featurevisor.com/docs/use-cases/remote-configuration...

You can also generate types as a package for compile-time safety:

- Code generation: https://featurevisor.com/docs/code-generation/

The post and the comments here give me more ideas on how to improve it with more features now.

Re: Feature Flags: Theory vs. Reality

#16
post #14
post #9

Earlier quoted context omitted.

A softer solution is to name and shame with periodic "leaderboard" emails to the org showing how many experiments each team has failed to clean.

But isn't spam like that exactly the sort that will ultimately get completely ignored? I probably get a dozen or so such barely-relevant internal emails a day where I work, and have learned how to recognize them by the sender and subject line, and ignore them. A "leaderboard" email would 100% be one that I ignore as a time-waster.

No because the whole org sees it and the org head can tell your team's manager to get the house in order. We did this at my last company, albeit with migrations rather than feature flags. Same idea.

I believe I read about it in a book (perhaps Software Engineering at Google) in the context of test coverage; using a leaderboard for gamification.

Re: Feature Flags: Theory vs. Reality

#17
I work more in the firmware space, so my experience with feature toggles is always with half-baked tooling and limited ability to change deployed products. We do use continuous development within the organization, so there is still a lot of applicability, but it's always interesting to see the way similar problems get addressed in a higher-level and more online environment.

That said, I'm surprised this article doesn't mention the two words that always come to my mind when I see toggles: combinatorial explosion. Several times I've worked on projects that went way too toggle-happy and decided that new functionality should be split into indefinite life "features". Just in case the company someday wants to sell a model without that feature. Of course, when an old toggle finally gets turned off a year later, you realize that it crashes the system because several other features kind of half depend on them.

Re: Feature Flags: Theory vs. Reality

#18
post #11

My biggest problem with 3rd party feature flag setups is that I have high expectations for them and it is technically difficult to meet all of them: - local/static access: it should not have to call out to a 3rd party server to get basic runtime config - unused-flag detection: flags should have three reported states: never used, recently used, not recently used. These will be different from the user-controlled states…

I think a lot of the solutions come close but don't quite get there. It seems like there's kind of a divide between the open source solutions that are probably more sensitive to the day-to-day pain points of developers and the bigger managed service players that seem to be optimizing for contract size.

Hadn't thought of the frontend build hashing idea - like that a lot

Re: Feature Flags: Theory vs. Reality

#19
One thing I see missing in this article is another huge cost to these things.

What happens when your homegrown feature flag microservice (because why pay for a hard cost when you can have the soft cost of making your own) goes down, even temporarily.

Sane defaults at code review time, before launch aren't always the sane defaults after a feature has fully launched, or nearly fully launched.

I've seen more than a few egregious outages due to a feature flagging tool being down and taking the user experience back a year or two.

Re: Feature Flags: Theory vs. Reality

#20
post #16
post #14

Earlier quoted context omitted.

But isn't spam like that exactly the sort that will ultimately get completely ignored? I probably get a dozen or so such barely-relevant internal emails a day where I work, and have learned how to recognize them by the sender and subject line, and ignore them. A "leaderboard" email would 100% be one that I ignore as a time-waster.

No because the whole org sees it and the org head can tell your team's manager to get the house in order. We did this at my last company, albeit with migrations rather than feature flags. Same idea. I believe I read about it in a book (perhaps Software Engineering at Google ) in the context of test coverage; using a leaderboard for gamification.

Ahh, so the real target for such an email is management rather than the rank and file? That makes more sense. But surely, it would be better to send email just to those people and put the data up on the company intranet for those devs who are curious.
Post reply on HN