Live data from Hacker News

Feature Flags: Theory vs. Reality

bpapillon.com

21–30 of 92 posts

Re: Feature Flags: Theory vs. Reality

#21

I've definitely lived with the zombie flags problem. Teams ship experiments that double the size of a piece of code, but never go back to refactor out the unused code branches. In shared codebases this becomes a nightmare of thousands of lines of zombie code and unit tests. This is a social problem as much as a technical one: even if you have LaunchDarkly, DataDog etc making very clear that a flag isn't used, getting…

> getting a team to prioritise cleanup is difficult

This is the limitation that breaks every development practice people come up with.

That idea of formalizing the cleanup and requiring it for deployment is very interesting. It may be possible to extend it to other contexts.

Re: Feature Flags: Theory vs. Reality

#22
post #20
post #16

Earlier quoted context omitted.

No because the whole org sees it and the org head can tell your team's manager to get the house in order. We did this at my last company, albeit with migrations rather than feature flags. Same idea. I believe I read about it in a book (perhaps Software Engineering at Google ) in the context of test coverage; using a leaderboard for gamification.

Ahh, so the real target for such an email is management rather than the rank and file? That makes more sense. But surely, it would be better to send email just to those people and put the data up on the company intranet for those devs who are curious.

You can but transparency is a good thing, and if you see it you can fix it before your manager chides you.

Re: Feature Flags: Theory vs. Reality

#23
as a pm, there's a whole set of jobs that occur post-rollout that have often been poorly handled at companies i've been at. those include packaging, customer operations like allow-listing long-lived features for certain companies, optimization of bundles, etc.

when we've built our own homegrown system, it's opaque and often neglected. when we've used feature flag tools, we co-opt them to do things they're not meant to support (e.g. persistent toggles in admin panels) so end up with complexity in the code and in operational processes around it.

agree wholeheartedly with points in this article ... there are issues with how we manage flags generally, but we also bias towards assuming that once a feature is live, we can and should move on -- the feature is now persistent, part of a package, and it won't change frequently or ever.

the reality is the feature lifecycle takes on a very different shape, and, at least in my experience, current FM tooling isn't built to accommodate that.

Re: Feature Flags: Theory vs. Reality

#25

I work more in the firmware space, so my experience with feature toggles is always with half-baked tooling and limited ability to change deployed products. We do use continuous development within the organization, so there is still a lot of applicability, but it's always interesting to see the way similar problems get addressed in a higher-level and more online environment. That said, I'm surprised this article doesn…

> decided that new functionality should be split into indefinite life "features"

Yeah, once you do that you have settings, and not feature flags anymore.

Adjustable settings do come with a high risk of combinatorial explosion. Ideally, you separate the system functionality to control this problem, but that's not always possible.

Re: Feature Flags: Theory vs. Reality

#26

I've definitely lived with the zombie flags problem. Teams ship experiments that double the size of a piece of code, but never go back to refactor out the unused code branches. In shared codebases this becomes a nightmare of thousands of lines of zombie code and unit tests. This is a social problem as much as a technical one: even if you have LaunchDarkly, DataDog etc making very clear that a flag isn't used, getting…

just went through a massive layoff and reams of flags in LD no one understands, extra fun! best part LD is so expensive we have to share hot seats

Re: Feature Flags: Theory vs. Reality

#27
post #22
post #20

Earlier quoted context omitted.

Ahh, so the real target for such an email is management rather than the rank and file? That makes more sense. But surely, it would be better to send email just to those people and put the data up on the company intranet for those devs who are curious.

You can but transparency is a good thing, and if you see it you can fix it before your manager chides you.

It assumes management gives a rat's ass about it.

In the original comment:

> Especially if their PM leaned on engineers to make the experiment "quick n dirty" and therefore hard to clean up.

The PM then just needs to say that they don't have the time to cleanup because they have shit to do, and there's folks out there who pretty much live for this, and are usually well seen by management because they're management's executioner: no matter how shit the idea and execution, they'll get it rammed in.

Re: Feature Flags: Theory vs. Reality

#28
I've got a few friends that work at LaunchDarkly, and from what I can tell, they've got a very good handle on the challenge. Better than the equivalent vendors in my business, anyway. I've had some great talks with the LD people, even though, strictly speaking, I don't get my paychecks from programming, per se.

What brought me into the talks was that the feature flag problem is a similar scope to the central one faced by CCSs (component content systems). By definition, CCS requires the content equivalent of feature flags, implemented in a variety of ways, depending on . . lots of things. That problem is this: both transclusion and conditionals necessarily couples the content to the business or product architecture. Ergo, when the product architecture goes bananas, so does your content system, and you find yourself with documents that aren't meaningful in a linguistic sense, or which just break the processor. This occurs in the content context because the natural language of a unified document is replaced in a CCS with the product or business architecture; how is a document chunked, what business needs do the conditions satisfy, at what support level are document deliverables composed. In a code context, the constructed syntax of the programming language is getting chopped by the conditionals driven from the business side; there's even more variance here regarding how code interacts with business.

So not the same problem, but the same class of problem: regular rules that have to integrate with non-regular, non-linguistic business rules.

I have a tiny chip on my shoulder regarding CCS systems, because I have seen so many years flushed down the "re-use craze" by businesses that had zero business trying to re-use anything. Feature flags are somewhat in the same bucket - a lot of things that a business wants to use flags for should really, really, really be built into the code or abstracted away - but of course a programming language has far richer ways to deal with bad abstractions than a markup language does. Which of course can be a double edged sword.

Re: Feature Flags: Theory vs. Reality

#29
These are real problems but not insurmountable. I think the author does an excellent job of laying out the problem and has pretty decent solutions in mind.

I caution strongly against the proposed solution to fail CI if zombie flags are detected. CI should ONLY fail if there are changes in the branch that cause the failure. Detecting zombie flags (eg, this branch contains a flag which has been turned on and untouched for 90 days) is setting a CI time bomb. Find another way to alert developers of the zombie instead of failing good code at CI time.

Re: Feature Flags: Theory vs. Reality

#30

I've definitely lived with the zombie flags problem. Teams ship experiments that double the size of a piece of code, but never go back to refactor out the unused code branches. In shared codebases this becomes a nightmare of thousands of lines of zombie code and unit tests. This is a social problem as much as a technical one: even if you have LaunchDarkly, DataDog etc making very clear that a flag isn't used, getting…

The tricky requirement that ends up existing and torpedoing attempts to clean up feature flags is a requirement for long-term holdback.

e.g. "Test was successful so it's rolling out to all users, minus a 0.5% holdback population for the next 2 years"

This then forces the team to maintain the two paths for the long-term, ensuring the team might get re-orged / re-prioritize their projects sometime a year later making the cleanup really hard to eventually enforce.

Post reply on HN