Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Feature Flags: Theory vs. Reality

bpapillon.com

Post reply on HN
Print view View original

41–50 of 92 posts

Re: Feature Flags: Theory vs. Reality

#41

Post by no_wizard »

zero_shift wrote: post #4

I've definitely lived with the zombie flags problem. Teams ship experiments that double the size of a piece of code, but never go back to refactor out the unused code branches. In shared codebases this becomes a nightmare of thousands of lines of zombie code and unit tests. This is a social problem as much as a technical one: even if you have LaunchDarkly, DataDog etc making very clear that a flag isn't used, getting…

This is the only way, more or less, to enforce any code standards, whether its refactoring, quality, testing, docs etc.

If it doesn't break the build (or do anything else that stops it from moving forward) there will always be external pressure to get things out "ASAP" despite in most circumstances "ASAP" isn't required. If you can't full on stop whats happening, it becomes exponentially harder to enforce anything.

Re: Feature Flags: Theory vs. Reality

#42

Post by esafak »

adamesque wrote: post #38
SketchySeaBeast wrote: post #33

Earlier quoted context omitted.

> "Test was successful so it's rolling out to all users, minus a 0.5% holdback population for the next 2 years" Man, I couldn't imagine being a user in such a situation. "Oh, I guess I'm just not getting the better functionality?" Even worse if I were a paying customer.

It’s actually usually the paying customers asking via support to be added to the holdback, improved experience or no. This is more true for larger flags that substantially change the experience and may not implement niche or edge-case functionality. Obviously you want to avoid these kinds of tests if possible but it’s not always possible.

Users should not be allowed to select their treatments; it defeats randomization, which is what allows causal inference.

Re: Feature Flags: Theory vs. Reality

#44

Post by no_wizard »

benpapillon wrote: post #5
zellyn wrote: post #3

Modern feature flag tooling (eg. LaunchDarkly) cover most of the uses here. It'll even tell you whether flags are useful or not (if you push evaluation data back upstream).

Good point. It's possible the real issues have more to do with price point/positioning and product UX. My experience with LaunchDarkly has been that a lot of these hygiene-related exist only in their top tier enterprise plans, and even below that point the cost of the tool starts to draw attention. On the product UX side - I've found these tools are designed for engineering/Devops users but (whether by design or not)…

I recommend giving Statsig[0] a try. I was actually surprised at how well they did stuff like this around Feature Flag management.

[0]: https://www.statsig.com/

Re: Feature Flags: Theory vs. Reality

#45

Post by SketchySeaBeast »

adamesque wrote: post #38
SketchySeaBeast wrote: post #33

Earlier quoted context omitted.

> "Test was successful so it's rolling out to all users, minus a 0.5% holdback population for the next 2 years" Man, I couldn't imagine being a user in such a situation. "Oh, I guess I'm just not getting the better functionality?" Even worse if I were a paying customer.

It’s actually usually the paying customers asking via support to be added to the holdback, improved experience or no. This is more true for larger flags that substantially change the experience and may not implement niche or edge-case functionality. Obviously you want to avoid these kinds of tests if possible but it’s not always possible.

Ah, that makes way more sense.

Re: Feature Flags: Theory vs. Reality

#46

Post by daliwali »

At my work, I have a somewhat clever (or idiotic) technical solution to the problems of feature flags: they are actually implemented as feature modules that monkey-patch the base application in runtime.

There are a few benefits: removing features is dead simple, just delete the whole feature module, and there's no conditional branching in the base application.

There are some drawbacks too: the base application must have entry points for the feature modules to overwrite. Usually the default values are no-op or some default behavior. Features also must implement setup and teardown, which can take longer to write than a conditional.

Re: Feature Flags: Theory vs. Reality

#47

Post by sb8244 »

zero_shift wrote: post #4

I've definitely lived with the zombie flags problem. Teams ship experiments that double the size of a piece of code, but never go back to refactor out the unused code branches. In shared codebases this becomes a nightmare of thousands of lines of zombie code and unit tests. This is a social problem as much as a technical one: even if you have LaunchDarkly, DataDog etc making very clear that a flag isn't used, getting…

The big issue I ran into with zombie flags is that new features were always prioritized over cleanup. Engineering could "fight for time" to get things done, but there were always other priorities that needed to be addressed.

No tool you have will solve that, whomever owns the product team time allocation needs to be onboard with the idea of cleaning up old code.

Re: Feature Flags: Theory vs. Reality

#48

Post by sb8244 »

My story of when bad feature flag hygiene resulted in a real technical problem is when our Redis kicked over one day. We had good monitoring so it was easy to identify the problem: network was saturated at 1 GB/s.

I traced the problem back to the fact that we had 100+ feature flags that were fully launched, but still loaded into the backend when "all feature flags" were loaded for a team. The way this was implemented returned all team IDs that had the feature flag, and the way this was done had some flags with multiple thousand IDs in them.

So 100+ flags, many with 2000+ int entries.

We ended up quickly shipping some code to mark features GA, so they wouldn't be loaded from Redis. Cut usage by 99% instantly.

Re: Feature Flags: Theory vs. Reality

#49

Post by GeorgeMac »

benpapillon wrote: post #6
GeorgeMac wrote: post #2

We're attempting to address some of these problems at https://www.flipt.io/gitops . Having your flags defined as configuration and committed to repository opens up a range of possibilities in terms of static analysis. Additionaly, we've got a prototype static analysis tool to finding calls to our feature flag clients in both Go and Rust too.

Hadn't seen this, looks very cool! The static analysis piece seems difficult, but even considering that I've been a little surprised not to see more attempts.

Yeah, it is surprising not to see more attempts out there! GitHub's TreeSitter sits at the core of our attempt. Definitely feels like the right tool with the right potential. We plan to open source it sometime soon.

Re: Feature Flags: Theory vs. Reality

#50

Post by djbusby »

jasdeepg1987 wrote: post #23

as a pm, there's a whole set of jobs that occur post-rollout that have often been poorly handled at companies i've been at. those include packaging, customer operations like allow-listing long-lived features for certain companies, optimization of bundles, etc. when we've built our own homegrown system, it's opaque and often neglected. when we've used feature flag tools, we co-opt them to do things they're not meant t…

Can you describe the different shape? What does it turn into? Once live I clean up the FF. But, may introduce new ones as the now-live feature gets tweaked.
Post reply on HN