Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenFeature – a vendor-agnostic, community-driven API for feature flagging

github.com

Post reply on HN
Print view View original

21–30 of 67 posts

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#21

Post by adontz »

A little story from personal experience.

Most people think of feature flags as boolean on/off switches, maybe per user on/off switches.

If one is testing shades of colors for a "But Now!" button that may be OK. Regarding more complex tests my experience is that there are not a lot of users who tolerate experiments. Our solution was to represent feature flags as thresholds. We assigned a decimal number [0.0, 1.0) to each user (we called it courage) and a decimal number [0.0, 1.0] to a feature flags (we called it threshold). That way not only we enabled more experimental features for most experiment tolerant users, but these were the same users, so we could observe interaction between experimental features too. Also deploying a feature was as simple as rising it's threshold up to 1.0. User courage was 0.95 initially and could be updated manually. We tried to regenerate it daily based on surveys, but without much success.

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#22

Post by vault »

adontz wrote: post #21

A little story from personal experience. Most people think of feature flags as boolean on/off switches, maybe per user on/off switches. If one is testing shades of colors for a "But Now!" button that may be OK. Regarding more complex tests my experience is that there are not a lot of users who tolerate experiments. Our solution was to represent feature flags as thresholds. We assigned a decimal number [0.0, 1.0) to e…

From your naming, I would have done the opposite :) Start with courage 0.05 and show experiments whenever it is greater than the threshold. To enable a feature for everybody, you lower the threshold to 0.

How did you measure "experiment tolerance"?

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#23

Post by withinboredom »

adontz wrote: post #21

A little story from personal experience. Most people think of feature flags as boolean on/off switches, maybe per user on/off switches. If one is testing shades of colors for a "But Now!" button that may be OK. Regarding more complex tests my experience is that there are not a lot of users who tolerate experiments. Our solution was to represent feature flags as thresholds. We assigned a decimal number [0.0, 1.0) to e…

Interesting. At one place I worked, employees were excluded from experiments (they had to enable the flag personally to see them) by default. At one point, we had so many experiments that literally nobody except employees were using "that version" of the software. Everyone else was using some slightly different version (if you counted each feature as a version), and there were thousands and thousands of versions in total.

We ended up creating just ~100 versions of our app (~100 experiment buckets), and then you could join a bucket. Teams could even reserve sets of buckets for exclusive experimentation purposes. We also ended up reserving a set of buckets that always got the control group.

You've approached it a different way, and probably a more sustainable way. It's interesting. How do you deal with the bias from your 'more courageous' people?

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#24

Post by vasco »

adontz wrote: post #21

A little story from personal experience. Most people think of feature flags as boolean on/off switches, maybe per user on/off switches. If one is testing shades of colors for a "But Now!" button that may be OK. Regarding more complex tests my experience is that there are not a lot of users who tolerate experiments. Our solution was to represent feature flags as thresholds. We assigned a decimal number [0.0, 1.0) to e…

> User courage was 0.95 initially and could be updated manually. We tried to regenerate it daily based on surveys, but without much success.

Based on this ending, the courage bit sounds clever but is misguided. It adds complexity in a whole other variable, yet you have no way of measuring it or even do a good assessment.

I thought you were going to describe how you calculated courage based on the statistical usage of new features vs old features when exposed to them to update courage, meaning people who still keep using the product when it changes have more courage so they see more changes more often. But surveying for courage (or how easy they deal with change) is probably the worse way to assess it.

But even that I don't know what purpose would have because now you destroyed your A/B test by selecting a very specific sub population, so your experiment / feature results won't be good. I'm assuming here a product experimentation approach being used, not just "does it work or not" flags.

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#25

Post by siva7 »

adontz wrote: post #21

A little story from personal experience. Most people think of feature flags as boolean on/off switches, maybe per user on/off switches. If one is testing shades of colors for a "But Now!" button that may be OK. Regarding more complex tests my experience is that there are not a lot of users who tolerate experiments. Our solution was to represent feature flags as thresholds. We assigned a decimal number [0.0, 1.0) to e…

Sounds like an overengineered solution to something that can be solved as simple as with a checkbox "i would like to get access to experimental features" in the UI.

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#26

Post by dangsux »

withinboredom wrote: post #23
adontz wrote: post #21

A little story from personal experience. Most people think of feature flags as boolean on/off switches, maybe per user on/off switches. If one is testing shades of colors for a "But Now!" button that may be OK. Regarding more complex tests my experience is that there are not a lot of users who tolerate experiments. Our solution was to represent feature flags as thresholds. We assigned a decimal number [0.0, 1.0) to e…

Interesting. At one place I worked, employees were excluded from experiments (they had to enable the flag personally to see them) by default. At one point, we had so many experiments that literally nobody except employees were using "that version" of the software. Everyone else was using some slightly different version (if you counted each feature as a version), and there were thousands and thousands of versions in t…

[dead]

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#28

Post by webda2l »

A coding world with more standardization will be better world.

I met this week "Standardized Interface for SQL Database Drivers" https://github.com/halvardssm/stdext/pull/6 by example then https://github.com/WICG/proposals/issues too.

Huge work to get everybody on the same page (About my previous example, it's not well engaged by example https://github.com/nodejs/node/issues/55419), but when done and right done, it's a huge win for developers.

PHP PSR, RFC & co are the way.

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#29

Post by skeeter2020 »

adontz wrote: post #21

A little story from personal experience. Most people think of feature flags as boolean on/off switches, maybe per user on/off switches. If one is testing shades of colors for a "But Now!" button that may be OK. Regarding more complex tests my experience is that there are not a lot of users who tolerate experiments. Our solution was to represent feature flags as thresholds. We assigned a decimal number [0.0, 1.0) to e…

This seems really complex, specifically in the area where I find product, CS & marketing least likely to want it: targeting and controlling their audience. Sounds like a cool thought experiment, fun and challening to implement and not really practical or useful.

If you have a huge userbase and deploy very frequently FFs are great for experiments, but for the rest of us they're primarily a way to decouple deploys from releases. They help with the disconnect between "Marketing wants to make every release a big event; Engineering wants to make it a non-event". I also find treating FFs as different from client toggles is very important for lifecycle management and proper use.

More than the binary nature I think the bigger challenge is FFs are almost always viewed as a one-way path "Off->On->Out" but what if you need to turn them off and then back on again? It can be very hard to do properly if a feature is more than UI, that might cause data to be created or updated that the old code then clobbers, or issues between subsystems, like microservices that aren't as "pure" as you thought.

Re: OpenFeature – a vendor-agnostic, community-driven API for feature flagging

#30

Post by skeeter2020 »

fire_lake wrote: post #13
ygouzerh wrote: post #12

Earlier quoted context omitted.

Do you mean feature flags? This enable you to change the configuration at the runtime. Ex: A/B Testing and changing a behavior for a subset of users, disable feature when you want it (particularly useful when you are in Trunk Based Development and don't want to deploy a beta feature to everyone for example).

No I mean an entire framework and set of software components for doing feature flags.

If you have a single database than maybe you can (and should?) just start with a basic, single table approach, but as you grow in size and complexity FF management can become a challenge, with reporting gaps and feature release management. I usually see two charateristics with the former approach: growth in the # of FFs over time and a messy Excel report for what they are, do and if anyone still hits the old code. This might be fine for a while, or forever, but often gets painful.
Post reply on HN