31–40 of 50 posts
Earlier quoted context omitted.
That's why he said apocryphal..
Which means they are of doubtful origin. There is no doubt that this particular story is a complete fabrication
Earlier quoted context omitted.
Piranha co-author here. I'd say you trust it a bit more than we do, then ;) As the paper and blog post mention, we use Piranha to create diffs/PRs against the original author of the code. Developers are expected to code-review the changes before they are ever landed. As used within Uber, Piranha won't ever auto-land code deletions. That still saves the time of: a) remembering that the flag is stale and must be remove…
Follow up question: In general, when a diff is generated, is the standard process to just do a manual code-review and CI, or do you also test it manually to check for unexpected side-effects?
Earlier quoted context omitted.
this eliminates the ability to do a revert though if things don't work as expected at %100
I'm not doubting it's impossible, but I can't come up with any scenarios where something would work perfectly at 95% but then break at 100%. What could be some examples of things that can break like that?
Earlier quoted context omitted.
Can you expand a bit on what you mean here? How would you introduce feature flags in a way that doesn't leave any debt behind in contrast to their approach? Or are you saying that feature flags themselves are inheritly debt-generating (which I think they acknowledge, but are taking it on prudently & deliberately) — in which case, what would you do instead?
Actually what I meant was this tool is kind of a after thought of bad feature flag life cycle. When you set a feature flag, normally you should put a deadline on it, after this deadline, you should either choose to keep it, or remove it or you can extend the deadline. But seems like they accumulated a lot of flags, which stayed stale over time, and went unchecked and lack of feedback created the debt. Also unrelated…
This is actually a feature we did identify as important to have in order to increase Piranha's effectiveness, and it is being added to our internal flag tracking, but it doesn't negate the need for the tool. An expiration date makes it easier for the tool to know when to run on a given feature (rather than using heuristics based on % rollout and days without changing the experiment), it still means that Piranha: a) reduces manual effort by auto-generating the (candidate) removal patch, and b) acts as the reminder portion for the expiration, so it's more actionable than just adding a task.
The thing to note is that, even on a steady state where flags are removed as soon as they go stale, enough new experiments are being created every day that reducing the time spent cleaning them up is valuable.
Also, you definitely don't want to block someone from fixing a crash because they have pending expired flags, so all you can really do with any expiration policy is to remind them. With Piranha, you are reminding them and reducing the friction to solve the issue. After all, the diff is right there for them to review and click 'land' on.
As for the hardcoded flag value in your example above? What does that accomplish? It looks to me like you'd only be shipping dead code, since there is no runtime way of re-enabling the `RIDES_NEW_FEATURE` behavior (which is the main difference between "rolled 100%" vs "Piranha-removed"). It also makes harder to remove the related code later, since the semantic information about it being part of the feature is lost. If it's just about having the old code available, then version control does that already, no? What am I missing?
Earlier quoted context omitted.
Follow up question: In general, when a diff is generated, is the standard process to just do a manual code-review and CI, or do you also test it manually to check for unexpected side-effects?
Manual code review and CI only. In our experience of deleting more than 2.5K flags, testing would have helped in one case but then the code was not tested well when the flag was introduced.
Earlier quoted context omitted.
Actually what I meant was this tool is kind of a after thought of bad feature flag life cycle. When you set a feature flag, normally you should put a deadline on it, after this deadline, you should either choose to keep it, or remove it or you can extend the deadline. But seems like they accumulated a lot of flags, which stayed stale over time, and went unchecked and lack of feedback created the debt. Also unrelated…
> When you set a feature flag, normally you should put a deadline on it, after this deadline, you should either choose to keep it, or remove it or you can extend the deadline. This is actually a feature we did identify as important to have in order to increase Piranha's effectiveness, and it is being added to our internal flag tracking, but it doesn't negate the need for the tool. An expiration date makes it easier f…
What I was trying to say was basically, without tooling also you can manage the debt from feature flags.
Answer to the RIDES_NEW_FEATURE question is about "you definitely don't want to block someone from fixing a crash because they have pending expired flags" mainly.
When I am disabling a flag, if I set isNewFeatureEnabled to False, basically, I am removing bloat instantly. Then when I have time to review the code I can also remove the dead code. Actually this is fixing the concerns in your blog post about "accidental activation" and "bloat" without waiting developers to fix the code. Piranha can set flag to stale value, then later can send the developer task to fix the dead code.
My flow is little bit more complicated then I replied actually, I have also assets etc related to feature flags. CI pipeline also removing non-used assets for that flag when it is stale. So basically I have more like: function isNewFeatureEnabled { return isFlagEnabled(flag) && getFlagValue("RIDES_NEW_FEATURE") }
What I am curious on this topic, do you have any kind of conflict detection for your feature flags?
Earlier quoted context omitted.
> When you set a feature flag, normally you should put a deadline on it, after this deadline, you should either choose to keep it, or remove it or you can extend the deadline. This is actually a feature we did identify as important to have in order to increase Piranha's effectiveness, and it is being added to our internal flag tracking, but it doesn't negate the need for the tool. An expiration date makes it easier f…
I am in favor of all automation and I totally agree that the tool is valuable. I am sorry if my comments seemed on other direction. What I was trying to say was basically, without tooling also you can manage the debt from feature flags. Answer to the RIDES_NEW_FEATURE question is about "you definitely don't want to block someone from fixing a crash because they have pending expired flags" mainly. When I am disabling…
Can you elaborate on what you mean by conflict detection? Are you trying to understand how flags are dependent on each other?
One of our internal experimentation systems handles this by capping rollouts to 95%. So the author is forced to clean up their code if they want a full rollout.
How does the system decide which code needs to be deleted?
Earlier quoted context omitted.
I am in favor of all automation and I totally agree that the tool is valuable. I am sorry if my comments seemed on other direction. What I was trying to say was basically, without tooling also you can manage the debt from feature flags. Answer to the RIDES_NEW_FEATURE question is about "you definitely don't want to block someone from fixing a crash because they have pending expired flags" mainly. When I am disabling…
BTW, we also have a tech publication at https://github.com/uber/piranha/blob/master/report.pdf where we discuss some of the design tradeoffs pertaining to the stale flag cleanup problem. Can you elaborate on what you mean by conflict detection? Are you trying to understand how flags are dependent on each other?
Basically sometimes I have some conflicting flags that can introduce bugs. Especially some rarely used flag and a new feature. Basic example, 2 different flags, setting same property to different values on an UI object.
Although more testing coverage probably can help, but I am curious, if you have some automation to detect those cases.
Earlier quoted context omitted.
BTW, we also have a tech publication at https://github.com/uber/piranha/blob/master/report.pdf where we discuss some of the design tradeoffs pertaining to the stale flag cleanup problem. Can you elaborate on what you mean by conflict detection? Are you trying to understand how flags are dependent on each other?
Thanks a lot, I will read it asap. Basically sometimes I have some conflicting flags that can introduce bugs. Especially some rarely used flag and a new feature. Basic example, 2 different flags, setting same property to different values on an UI object. Although more testing coverage probably can help, but I am curious, if you have some automation to detect those cases.