Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uber open-sources tool to automatically clean up stale code

infoq.com

Post reply on HN
Print view View original

51–60 of 70 posts

Re: Uber open-sources tool to automatically clean up stale code

#51

Post by erik_seaberg »

wahlrus wrote: post #21
erik_seaberg wrote: post #19

Earlier quoted context omitted.

The flag becomes a percentage. The feature isn't always off or always on, it's on for a small fraction of workload, and then that fraction grows as you demonstrate it's safe. Ideally you want metrics that tell you whether the experiment is worse or better than the control group.

What you're describing is a common way of using feature flags—except the percentage part comes from how you manage the servers running the binary with config. I.e. on day one, 5% of servers in cluster get True for the flag value. The double the percentage every day until 100% or otherwise rollback if it's a bad cut.

Partitioning by instance works if you have enough instances to avoid big increases, but at that point you can just deploy known-good and new-feature builds. Runtime checking helps if it's a lot faster than rolling back to the known-good build, or if you're doing concurrent experiments (you may not have enough instances to try every possible combination).

Re: Uber open-sources tool to automatically clean up stale code

#52

Post by creyes »

dahfizz wrote: post #42
creyes wrote: post #39

Is this drastically different than https://unused.codes/ ?

From a quick reading, yes. Your tool identifies code that is never run. The featured tool identifies code that never _will be_ run if we make X change to the API.

oh interesting, tbanks for the clarification!

Re: Uber open-sources tool to automatically clean up stale code

#53

Post by mkr-plse »

creyes wrote: post #39

Is this drastically different than https://unused.codes/ ?

For Piranha: a) Code related to stale flags is deleted b) Determination of staleness is based on status of the feature. c) Patch is created automatically and in a majority of the cases, compiles and passes tests.

Based on my understanding of unused codes, a) unused codes is used to delete deadcode independent of features b) determining the deadcode is based on their usage in tests c) unclear whether the code is flagged for deletion or a patch created.

Re: Uber open-sources tool to automatically clean up stale code

#54

Post by jerzyt »

read_if_gay_ wrote: post #14
laydn wrote: post #10

I find it strange that a company which provides taxi hailing/routing technology felt the need to write a code cleanup tool.

You can make most of the Big N sound silly like this. Amazon? Basically a warehouse. Netflix, YouTube? They just stream video. Facebook, Twitter? CRUD websites. It all sounds like anyone can put something like that together, but try scaling up to billions of users.

Exactly. And each one of these companies had built the business around a key idea. I interview a lot of candidates for data scientists, and one of my favorite questions is what makes one of these companies what they are.

Re: Uber open-sources tool to automatically clean up stale code

#55

Post by evv »

oars wrote: post #48
jdxcode wrote: post #47

Earlier quoted context omitted.

it stands for "open-source software". It's never abbreviated as "OS" for obvious reasons.

Thanks, I guess I'll add "Open Source Software" to my list of unusual yet widely understood verbs. The English language confuses a native German speaker once again!

To be fair, I am a native english speaker and this is the first time I've seen it used as a verb. "OSS" generally acts as a noun. In this instance I would have used "open source" as the verb of what you could do to your proprietary Software, after which it would become OSS.

Re: Uber open-sources tool to automatically clean up stale code

#56

Post by RSZC »

erik_seaberg wrote: post #51
wahlrus wrote: post #21

Earlier quoted context omitted.

What you're describing is a common way of using feature flags—except the percentage part comes from how you manage the servers running the binary with config. I.e. on day one, 5% of servers in cluster get True for the flag value. The double the percentage every day until 100% or otherwise rollback if it's a bad cut.

Partitioning by instance works if you have enough instances to avoid big increases, but at that point you can just deploy known-good and new-feature builds. Runtime checking helps if it's a lot faster than rolling back to the known-good build, or if you're doing concurrent experiments (you may not have enough instances to try every possible combination).

Having done it both ways this would not be my recommendation unless it's necessary - I think it adds a fair amount of complexity.

Some considerations: you'll need some sort of storage mechanism for these flags - is that a centralized configuration service for all your services? Maybe just a table in your database? But database / network calls are expensive to be adding to every single time your code executes the path in question - maybe it makes sense for your service to cache these values locally...but then doesn't that lose part of the purpose of 'fast rollbacks'? Maybe instead of a local cache you spin up a redis instance - but what if this goes down? Will all your instances default to the same value? Etc, etc, etc.

I'm not saying this approach is bad, only that it has complexity, and I find I generally can get away without it.

Re: Uber open-sources tool to automatically clean up stale code

#57

Post by zacksinclair »

burtonator wrote: post #24

We just wrote one that does something similar for Typescript if anyone wants us to OSS it... The idea is that any stale code causes a HUGE amount of headache and removing it can be a life safer.

I'd run something like that over my TS repos for sure!

Re: Uber open-sources tool to automatically clean up stale code

#58

Post by ygra »

sushid wrote: post #49
oars wrote: post #48

Earlier quoted context omitted.

Thanks, I guess I'll add "Open Source Software" to my list of unusual yet widely understood verbs. The English language confuses a native German speaker once again!

I wouldn't call it unusual although I hear FOSS more often.

As a noun I'd agree on it not being unusual, but as a verb it's weird.

Re: Uber open-sources tool to automatically clean up stale code

#59

Post by tomkinson »

mkr-plse wrote: post #50
burtonator wrote: post #24

We just wrote one that does something similar for Typescript if anyone wants us to OSS it... The idea is that any stale code causes a HUGE amount of headache and removing it can be a life safer.

Piranha author here -- Will you be willing to contribute it to Piranha?

@burtonator yes curious what you have or maybe it's a better fit under Piranha?

Re: Uber open-sources tool to automatically clean up stale code

#60

Post by Twirrim »

wahlrus wrote: post #21
erik_seaberg wrote: post #19

Earlier quoted context omitted.

The flag becomes a percentage. The feature isn't always off or always on, it's on for a small fraction of workload, and then that fraction grows as you demonstrate it's safe. Ideally you want metrics that tell you whether the experiment is worse or better than the control group.

What you're describing is a common way of using feature flags—except the percentage part comes from how you manage the servers running the binary with config. I.e. on day one, 5% of servers in cluster get True for the flag value. The double the percentage every day until 100% or otherwise rollback if it's a bad cut.

Then rolling forwards and backwards is a whole deployment away, or mucking about with infrastructure, vs tweaking a percentage flag somewhere.

If you want to get fancy with changes (and I've seen it done) you have something else capable of controlling that percentage setting that is tied in to your monitoring. Start out low, say 1% of requests hitting the new path. Automatically ramp up over time to full 100%. If you see failures, automatically drop back to 0% until it can be ascertained that the failure didn't come from the new code.

Post reply on HN