Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing on production

marcochiappetta.medium.com

Post reply on HN
Print view View original

41–50 of 101 posts

Re: Testing on production

#41

Post by smallerfish »

mwint wrote: post #22
civilized wrote: post #20

I have a dumb question as a non-SWE who is curious about software engineering. I've heard "feature flags" are popular these days, and I understand that that's where you commit code for a new way of doing things but hide it behind a flag so you don't have to turn it on right away. Now, if I want to test in prod, couldn't I just make the flag for my new feature turn on if I log in on a special developer test account? A…

Feature flags sound great, but a company I’ve been consulting for has been using them to their own detriment. Seems like many bugs are due to a (production!) user not having the right combinations of flags enabled. There ends up being code to deal with what happens when various combinations of flags are on/off, and that code doesn’t get tested much. And teams spend a lot of time just removing flags. This isn’t a safe…

> Seems like many bugs are due to a (production!) user not having the right combinations of flags enabled.

In my experience, feature flags work best if you aim to remove them as quickly as possible. They can be useful to allow continual deployment, and even for limited beta programs, but if you're using them to enable mature features for the whole customer base, they're no longer feature flags.

Re: Testing on production

#42

Post by Shrezzing »

I enjoyed the entire article except this part:

> Unfortunately there is no easy way to distinguish between people who are good and need a paycheck from people who just need a paycheck. But you sure as hell don’t want the latter in your team.

If you can't tell them apart, then the distinction is unimportant. So if among the group of people who need paychecks, good is indistinguishable from non-good, the comment serves no purpose other than needless elitism.

Re: Testing on production

#43

Post by gwright »

routerl wrote: post #15
anoy8888 wrote: post #5

I don’t understand why people redefine words just make their point. It can be confusing at best and at worst change the me meaning of words when it becomes viral. “Smart” people means smart people. It shouldn’t be used to mean junior dev who are trying to hard to prove themselves and over engineer or choose the wrong approach. So many words have changed their original meaning because someone decides to write a viral…

Oh right, the "original meaning" of "smart"... so you must mean "pain or ache"? I really don't see how that's relevant to the article. Words change, they always have, they always will. Get over it. And anyway, the article's usage is consistent with the well-established phrase "smart guy", within which the word "smart" carries a sarcastic and derisive tone.

> Words change, they always have, they always will. Get over it.

While this is true, I think it is helpful to communication to resist changes to language. This isn't the same thing as opposing change entirely, but language needs to have a certain stability and common understanding to maximize its usefulness.

Re: Testing on production

#44

Post by solatic »

Shrezzing wrote: post #42

I enjoyed the entire article except this part: > Unfortunately there is no easy way to distinguish between people who are good and need a paycheck from people who just need a paycheck. But you sure as hell don’t want the latter in your team. If you can't tell them apart, then the distinction is unimportant. So if among the group of people who need paychecks, good is indistinguishable from non-good, the comment serves…

It's implied that it isn't easy to distinguish them during interviews. After they join your team, it's very easy to distinguish them.

Re: Testing on production

#45

Post by JohnBooty »

xnorswap wrote: post #27
mwint wrote: post #22

Earlier quoted context omitted.

Feature flags sound great, but a company I’ve been consulting for has been using them to their own detriment. Seems like many bugs are due to a (production!) user not having the right combinations of flags enabled. There ends up being code to deal with what happens when various combinations of flags are on/off, and that code doesn’t get tested much. And teams spend a lot of time just removing flags. This isn’t a safe…

I'm going to go further and say that Feature Flags are a nightmare and should be avoided. Because instead of just being used to stage roll-out, they get used to configure different environments for different customers. You not only waste time with "Remove feature flag X" stories if all customers end up with the feature, you also slow down the response time of some categories of bugs, because you end up having to stop…

They can be very very very nice if you have a lengthy (or perhaps just unpredictable) build/deploy process. And/or if you have lots of teams working independently on the same monolith.

Suppose you have daily production builds. You are rolling out Feature XYZ. You would like to enable it in prod, but you would like to monitor it closely and may need to turn it off again. Feature flags allow that.

Ultimately what's being achieved is a decoupling of configuration and deployment.

    Maybe I'm ranting about "misuse of feature flags", but 
    I don't like to pontificate about how things ought to be, 
    but how in my experience they actually are.
Similarly, I might just be making excuses for bad build/deploy processes. =)

At my last job we relied heavily on feature flags via Launch Darkly. I will admit: it was somewhat of a band-aid for the fact that our build process was way too slow and flaky, and that we had too many teams working on an overstuffed monolith.

Re: Testing on production

#46

Post by chpmrc »

Welp! For some reason someone at HN decided to change the title and bump this down to the 11th position (atm). Not sure what I did wrong here but it feels pretty crappy...

@dang any chance you could help here? :(

Re: Testing on production

#47

Post by JohnBooty »

chpmrc wrote: post #9
anoy8888 wrote: post #5

I don’t understand why people redefine words just make their point. It can be confusing at best and at worst change the me meaning of words when it becomes viral. “Smart” people means smart people. It shouldn’t be used to mean junior dev who are trying to hard to prove themselves and over engineer or choose the wrong approach. So many words have changed their original meaning because someone decides to write a viral…

Oh what an accomplishment it would be, to be able to change the meaning of the word "smart" with a single article! (Don't take it too seriously, like I said this is mostly a brain dump, I'm sure there's a lot of stuff that can be improved)

I like your usage of "smart" in the article.

I see this challenge a lot in the industry. The young engineers truly are smart, even brilliant, but lack wisdom and experience.

Re: Testing on production

#48

Post by chrisfowles »

gwright wrote: post #43
routerl wrote: post #15

Earlier quoted context omitted.

Oh right, the "original meaning" of "smart"... so you must mean "pain or ache"? I really don't see how that's relevant to the article. Words change, they always have, they always will. Get over it. And anyway, the article's usage is consistent with the well-established phrase "smart guy", within which the word "smart" carries a sarcastic and derisive tone.

> Words change, they always have, they always will. Get over it. While this is true, I think it is helpful to communication to resist changes to language. This isn't the same thing as opposing change entirely, but language needs to have a certain stability and common understanding to maximize its usefulness.

One of the reasons that English has been so successful as a global business language is its ability to be flexible and splodgable and still make sense.

Re: Testing on production

#49

Post by pravus »

contravariant wrote: post #18

An interesting perspective I once heard from an information security expert is that there's a difference between risks and 'things that can go wrong'. Something is only an actual risk if it hurts the bottom-line. In particular quite a few things that can go wrong don't carry that much risk, and conversely something that is hard but not impossible to go wrong may carry huge amounts of risk. The trick with this perspec…

When I was doing disaster recovery we measured three things: 1) the likelihood of a particular scenario, 2) the magnitude of the impact if the scenario unfolded, and 3) the level of effort expected to recover. It was the combination of these three things that prioritized decision making.

We were working towards a business continuity plan which can include incidents like your main office and operations being destroyed and having to quickly relocate all services to 3rd-parties using off-site backups with minimal staff. While that was a worst-case, a primary focus was just getting a notification site up and running in the event of a network outage because that was vastly more frequent and had high visibility.

It was a very interesting project and I learned quite a bit about how to think comprehensively about the solutions we provided.

Post reply on HN