There is a lot of talk of feature flags but that only works when you can realistically delay your feature 2 weeks so that the 'special' set of clients can see it early. What we've seen that has worked is essentially demoing the feature in staging/test to stakeholders at the major/demanding client. They are happy that they aren't taken by surprise and then we provide training materials for major feature/UI changes to their users. Never have had an issue, client is happy and essentially zero technical involvement.
Ask HN: Handling customers that want feature previewing?
51–60 of 68 posts
Re: Ask HN: Handling customers that want feature previewing?
#52I would suggest pre-defined staged rollout based on customer preference.
For example, create three category Alpha, Beta, Stable.
Alpha: Gets new features immediately and gets support within 24hours if something breaks.
Beta: Gets new features Every 45 week, and gets priority support if needed.
Stable: Gets new features quarterly, Mostly they are forced to upgrade from old features which you dont want to support, But they will be essential volume of your revenue.
alternatively, you can give them sandboxed demo read-only(resets every night at 12 o'clock) account for testing.
Re: Ask HN: Handling customers that want feature previewing?
#53Most of the discussion here will be technical, especially around feature flags and friends. That’s fine, but please don’t miss out that this is a clear signal to charge these customers a lot more money.
> a clear signal to charge these customers a lot more money. Interesting, can you explain this a little more? Some of our customers chomp at the bit for particular features, and when we can we give them early access. We do not charge them more for arguably huge amounts of value add to the product. I wonder how to have these types of conversation, internally and with users. I'm a PM but have a direct line to managemen…
I think the ways to approach it are:
- this incurs real costs, so we should charge for this. (The charge doesn't have to be related to the costs! But if you're doing something extra there should be an extra line on the invoice)
- this customer wants something that other customers don't have. There's all sorts of real-world services where you can pay to board the plane first or get to the front of the concert. These may be several times more expensive than the regular service. That lets you phrase it in fawning terms ("you can join our Super Elite Ultra Platinum Preview Fellows") and thereby extract much more money.
Re: Ask HN: Handling customers that want feature previewing?
#54Feature flags is usually how we handle this. We have the flags anyway for developer use, so making them user specific was not that much incremental effort though we have had a few cases where old workflows were not removed when they should have been. We did once have a customer with their own environment with a different version deployed, as they were initially a major client with such demands but after a few years i…
+1 on feature flags. I find them to be a pain, but fussy customers insist "that we need to test it in staging first".
Re: Ask HN: Handling customers that want feature previewing?
#55You need at least 2 sandbox environments: one that's more development-oriented (little to no restrictions on merging), and one that's more release-oriented (as close as possible to production).
Ideally you have 3, and the third one is not used except to demo new features to customers AFTER they have been released (and perhaps, hidden behind a feature flag).
If you only have one sandbox, then it becomes a brawl between development, sales, and management.
More than three is overdoing it and assuming a technical burden for no benefit. (No, you don't need the capacity to spin up sandboxes willy-nilly. Three is enough.)
Re: Ask HN: Handling customers that want feature previewing?
#56Figure out why they're asking that. This appears to be an "XY problem"[0].
An example of this is a friend asking me how to weld a wire to a thin metallic plate. I digged further to find the actual problem for which his solution was to weld the two. He said it was a fuse's wire that melted and he wanted to weld a bigger wire to avoid that, to which I replied: don't. The whole point of the fuse is sacrifice to protect down stream equipment.
The same when a relative asks how to format a disk as their solution to solve a problem.
Generally speaking, implementing customers' solutions is a recipe for disaster, especially with enterprise.
Ask what it is they are really trying to accomplish, and figure out a way to solve that. Listening for problems instead of solutions or implementations.
Re: Ask HN: Handling customers that want feature previewing?
#57Is it that the customers want early access to new features, or they don't want you breaking the product?
Re: Ask HN: Handling customers that want feature previewing?
#58Earlier quoted context omitted.
> a clear signal to charge these customers a lot more money. Interesting, can you explain this a little more? Some of our customers chomp at the bit for particular features, and when we can we give them early access. We do not charge them more for arguably huge amounts of value add to the product. I wonder how to have these types of conversation, internally and with users. I'm a PM but have a direct line to managemen…
If you also have a direct line to your users, you could try asking them how much they'd be willing to pay for either some new feature that's about to come out, or for the right to early-access. Get maybe 3–5 data points and you should have a pretty solid case to management (either to charge retroactively for existing features, to charge new users for existing features, or to charge all users for upcoming features).
rather than ask "how much they'd be willing to pay", it might be more compelling to simply say "It will cost $X for early access, limited to X slots, in exchange, {we do such-and-such}."
If they say "OK, great", you've got immediate revenue to bring back to your management, and a strong case to continue doing more of the same.
Customers routinely over-and-under-estimate future value, compared to what they're then willing to pay.
They're often wrong in both directions, they usually won't pay as much as they think they will, _and_ they'll often pay far more than they think you will.
There are many, many reasons for both kinds of errors, many of which can be de-risked in advance and safely ameliorated.
Re: Ask HN: Handling customers that want feature previewing?
#59Once you know you need to do this (and at every enterprise SaaS company I've seen or worked at, you eventually need to do this - the sibling comments about charging more $$$ are smart though):
* "Release train ahead" - code takes 2 weeks (or 4 weeks) to reach Production from merge, but it goes out to a Preview environment immediately or at the next release. So, you've forked your code into a Prod train, a Preview train, and mainline. You probably need the ability to hotfix to prod and preview fix to the week-ahead-of-prod preview environment, as well as roll normal development forward. You do gain "soak time" for code in the Preview environment, which is a slight advantage. And your QA is easier, because you have three discrete, testable things ("does it work in Prod, does it work in Preview, does it work in trunk"). But, you have another runtime environment which can diverge, which is a big disadvantage.
* Feature flagging by "release train," single execution environment. This is probably the most "Software Theorist Friendly" way to do this. You don't have to maintain/debug multiple environments beyond what you're already doing. Some customers are in the "preview ring" and get more features toggled on. Single runtime environment, single codeline. Fixes are just fixes. QA is a little harder depending on how you set up the possible "rings" and combinations, and the operational blast radius of your Preview code is now the same as your Production code (make a bad query that kills a backend service in Preview code, well, that's also Prod code. Oof.).
* Free-range feature flagging. Some complex arbiter system manages a huge range of feature toggles on a per-tenant/per-customer basis. Having this range of operational knobs is very fun and works great for customers who are particularly risk averse. It also lets you sell SKU-packs of flag combinations, roll specific features to specific "Early Access Programs," and so on. But, you get a combinatorial explosion of feature flag permutations, which depending on how coupled your code is and how intelligent your QA system is, can become a disaster: "Oh no, FooFeature only works with BarFeature on but our QA system always has BarFeature on so we didn't catch the regression! And only one customer has BarFeature disabled for legacy reasons!"
* Some combination of 1, 2, and 3 - for example a Preview environment that runs the same _code_ as Prod, but with the toggles set differently. A few feature flags which can be enabled for early access programs, but a general rule that code needs to live in a release train. This is my favorite approach. Having a separate Preview environment eliminates the blast radius concern of Preview-toggled code breaking Prod. Operational costs are higher, but these can be passed to the customer (making customers buy preview environments as a line-item is pretty popular). But by maintaining single codeline (rather than deploying a fork of last week's code to Prod), the code-theory is still easy, you don't have divergent release trains, and you don't have the combinatorial explosion that comes from a free-range feature flagging system.
Re: Ask HN: Handling customers that want feature previewing?
#60Most of the discussion here will be technical, especially around feature flags and friends. That’s fine, but please don’t miss out that this is a clear signal to charge these customers a lot more money.
> a clear signal to charge these customers a lot more money. Interesting, can you explain this a little more? Some of our customers chomp at the bit for particular features, and when we can we give them early access. We do not charge them more for arguably huge amounts of value add to the product. I wonder how to have these types of conversation, internally and with users. I'm a PM but have a direct line to managemen…
enterprise support contracts provide access to early release builds / feature flags / etc, or require signing up to be a design partner (ex: logo + joint talks)