Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accident Forgiveness

fly.io

Post reply on HN
Print view View original

91–100 of 310 posts

Re: Accident Forgiveness

#91

Post by hinkley »

srockets wrote: post #77
AtlasBarfed wrote: post #15

Earlier quoted context omitted.

Another reason to be cloud agnostic. They are counting that switching cost will make you eat the bill. If you have to be cloud, do dev in one cloud and test/prod in another. I know, I know, easier said than done.

An argument that is often only made by folks who didn't ran anything at scale on a public cloud. If you've ever set in a room when a cloud deal was signed, you'd know it cable: you don't get to pay less by threatening to switch providers (I've seen people try to suggest that and get laughed out of the room), but by buying more from a single one. Hence accident forgiveness is in the same marketing bucket as free credi…

In the IBM, HP days it wasn’t about paying less per se by threatening to switch providers, it was getting more concessions (of which money might be one). Some big companies had both systems so they could play them off of each other. Time to order more hardware, who will kiss our butts more?

I’d be very surprised if that doesn’t still exist for the big boys. Though most of us are not big boys, and half of the biggest boys are cloud providers themselves.

Re: Accident Forgiveness

#93

Post by bitbasher »

tonnydourado wrote: post #89
bitbasher wrote: post #38

There is a very obvious fix for surprise billing. Enforce a billing cap and terminate service if it's met. Even better if you send alerts when the cap is approaching. If I pay $39/month, a default cap should be $39 per-month. Otherwise, let me set a cap I am comfortable with. Surprise billing is never good for customers, only the business.

When you're paying $39/month for something that generates $0/month, that is a very sensible policy. When you're paying $50,000/month for something that generates $200,000/month in value, or if an outage can generate $100,000/month in costs, or if the people that can fix an outage cost $100,000/year, then it's not.

Then that company's threshold for "sensible monthly cost" would be a lot higher. 500k? 1m? Give them the option to set something.

Re: Accident Forgiveness

#94

Post by hinkley »

tptacek wrote: post #32
osbulbul wrote: post #29

the biggest reason of I am not using aws, google cloud, vercel etc... for my personal projects is surprise bills. I am not earning anything from them so I can only put $50 but can't afford $500 or $5000. so still i will feel much more secure if I can put hard limit and absolutely sure my bill will never go above $50. (or cloud billing insurance :)) but you got my attention, i can try fly.io

Except for things like student accounts, which we're working on, I don't think hard limits are coming any time soon. Our expectation is that the enforcement of hard billing limits would mostly make customers furious with us. If you read to the end of the post, the direction we're going is preemptive detection of weird billing spikes, so you don't even have to notice and ask us. If you're really only looking to spend…

If there is a class of people who get steered to your service by a small number of organizations, you might. Universities, trade schools, a podcaster you’re sponsoring.

You can’t scale to individualized service for $50 per month/quarter/year users, that’s true. But you can shape policy for a demographic with… shall we call it flocking behavior, for lack of a better term?

Re: Accident Forgiveness

#95

Post by belthesar »

tptacek wrote: post #74
ustad wrote: post #73

Earlier quoted context omitted.

In your article or in the posts here you have not said why you think its a crazy idea.

I've said it over and over again: if you have a serious app, and someone somehow steals a credential from you and uses it to light up a bunch of crypto miners, you don't want us shutting down your main app in response. We perceive it mostly as a feature that will blow people up. We agree about the underlying problem! You don't want to spend $5000 in a month for services you never wanted. We don't want you spending th…

I'll be honest, I think you've got an uphill battle to convince so many of us that are concerned about accidental pricing that this is exactly the thing we actually want, vs. what we feel we want. This position parrots what Vercel's leadership said when someone had a massive surprise bill, and the story made the rounds both here and elsewhere.

To be super honest, you might be right too. You might go down a huge engineering effort to build this in only for 5% of your customers to ever engage it. I think the real question is what percentage of your customers will feel better knowing that the have the choice to set those limits, and how will that comfort actually improve their trust with Fly, and cause them to choose it over another cloud provider.

It may end up being a lot like this feature we had in a platform that I used to support. It had a just-in-time analytics pipeline that at one point required tens of thousands of dollars in compute, storage and network hardware alone to function. Based on our analytics, it was barely used compared to the usage of the rest of our app, which made the zounds of resources and fairly frequent support attention it needed feel silly in comparison, so I advocated to sunset it. Product assured me that, regardless of how silly it might be to continue supporting this feature, it was a dealmaker, and losing it would be a dealbreaker.

So yeah, y'all might be right in that the majority of your customers don't actually want it. But maybe what they do need is to know that it's there, ready for them if they ever need to engage with it.

Re: Accident Forgiveness

#96

Post by Alacart »

tptacek wrote: post #54
gwbas1c wrote: post #51

Earlier quoted context omitted.

One thing I'm really curious about is why caps are so hard? (Perhaps this would result in a more technical blog post?) IE, you clearly don't want to terminate or shut down an account if they get too close to a cap. But what about things like a warning email, service slowdown, ect? Likewise, the old "slashdotted" or "hug of death" might be an appropriate result when something goes beyond a reasonable safety buffer? An…

We talk about warnings in the post. We'll do that at some point. The hard part about caps is what to do when someone hits them. If there was a way to make caps work for our core customers, we'd do it. We're open to ideas. A theme of our work this past month and these next several months is extracting maximal value from ANFWWAONW, our new billing system. The thing you have to remember though is that our belief about o…

Configurable warnings as webhooks would be pretty cool. Then I can automate whatever needs to happen on my side.

I already automate apps, machines, etc with the machines API and GraphQL, so my big worries in this area are:

- Woops, some bad logic deployed too many machines (sounds like this policy helps) - Some kind of mistake or attack that just explodes bandwidth usage suddenly

Re: Accident Forgiveness

#97

Post by hinkley »

gwbas1c wrote: post #16

> and may only come within tens of dollars of accurately predicting your water bill. Whenever I've had to water new grass seed I've always been surprised by my water bill.

There’s a guy on Reddit a couple days ago showing off his softball sized watermelon he grew with a $100 watering bill. The struggle is real.

Re: Accident Forgiveness

#98

Post by joshfraser »

trjordan wrote: post #83
joshfraser wrote: post #27

In other words, they are socializing the costs. Servers and electricity aren't free. Wouldn't it be better for the customer if they had no accident forgiveness and passed those cost savings along? Instead, you are paying for other peoples mistakes plus all the extra overhead caused from fraud that they incentivized.

The servers are already bought; the electricity cost is negligible for actual accidents. As mentioned in the post: the hosting providers don't actually pay marginal costs for transient mistakes, so neither should honest customers.

And what about the extra costs fighting fraud now that they've advertised this policy?

Re: Accident Forgiveness

#99

Post by gary_0 »

bitbasher wrote: post #63
tptacek wrote: post #53

Earlier quoted context omitted.

Nobody is going to charge you $100k.

Perhaps that is true, but the stress and anxiety from seeing 100K bill is real and has an impact. https://www.forbes.com/sites/sergeiklebnikov/2020/06/17/20-y...

Just in case you don't know, the person you're replying to is the author of the blog post, and they mention in it that being stressed out by (potential) unexpected large bills is something they are aware of.

Although even then, "we will only threaten to charge you $100k without meaning it" isn't much of a reassurance.

Re: Accident Forgiveness

#100

Post by tptacek »

bee_rider wrote: post #87
tptacek wrote: post #74

Earlier quoted context omitted.

I've said it over and over again: if you have a serious app, and someone somehow steals a credential from you and uses it to light up a bunch of crypto miners, you don't want us shutting down your main app in response. We perceive it mostly as a feature that will blow people up. We agree about the underlying problem! You don't want to spend $5000 in a month for services you never wanted. We don't want you spending th…

I think your position is essentially that you mostly want to serve people who have serious apps, for whom the cap idea doesn’t work. And you definitely know more about the general trajectory of your customers than all of us sitting here randomly speculating. But, is it really so uncommon for an application to transition from unserious, I want to cap, to serious? I guess I’m slightly confused because I thought one of…

Sure, but bear in mind that across the entire industry of public clouds, which has been around for something like 2 decades now, nobody really has this feature. In fact, didn't Google have this feature and then pull it?
Post reply on HN