Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accident Forgiveness

fly.io

Post reply on HN
Print view View original

151–160 of 310 posts

Re: Accident Forgiveness

#151

Post by School-Cotton »

dlisboa wrote: post #103
belthesar wrote: post #95

Earlier quoted context omitted.

I'll be honest, I think you've got an uphill battle to convince so many of us that are concerned about accidental pricing that this is exactly the thing we actually want, vs. what we feel we want. This position parrots what Vercel's leadership said when someone had a massive surprise bill, and the story made the rounds both here and elsewhere. To be super honest, you might be right too. You might go down a huge engin…

> You might go down a huge engineering effort to build this This is an overlooked issue: billing caps are hard to implement and will likely incur losses for the cloud company that does. Take an object storage service as an example. Imagine Company X has a hard cap at US$ 1000 but some bug makes their software upload millions of files to a bucket and rack up their bill. Since objects are charged in GB/month they will…

$1000/mo in S3 deep glacier storage buys you a petabyte (a million gigabytes) of storage. It’s hard to imagine such a small customer uploading a petabyte without noticing, and part of what happens when you hit the cap could be moving things from normal object storage to glacier.

Re: Accident Forgiveness

#152

Post by tgsovlerkhgsel »

tptacek wrote: post #43
bitbasher wrote: post #38

There is a very obvious fix for surprise billing. Enforce a billing cap and terminate service if it's met. Even better if you send alerts when the cap is approaching. If I pay $39/month, a default cap should be $39 per-month. Otherwise, let me set a cap I am comfortable with. Surprise billing is never good for customers, only the business.

I promise, you are not the first person to have thought of this, and, believe it or not, there are reasons other than malice and avarice that cloud providers don't terminate service based on billing caps. Terminating service is a big deal. We agree completely about surprise billing.

Terminating service is a big deal for commercial customers' production environments.

Reversibly (i.e. shut down compute, don't delete anything, allow the customer to review, fix and reinstate quickly) terminating service is a minor annoyance for hobby/experimental setups, and in those, it's much more preferable than having to open a support ticket to deal with a massive bill.

Having quotas that the customer can increase themselves (but has to manually choose to increase) on storage prevents storage related surprise bills, and the rest you can shut down (optionally, make the user choose up front what they would prefer).

What am I missing? Too many commercial customers picking "experimental" initially and forgetting to change it?

Re: Accident Forgiveness

#153

Post by Aeolun »

tptacek wrote: post #54
gwbas1c wrote: post #51

Earlier quoted context omitted.

One thing I'm really curious about is why caps are so hard? (Perhaps this would result in a more technical blog post?) IE, you clearly don't want to terminate or shut down an account if they get too close to a cap. But what about things like a warning email, service slowdown, ect? Likewise, the old "slashdotted" or "hug of death" might be an appropriate result when something goes beyond a reasonable safety buffer? An…

We talk about warnings in the post. We'll do that at some point. The hard part about caps is what to do when someone hits them. If there was a way to make caps work for our core customers, we'd do it. We're open to ideas. A theme of our work this past month and these next several months is extracting maximal value from ANFWWAONW, our new billing system. The thing you have to remember though is that our belief about o…

You are comfortable scaling a service down to zero when configured to do so though.

Of course that comes up again when anyone sends a request, but that feels sort of in the same category.

That said, I do understand that building your service for people like me that’d rather be restricted to just $50/month doesn’t really make sense.

Re: Accident Forgiveness

#154

Post by ec109685 »

maliker wrote: post #123
ToucanLoucan wrote: post #36

Earlier quoted context omitted.

Anyone else remember we had widely available and completely understandable options to rent everything from baseline web-hosting all the way up to private rack services for actual years before AWS came along and apparently all of that was forgotten, Warhammer 40K style? I've rented a VPS from a vendor for going on 20 years now (Holy fuck I'm old) and I've never once been surprised at the bill.

You know that book "JavaScript: The Good Parts"? There needs to be a similar one "AWS: just the good parts". It would probably talk about EC2 (VPS), S3 (cheap bulk storage), SES (emails), and that's about it. When folks get into the elastic-super-beanstalk-container-swarm-v0.3 products, that's when they really kill themselves on the bills. That said, yes, just using a VPS vendor is the easy way to stick to the good p…

AWS the Good Parts (a bit dated now): https://dvassallo.gumroad.com/l/aws-good-parts

Re: Accident Forgiveness

#155

Post by ec109685 »

School-Cotton wrote: post #144
tptacek wrote: post #43

Earlier quoted context omitted.

I promise, you are not the first person to have thought of this, and, believe it or not, there are reasons other than malice and avarice that cloud providers don't terminate service based on billing caps. Terminating service is a big deal. We agree completely about surprise billing.

Okay, what are the reasons?

Big spike of real traffic and your site / app / database / system goes down.

Re: Accident Forgiveness

#157

Post by filleokus »

commercialnix wrote: post #131
javcasas wrote: post #69

Earlier quoted context omitted.

They laugh at me for constructing as much as I can from the infra as plain kubernetes objects. Keep laughing, I can switch clouds in less time than you take for figuring out cloud costs.

Keeping everything, including mission critical databases, "inside" Kubernetes is smart. The smart people keep everything "in" Kubernetes. Everyone else can pay up.

Like everything else it's a tradeoff. If you are running _everything_ inside Kubernetes you can easily move away, but I think you're loosing much of the benefit of being in the $big_cloud to begin with. If you have the staffing to provide your own db's, block storages, caches, IAM, logging/monitoring, container registries etc in a reliable way "as a service", the jump to some much cheaper bare metal hosting is not that far.

For me the sweet spot is to have all compute in Kubernetes and stick to open source or "standard" (e.g S3) services for your auxiliary needs, but outsource it to the cloud provider. Fairly easy to move somewhere else, while still keeping the operational burden down.

But agree that having e.g all your databases in a proprietary solution from the cloud vendor seems sketchy.

Re: Accident Forgiveness

#159

Post by srazzaque »

Whilst it doesn't have the mindshare or some features which are tablestakes for enterprise customers, I've found Linode's pricing to be extremely predictable for small projects. Even post Akamai acquisition. I'm sure other smaller players are also just fine.

Couple that with the fact you can achieve quite a bit on simple set-ups that are adequately sized to begin with, you can save quite a bit.

Not all of us need elasticity, or environments being spun up/down on commit.

Re: Accident Forgiveness

#160

Post by Aeolun »

School-Cotton wrote: post #150
tptacek wrote: post #74

Earlier quoted context omitted.

I've said it over and over again: if you have a serious app, and someone somehow steals a credential from you and uses it to light up a bunch of crypto miners, you don't want us shutting down your main app in response. We perceive it mostly as a feature that will blow people up. We agree about the underlying problem! You don't want to spend $5000 in a month for services you never wanted. We don't want you spending th…

Giving people the ability to use caps as an optional feature doesn’t force anyone to use it. Also, you could turn things off in the inverse order they were turned on until the cap is satisfied. So all the crypto miner instances would be turned off before database backups being deleted.

When you go over the cap everything turns off, otherwise, what’s the point?

Should you turn things off until you reach a certain spend rate? Then you can set your cap high enough for your server +1, but no more.

Post reply on HN