Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accident Forgiveness

fly.io

Post reply on HN
Print view View original

231–240 of 310 posts

Re: Accident Forgiveness

#231

Post by Aeolun »

tptacek wrote: post #172
Aeolun wrote: post #153

Earlier quoted context omitted.

You are comfortable scaling a service down to zero when configured to do so though. Of course that comes up again when anyone sends a request, but that feels sort of in the same category. That said, I do understand that building your service for people like me that’d rather be restricted to just $50/month doesn’t really make sense.

I don't think anyone with a serious app running on us will use a cap. Just stay fixated on this scenario: a deploy-only token gets stolen, and the attacker (like most cloud attackers) uses it to stand up a bunch of Monero miners. As a consequence... their main app goes down? Who would be OK with that?

I think the cap (if they had any) would probably reflect the amount they stand to lose if their app goes down. If your app brings in $1000/h, you don’t necessarily care about spending that amount on servers. When your costs rise to $10k/hour, you might want to go with the nuclear option.

Of course it’s nicer if you can be certain that your provider is going to refund you the excess, but I feel like it’s hard to count on it. Or at least, harder than having explicit rules, which you just can’t really do for those sitations that are sensitive to fraud.

Honestly, if I did set a cap I’d be very much aware of the fact my app could suddenly die in a situation where my deploy token were stolen (but it wouldn’t matter for me, since it’s a hobby project, I care about controlling costs, not uptime).

Re: Accident Forgiveness

#232

Post by solidasparagus »

tgsovlerkhgsel wrote: post #152
tptacek wrote: post #43

Earlier quoted context omitted.

I promise, you are not the first person to have thought of this, and, believe it or not, there are reasons other than malice and avarice that cloud providers don't terminate service based on billing caps. Terminating service is a big deal. We agree completely about surprise billing.

Terminating service is a big deal for commercial customers' production environments. Reversibly (i.e. shut down compute, don't delete anything, allow the customer to review, fix and reinstate quickly) terminating service is a minor annoyance for hobby/experimental setups, and in those, it's much more preferable than having to open a support ticket to deal with a massive bill. Having quotas that the customer can incre…

No one really cares enough about hobby developers as a customer segment to rebuild the billing infrastructure to make this possible. Scalable billing at huge scale is solved at the cost of latency and being "eventually correct" (unless it has changed). To add a price cap feature, eventually correct isn't enough and then you ask yourself who would actually use it and you have to scroll really far down your list of biggest customers until you get to someone who wants it.

Re: Accident Forgiveness

#233

Post by franga2000 »

jiveturkey wrote: post #167
tmnvix wrote: post #121

Earlier quoted context omitted.

My preferred option would be to have an optional billing cap that I can enable knowing full well that if it is exceeded the service would be terminated (obviously with notifications as that cap is approached). I could then apply this to simple hobby projects and such, while not having the risk of termination apply to more serious applications (though a 'soft cap' would be nice here so that I could still receive notif…

As a certain kind of user, you probably do think that. But I also think I should be able to have a root level admin account without MFA. The consensus is that no, that should not be up to the customer. It's different here, sure, but the providers optimize for not letting customers shoot themselves in the foot, and remediation via bill forgiveness is a fine solution -- from the provider POV.

You should be able to have a root level admin account with no 2FA! I would print mine and keep it in a tamper-evident envelope in a safe at my lawyer's office with instructions for when and who can get it.

A company isn't liable if their customer gets themselves hacked because they decided to not use any of the many MFA options available to them and neither is a company liable if the customer set a billing limit rule that they executed correctly.

Companies can simply not be trusted to tell the difference between a foot-gun and a..whatever a good kind of gun would be...

Re: Accident Forgiveness

#234

Post by Tiberium »

perchlorate wrote: post #120
bdcravens wrote: post #4

Earlier quoted context omitted.

There's plenty of situations you don't hear about. A few years ago I f'ed up and accidentally pushed keys to a public repo, and by the next morning, we racked up $50k in AWS charges from crypto miners. We reached out, they gave us a security checklist that if we followed, they'd take off the charges. We did, and by Monday (my code push was Friday evening) the charges were taken off. No public shaming required.

You likely already know that, but to anyone else interested: a good way to prevent these kinds of situations is to run 'nosey parker' on your git repo before pushing it to a remote. It will dig through your code and configs, looking at files and through all the git history, and highlight anything that looks like tokens, passwords, keys, etc. You can set it as a pre-commit hook to block the offending code from even be…

https://github.com/trufflesecurity/trufflehog is a similar tool but checks for far more secrets, so I think it'd be a better choice.

Re: Accident Forgiveness

#235

Post by isoprophlex »

osbulbul wrote: post #29

the biggest reason of I am not using aws, google cloud, vercel etc... for my personal projects is surprise bills. I am not earning anything from them so I can only put $50 but can't afford $500 or $5000. so still i will feel much more secure if I can put hard limit and absolutely sure my bill will never go above $50. (or cloud billing insurance :)) but you got my attention, i can try fly.io

Hetzner. 40$/mo gets you a 10-core, 20-thread intel box, 64 gb ram, 1 TB storage, unlimited traffic.

Re: Accident Forgiveness

#236

Post by genewitch »

tptacek wrote: post #204
genewitch wrote: post #203

Earlier quoted context omitted.

i mentioned this as well in another comment to parent. Hopefully it catches on. "Hey, SMB? you probably don't need our services. While AWS, GC, etc would be happy to take your money anyhow, [...]" I dunno. obviously any sort of thing like this has to clear all the departments because i imagine it increases support load.

No SMB is OK with us zapping their services because some screwed up deployment or stolen token exceeded a billing cap. You're not talking about SMBs, you're talking about people deploying random personal stuff. We love those people, we're happy to have them here, but we do not price the product specifically for them. I get that this is a lot of venting about people's issues with cloud providers writ large, but damn.

i said SMB because SMB doesn't need cloud. if you have enough users to warrant cloud you're no longer S or M. I was actually going to start to concede that you might be right, maybe i should have said small-time users or something more eloquent; but no, i'll stand behind what i said. Dissuade all SMB and small users as much as possible. then there's 0% chance they get a surprise cloud bill. I solved the problem, yay.

as i mentioned elsewhere i'm intimately familiar with pretty much every intimate detail of "cloud" from hardware, software, network, cooling, and ops (i wouldn't call me a dev. I don't think anyone else would or should, either.) I've bootstrapped cloud services from empty racks twice and repurposed existing hardware for cloud once.

I understand why there's no "cap" available on any cloud services. I mostly have a problem with capitalism, which is ironic, considering this site.

Re: Accident Forgiveness

#237

Post by campers »

ctennis1 wrote: post #111

Google Cloud will not cut you a break unless you have an incredible about of pull. Even then, you won't get much of one.

Being at the level where you have an account manager should help. Our technical account manager always said if we ever accidentally rack up a bunch of cloud costs we should at always ask if we can get a refund, and as long as its infrequent there's a good chance to get some credits.

Re: Accident Forgiveness

#238

Post by campers »

> Meanwhile: like every public cloud, we provision our own hardware, and we have excess capacity. Your messed-up CI/CD jobs didn’t really cost us anything

This^ Don't feel bad about asking for credits when you accidentally make a costly mistake.

Re: Accident Forgiveness

#239

Post by CaptainFever »

pocketarc wrote: post #109
mappu wrote: post #102

Earlier quoted context omitted.

You might be happier with a fixed-size VPS, instead of a cloud provider,

I started writing this comment by saying: Exactly - the whole point of a cloud provider is scalability. If you're doing a personal hobby project, get off big scalable clouds and get yourself one (or multiple!) fixed-price VPS or dedicated servers. But as I think of it, I think what people really want, for hobby projects, is not so much the scalability, but the managed offerings. They want zero-ops, zero-maintenance,…

An important part of hobby projects is the scaling to zero part too. I wager a lot of hobby projects use cloud simply because it's free or almost free (e.g. 2 cents a month), which isn't the case if you rent a VPS.

Re: Accident Forgiveness

#240

Post by spion »

tptacek wrote: post #46
osbulbul wrote: post #42

Earlier quoted context omitted.

yeah I am pretty sure no cloud provider thinks people who can willing to spend small money. but guess what? if one of my projects start earning money, then I will continue to what I know best :) I think that's why every cloud provider still has configs for couple of $

We think all the time about people starting small projects here. I think a really good line to draw, if you want to understand how we think about this stuff, is between people who would be OK with us terminating their service because they mispredicted what their cap should be, and people who need their stuff to stay up and running and will just talk to support if they're concerned about billing. A shorter way to say…

You're selecting for people who feel they're priviledged enough to reach out to support and ask for bill forgiveness even when they may've messed up, repeatedly. This does select for something but I'm not 100% sure "people who are doing this work professionally" is that clear-cut, especially once you move past western cultural norms.
Post reply on HN