Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serverless: A lesson learned the hard way

sourcebox.be

Post reply on HN
Print view View original

101–110 of 139 posts

Re: Serverless: A lesson learned the hard way

#101

Post by SimonPStevens »

sudhirj wrote: post #82
dvfjsdhgfv wrote: post #57

Earlier quoted context omitted.

I know you're being sarcastic but I feel it's partly true. They announce so many different services, each month something new appears, but this very basic feature - bill capping - asked by users from the very beginning, has never been implemented. It's hard to believe they lack the skill or that it would be much more complicated than the current alert system.

According to the folks I've spoken to, they don't do bill capping because they have no way to safely shut down your workloads in any way - they'd prefer to let you know and have you do it. And having that choice is way better than a capping operation destroying your production database or causing downtime for your users.

I'm sorry, I just don't buy that. It doesn't have to be a hard cap, it could be a soft one. i.e. at £x your servers start shutting down, you'll get billed for a few extra minutes over your cap, before things have finished shutting down. Servers are totally capable of being shutdown without destroying databases.

Besides, we aren't really talking about production databases at large companies. The people who want caps are devs learning and experimenting. It could come with dislaimers that if you enable a cap and exceed it that your services will go offline unexpectedly, and that may leave databases in inconsistent states. But for a large number of usage scenerios that is a completely acceptable tradeoff.

The simple fact is, not having a cap certainly puts me off experimenting with a service due to a fear of a mistake causing a big bill. And developers learning and investigating a technology is what preceeds them recommending that technology to their companies.

Last time I looked Azure allows a zero spend cap on free accounts, but you can't change the amount to anything else, and once you remove it you can't switch the cap back on. Thats limited, but it's perfect for a learning environment.

If Azure can implement a zero spend cap, there is absolutly no reason that either AWS or Azure can't implement an x spend cap in exactly the same way.

Re: Serverless: A lesson learned the hard way

#102

Post by vacri »

ryandrake wrote: post #46
bluGill wrote: post #42

Earlier quoted context omitted.

Yes, but that isn't always the right answer. If your system is starting to cost "a lot" is that because of a bug (this case), or is it because your idea just "went viral" and you are not getting tons of paying customers signing up? If it is the latter you do not want any rate limiting, you want everything to scale as fast as possible (I hope there are no bugs on your end). Rate limiting means that your new customers…

It's almost as if... they should offer multiple options so customers could choose based on their business/hobby needs: 1. Warn me at $X but don't throttle me for any reason--I'll pay if I go viral 2. Warn me at $X and start throttling until I get to $Y at which point stop service and stop charging 3. Warn me at $X and stop service/charging immediately

AWS has so many services that trying to decide what to stop if you reach a billing threshold would be impossible to automate. Similarly, pricing is not built into the individual services APIs, so adding a per-item billing threshold would not be a trivial task.

> based on their business/hobby needs

AWS is not interested in hobbyists - other vendors are picking up the crumbs there.

Re: Serverless: A lesson learned the hard way

#103

Post by coldcode »

sah2ed wrote: post #90

Charged at a flat rate like the cost of a Digital Ocean $5/mo instance, would developers pay for such a service to provide automated notifications of service overages for all the major cloud providers? All a developer needs to do immediately after adding a credit card to AWS/Azure/GCP would be to create an IAM role with permission to automatically add and track fine-grained billing alarms and notify via email/sms for…

I looked at Lambda (we use AWS a lot at work) and decided to simply stay with a flat rate DO server. I know what it costs, no need to worry.

Re: Serverless: A lesson learned the hard way

#104

Post by vacri »

Terretta wrote: post #65
OzzyB wrote: post #29

Earlier quoted context omitted.

It's "serverless" in the sense that if the developer had provisioned a "server" then the max incursion of cost would equal the cost of that server, no more no less. So yeah, let's blame the developer, but let's not play like mistakes don't happen and they're not costly in the "serverless" world.

Use EC2, set an auto scaling threshold on CPU utilization, do something dumb, you’ll find you ran “a server” x n. It’s easy to burn tens or hundreds of thousands ‘accidentally’ on “server”, easier than on serverless. If you’re spending real money, you should have an account team. Talk to them if such a problem happens.

An autoscaling group has a max spend of '"a server" x n', where you set what n is. Autoscaling groups don't keep adding servers forever.

Re: Serverless: A lesson learned the hard way

#105

Post by lostcolony »

dvfjsdhgfv wrote: post #59
lostcolony wrote: post #26

Earlier quoted context omitted.

So run my static content only blog on dedicated hardware that I have to administer rather than throw it in an S3 bucket with a Cloudfront on it? No thank you. Qualify your statements.

You don't really need to administer it, there are plenty of hosting options, not just bare metal. And you can always add Cloudlflare anyway.

You're missing my point. "Serverless" just means "Applications built on cloud services rather than server(s) I have to administer".

The OP said I should run my own infrastructure. I -could- host my blog by running a web server atop a server I administer, sure. I'd have to take on all the infrastructural tasks of doing that, securing it, ensuring any availability/scalability concerns I may have are taken care of, etc, but I -could- do that.

Instead, S3 + Cloudfront (or, sure, any flavor of hosting and edge caching options you care for; I was not implying "Just AWS") means I don't have to worry about any of that. For me, the reduced level of control, increased availability, scalability, and easy "it just works", is worth the tradeoff. As is the pennies per month it costs me given the low utilization and pay-as-you-go model. It's hardly a scam.

Re: Serverless: A lesson learned the hard way

#106

Post by antaviana »

eridius wrote: post #28
cddotdotslash wrote: post #17

This is not a "Serverless" problem; this is a mistake a developer made that used a pay-per-use system. If I write code that launches EC2 instances and I accidentally set it to launch an instance every second instead of minute because I divided wrong, that's my fault.

It is a serverless issue because if you were using your own server, a mistake like this wouldn't have cost money, it would have just degraded your service (or possibly brought it offline). So I guess the question is, with a mistake like this, is it better to be charged hundreds or thousands of dollars, or to have your service degrade or go offline until you can fix it?

There are chances that degradation or unavailability are not free as in beer.

If the degraded or offline system is used by people, and these people cannot work, the cost can be a lot higher. For example, 10 people not able to work could cost something in the range of $250-$750 per hour.

Moreover, if customers are lost due to this degradation of service and CAC is high, then clearly the cheapest thing is a high bill by AWS, which probably is also capped by Amazon (and handled as an alert by Amazon).

Re: Serverless: A lesson learned the hard way

#107

Post by vacri »

occultist_throw wrote: post #52
concede_pluto wrote: post #38

Earlier quoted context omitted.

The vendor could still assign the debt to a collection agency or sue; a declined charge does not get you off the hook unless they decide it's not worth pursuing.

That's very true. That's why I provide generic usernames and everything. Because how the current providers offer service is as through a debt system. You rack up the $$$, and they tell you after the fact. I would greatly prefer to pay up front, and have services take my credit. That way, I could control my costs directly and concisely. No surprise billing. DOS'es get stopped by no more funds- they aren't the infinite…

If you're out of credit, what happens to your data-at-rest? You know, the stuff you're storing in their block storage, where they charge just for storing it? Should they just purge your data?

Re: Serverless: A lesson learned the hard way

#108

Post by AYBABTME »

cddotdotslash wrote: post #17

This is not a "Serverless" problem; this is a mistake a developer made that used a pay-per-use system. If I write code that launches EC2 instances and I accidentally set it to launch an instance every second instead of minute because I divided wrong, that's my fault.

It's a pricing model and limits issue from AWS. Same as you can't launch 1000 instances by mistake (there are account limits) you should be able to run hundred of thousands of recursive serverless calls by mistake (isn't there an account limit?).

Re: Serverless: A lesson learned the hard way

#109

Post by fenwick67 »

clentaminator wrote: post #19
mipmap04 wrote: post #15

Earlier quoted context omitted.

Azure has this. When you hit your spending limit, it shuts down your services.

And now we wait for the first reports of production services that were shutdown due to spending limits :)

I would rather production go down for a bit than have the whole company go bankrupt

Re: Serverless: A lesson learned the hard way

#110

Post by dvfjsdhgfv »

lostcolony wrote: post #105
dvfjsdhgfv wrote: post #59

Earlier quoted context omitted.

You don't really need to administer it, there are plenty of hosting options, not just bare metal. And you can always add Cloudlflare anyway.

You're missing my point. "Serverless" just means "Applications built on cloud services rather than server(s) I have to administer". The OP said I should run my own infrastructure. I -could- host my blog by running a web server atop a server I administer, sure. I'd have to take on all the infrastructural tasks of doing that, securing it, ensuring any availability/scalability concerns I may have are taken care of, etc,…

I wouldn't call it a scam. You need a certain level of expertise anyway, otherwise you not only screw up your service, but also, in the case of AWS, lose money. Practically speaking, the amount of work needed to set up a blog on S3 with CloudFlare is not that different from the one needed to deploy, say, a WordPress droplet on DigitalOcean. You just click a few times and it works, you can basically forget it. The only difference is that you are guaranteed not to pay more than $5 or whatever your monthly plan is.

Of course scalability is incomparable in both cases, so if it's something that really matters - and matters more than money - of course something like AWS is a better choice.

Post reply on HN