Live data from Hacker News

Serverless Horrors

serverlesshorrors.com

491–500 of 503 posts

Re: Serverless Horrors

#491
post #353

Earlier quoted context omitted.

With that model, your cost doesn't change, though. When/if you find you need more resources, you can (if you haven't been doing so) audit existing applications to clear out cruft before you purchase more hardware.

The cost of going through that list often outweighs the cost of the hardware, by a lot. And in a lot of cases it's hard to find out if a production application can be switched off. Since the cost is typically small for an unused application, I don't know if there are many people willing to risk being wrong

People always say stuff like this, and I just don’t buy it. It’s not that hard to analyze network traffic to see what does and doesn’t have active connections. When you’re relatively certain, shut it off for a week. If no one screams, delete it. If a month later someone is screaming, it’s their own damn fault for having no docs on something idle 90% of the time.

Re: Serverless Horrors

#492
post #353

Earlier quoted context omitted.

The cost of going through that list often outweighs the cost of the hardware, by a lot. And in a lot of cases it's hard to find out if a production application can be switched off. Since the cost is typically small for an unused application, I don't know if there are many people willing to risk being wrong

People always say stuff like this, and I just don’t buy it. It’s not that hard to analyze network traffic to see what does and doesn’t have active connections. When you’re relatively certain, shut it off for a week. If no one screams, delete it. If a month later someone is screaming, it’s their own damn fault for having no docs on something idle 90% of the time.

I've done many things that got new data ingested on a monthly basis. So say 29 days out of every month they would be idle.

Is it worth starting and stopping those kind of things? Probably not?

If you turn off a VM running something like that, because you didn't see any traffic for a day. Are you going to explain how you just shut it down to save a few dollars a month? I would very much like to see how that unfolds

Re: Serverless Horrors

#493
post #424

Earlier quoted context omitted.

So why isn't it the default yet? Why isn't unlimited scaling something you have to turn on ?

Because how you personally decide to handle cost overruns is up to you. AWS by itself can’t make that decision for you. The opposite problem when you do set low limits by default is that you constantly have to submit tickets to AWS to ask for service limit increases. How is AWS suppose to know whether you want to immediately scale or not? And before July of this year, there was no such thing as a “free tier AWS accou…

> How is AWS suppose to know whether you want to immediately scale or not?

Ask? This is not some impossible problem.

Yes, there is a UX challenge to be solved.

But also, doing so is well within the capabilities of a company like Amazon.

They simply have no incentive to help out since there is less money to be made by making it easier to spend less money. And, purely capitalistically, if you have to pick between a potential bug or misconfiguration that causes extra spending you can walk back with customer support, and a bug or misconfiguration that results in extra downtime for your 7+ figure customers, you pick the latter.

Their choice makes sense for their bottom line.

It's still bad UX for many users.

Re: Serverless Horrors

#494

Earlier quoted context omitted.

Indeed, if you keep your usage within the tier of usage that is free, they charge you exactly $0 per month. It is only for usage beyond that free tier that you are charged.

They should shut off your service if you go over the free tier, and all of these problems would disappear.

I agree that there should be an option for a user to select "I never want to pay so much as $0.01 and if there is a circumstance where I'd need to, I expect you to shut off my services and delete my data if needed in order to avoid me incurring any bills."

That would solve this issue, but needs to be an explicit opt-in, because a lot of users don't want that.

Re: Serverless Horrors

#495
post #493

Earlier quoted context omitted.

Because how you personally decide to handle cost overruns is up to you. AWS by itself can’t make that decision for you. The opposite problem when you do set low limits by default is that you constantly have to submit tickets to AWS to ask for service limit increases. How is AWS suppose to know whether you want to immediately scale or not? And before July of this year, there was no such thing as a “free tier AWS accou…

> How is AWS suppose to know whether you want to immediately scale or not? Ask? This is not some impossible problem. Yes, there is a UX challenge to be solved. But also, doing so is well within the capabilities of a company like Amazon. They simply have no incentive to help out since there is less money to be made by making it easier to spend less money. And, purely capitalistically, if you have to pick between a pot…

And AWS is suppose to do that across all 230+ services?

But as of July 15th of this year, there is actually a “free tier” that won’t let you spend over $200.

Before there were services with a free tier.

Re: Serverless Horrors

#496
post #426
post #360

Earlier quoted context omitted.

Did you do any training before launching the elastic beanstalk instance, or you just though a F-16 should be pretty easy to fly, at least according to most pilots?

An F-16 doesn't have a prominently-featured "getting started" tutorial, which has a bunch of step-by-step instructions getting a complete novice 40.000ft into the air at mach 2. AWS also provides training and education on how to use their services. If launching a "hello world" Elastic Beanstalk instance is so dangerous, why doesn't the tutorial require you to first provide proof that you are an AWS Certified Cloud Pr…

> doesn't the tutorial require you to first provide proof that you are an AWS Certified Cloud Practitioner?

An idea I can stand behind. Or do you just let any "self-learner" take care of your banking Oracle or IBM DB2 database...?

Re: Serverless Horrors

#497
post #360

Earlier quoted context omitted.

Did you do any training before launching the elastic beanstalk instance, or you just though a F-16 should be pretty easy to fly, at least according to most pilots?

Do you often find F16s that are (advertised as) free, and you get them with a click of 3 buttons?

Do you agree that if they would, they would still be an F-16 ?

Re: Serverless Horrors

#498
post #360

Earlier quoted context omitted.

Did you do any training before launching the elastic beanstalk instance, or you just though a F-16 should be pretty easy to fly, at least according to most pilots?

c’mon mate, be real. aws is absolute shit for beginers, this is such a bad comment

You might have not noticed...But you are making my point for me....

Re: Serverless Horrors

#499
post #352
post #310

Earlier quoted context omitted.

You're misunderstanding the offering. (Maybe that's their fault for using intentionally misleading language... but using that language in this way is pretty common nowadays, so this is important to understand.) For a postpaid service with usage-based billing, there are no separate "free" and "paid" plans (= what you're clearly thinking of when you're saying "tiers" here.) The "free tier" of these services, is a set o…

Oracle can do it.

Yes and no. Yes, if we're just specifically talking about the ability to support a free trial that will never bill you (i.e. what the OP was talking about); but no, if we're talking about the more-general ability to set spending limits and never be billed for overage (what this subthread drifted into discussing.)

Oracle Cloud has a 30-day free trial; and that free trial seems to have had some dedicated effort put into a whole divergent billing-infra path for it.

Under Oracle Cloud's free trial, you get a certain amount of spend ($300 in credits); and then, when your trial either expires (30 days) or you run that credit pool down to zero, your account is shut off.

Oracle do eat any marginal costs from your spend taking your credits "below zero" before they shut the account off, because your account was never billing to you anyway; it was billing to Oracle's marketing department as a lead-gen expense.

In other words, unlike Oracle Cloud's steady-state IaaS offering, their free-trial IaaS offering is actually a prepaid (but usage-billed) paradigm — with Oracle being the ones doing the pre-payment.

This works much like an oldschool prepaid phone plan, where you pay in every month to be given a certain number of [expiring/non-"rollover"] minutes/texts/MB of data; and then you get an itemized invoice at the end of the month for how close you came to "using up" each resource that month. And you very well can use up a resource's monthly paid allocation before the end of the month — e.g. "running out of texts" and being unable to send more, rather than those converting into something billed to you. (In a prepaid context, that "converting into being billed" is called "flex" or "pay-as-you-go" [PAYG] billing, and is usually some extra option you would have to enable, if offered at all.)

At scale, prepaid usage-billed systems are also asynchronous; to continue the telecom analogy, most phone-service providers won't re-aggregate your prepaid calling minutes to notice you've run out, until you hang up your current call. Only rarely do they have infra where the billing system can ping the telecom switches' control planes to say "hey, this guy just went over, hang up the call" — and when they do, they only do such checks on a 5-minute/30-minute interval, probably as a scheduled batch query.

But, yes, prepaid systems almost always do just eat any overage generated by this detection gap. This is usually safe, because prepaid systems are almost never elastic to the point that you could accrue nontrivial expenses during that short accounting gap.

When a system is that elastic, a systems architect responds by saying "this should be a postpaid system."

Which means that Oracle Cloud's free trial — insofar as it allows you to make use of truly-elastic resources with per-credit upstream basis costs, like FaaS compute — is probably vulnerable/exploitable. Oracle may sometimes be eating some hefty bills, where people on a free trial have wired their FaaS into a proxy fronting some already-highly-popular service.

This is mostly fine, if you have Oracle's treasury, because you'll still be doing KYC in advance of giving out these trials, so you'll only be letting any given individual do one trial.

But this does put Oracle in the territory of "having to think about people who buy burner identities on the black market [usually for ~$1] to sign up for services using them" + "having to think about people who sign up for their free trial and then sell that free-trial account's credentials on the black market [again, usually for ~$1]."

I haven't checked myself, but I would guess that like any other provider who sees this type of attack (e.g. Hetzner), Oracle Cloud likely has hardened registration flows that reject identities + cards from certain parts of the world; traffic fingerprinting heuristics that immediately shut down free trials if they start up a DDoS attack or the like; etc.

Which is something the other clouds get to skip thinking about entirely, by not having a true "free trial" with a prepaid model, and instead just offering e.g. a one-time $300 sign-up-bonus account credit.

---

But remember, we're only talking about the "free trial" here — something you only get access to for the first 30 days.

Oracle's free tier — the thing you have after the first 30 days — is no different than the one every other IaaS offers. It needs a billing account populated by your credit card; there's infrastructure to allow you to automate control-plane actions in response to billing thresholds being hit, but no offering that will wire anything up for you; etc.

In Oracle Cloud's free tier, you can set budget limits that will prevent new costed resources from being leased while your account is over that limit in a given month (which is certainly nice) — but those budget limits don't affect ongoing usage-based-billing of a resource. Your FaaS endpoints will continue to accrue vCPU-seconds of billed usage, until you — or some automation you wrote — shuts them off.

Re: Serverless Horrors

#500

Earlier quoted context omitted.

At Amazon scale, including a "we don't delete the data for 30 days if a bill isn't paid" clause is a plausible thing to include in the "free" tier. Paid tiers owe Amazon the contracted rate for the storage, as with any similar contract, and when Amazon deletes the data if payment isn't rendered when due is up to the terms of the contract.

There is no such thing as the “free tier” at least until July of this year. Some services are free for the first year up to a certain limit, some give you a bucket of free usage every month, etc.

Then you owe the contracted rate for the storage. These massive bills are almost never for storage, they're almost always for some sort of compute or transport left unrestricted. If you store 500TB you'll get an $11k/month bill, but the vast majority of the services can simply cut off usage at a limit. Even storage could prevent adding new data if you hit a pre-specified limit, so you'd only pay for the data you already had.

If I know my service should never use more than 1TB total I'd like to be able to set a limit at (say) 2TB total with warnings at 0.6TB & 1TB, thus limiting spend to $46/month on storage. Sure, my service will fail if I hit the limit, but if it's using double the storage I expect it to use something went wrong & I want to require manual action to resolve it instead of allowing it to leak storage unbounded.

This is not a particularly difficult problem to make significant improvements on. There are some edge cases (there always are) but even if spending limits were only implemented for non-storage services it'd still be better for customers than the status quo.

Post reply on HN