Live data from Hacker News

AWS costs every programmer should know (2019)

david-codes.hatanian.com

11–20 of 163 posts

Re: AWS costs every programmer should know (2019)

#11
post #4

It's interesting how AWS can keep so high prices on these. But it's just the beginning, the real money comes from when they convince you to run over a dozen vms/containers (all needing storage etc of course). You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. You can then get a pat on the h…

Hire a better engineer/architect.

Re: AWS costs every programmer should know (2019)

#12
post #4

It's interesting how AWS can keep so high prices on these. But it's just the beginning, the real money comes from when they convince you to run over a dozen vms/containers (all needing storage etc of course). You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. You can then get a pat on the h…

> You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. Why would you care about the redundancy on staging / dev? Just making up things to inflate AWS costs now.

If you're not testing az failovers you're probably just wasting money, like untested backups... But it's true that most people don't know this and aren't testing it because they don't know that az failovers don't automatically just work(tm). And the second downside of course would be asymmetry between the envs and divergence in the IaC etc resulting in more complexity and engineering work.

(Of course you're probably wasting money anyway since you don't actually business-wise need better uptime than the single AZ, and your complexity induced human fumbles will cause much more outages anyway, but this has been a main selling point of the decision to go to AWS, so the requirement needs to be defended)

Yes, you can build automation to have the redundant stuff up only sometimes, if you eat the engineering effort and complexity in your IAC and build automation... in the general vein of justifying engineering spend to offset AWS operating costs where running containers is very expensive!

TLDR: either way you end up paying for the very high markup in compute prices, it'll just be easier to excuse jumping through expensive hoops to "save money" on it.

Re: AWS costs every programmer should know (2019)

#13

The one AWS service that I pay for is S3 Glacier Deep Archive. It is very cheap insurance for knowing with virtual certainty that your data exists. Egress is expensive ($100/TB last I checked), but in the target use - backups of last resort - I will be very glad to pay that to have my data back. And if all goes right I'll never pay it. Personally I find ordinary S3 far too expensive to justify. Not so with Deep Archi…

Out of curiosity, what do you think a fair price for s3 would be?

I haven't had a problem with the storage prices, just egress fees can add up

(I don't work for aws or a cloud provider, just curious)

Re: AWS costs every programmer should know (2019)

#14
Network Egress always gets a bad rap but recently it's the SSD cost on AWS/GCP that has me upset. IMO storage needs to take on some of network egress's heat :)

$0.10 per GB-month for a basic SSD is outrageous compared to Hetzner et. al. a 4TB SSD for a month costs $400 on the big guys and like $50 on the commodity providers. It's not even a managed service!

Re: AWS costs every programmer should know (2019)

#15

The one AWS service that I pay for is S3 Glacier Deep Archive. It is very cheap insurance for knowing with virtual certainty that your data exists. Egress is expensive ($100/TB last I checked), but in the target use - backups of last resort - I will be very glad to pay that to have my data back. And if all goes right I'll never pay it. Personally I find ordinary S3 far too expensive to justify. Not so with Deep Archi…

Out of curiosity, what do you think a fair price for s3 would be? I haven't had a problem with the storage prices, just egress fees can add up (I don't work for aws or a cloud provider, just curious)

To be clear, I have no special insight into underlying S3 costs. Only the perspective of a customer.

My opinion is that Backblaze has done an awesome job of driving their storage costs down, scaling that up, and then reflecting that thriftiness in their pricing. Their technical communication (e.g. hard drive reliability reports, software engineering blogs) makes me believe in their competence. So when Backblaze sets a price, I expect that's pretty close to the minimum sustainable price, because they've spent a lot of effort on that.

B2 currently charges $5/TB/mo with lots of options for bandwidth that's too cheap to meter. This strikes me as pretty fair overall.

Re: AWS costs every programmer should know (2019)

#16
post #4

It's interesting how AWS can keep so high prices on these. But it's just the beginning, the real money comes from when they convince you to run over a dozen vms/containers (all needing storage etc of course). You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. You can then get a pat on the h…

Hire a better engineer/architect.

Indeed, this is often the next step. The new architect must have lots of AWS certifications. With the new architeture, the long term cost savings will be exponential, compared to the current projected cost curve!

This can optionally combine with the Kubernetes scenario.

Re: AWS costs every programmer should know (2019)

#17
It is strange that bandwidth/traffic/egress seems to be an afterthought in this post and I think the numbers in the linked blog post are wrong anyway.

Bandwidth out of AWS is at best 9 cents per gigabyte, which is HUGE and for many applications the primary cost.

Indeed when I saw the title of the post I thought "there's only one number you need to know with AWS, it's 9 cents per gigabyte".

Please correct me if I am wrong. I went looking to find the correct number but as always it is hard to find the exact prices.

Re: AWS costs every programmer should know (2019)

#18
post #12

Earlier quoted context omitted.

> You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. Why would you care about the redundancy on staging / dev? Just making up things to inflate AWS costs now.

If you're not testing az failovers you're probably just wasting money, like untested backups... But it's true that most people don't know this and aren't testing it because they don't know that az failovers don't automatically just work(tm). And the second downside of course would be asymmetry between the envs and divergence in the IaC etc resulting in more complexity and engineering work. (Of course you're probably…

If you’re building your own infrastructure in a data center then sure you absolutely want to test your redundancy.

But with AWS it’s a checkbox. It’s transparent to you and your applications. The infrastructure to host in multiple AZs is already in place. The only real issue with MultiAZ is the failover in RDS depending on the database you use could be seconds or 10s of seconds.

Re: AWS costs every programmer should know (2019)

#19
post #9

Earlier quoted context omitted.

I feel like AWS has a pretty good track record at this point for not increasing prices - certainly to the point that I trust them not to pull an Oracle. (Hypothetically, my volume of data stored with them is low enough that even a 10x gouge would just make me angry, not wound me financially. There's not a mechanism for these archives to incur explosive costs, but your point is why I avoid other less-predictable cloud…

Don’t be silly. It’s a game theoretic certainty that they will indeed pull an Oracle.

There's no need to act condescending or call people names ("silly"). A 15 year track record and a public commitment aren't things you can hand wave away.

I'd also point out there's no substance behind your "game theoretic" jargon.

Re: AWS costs every programmer should know (2019)

#20
post #12

Earlier quoted context omitted.

If you're not testing az failovers you're probably just wasting money, like untested backups... But it's true that most people don't know this and aren't testing it because they don't know that az failovers don't automatically just work(tm). And the second downside of course would be asymmetry between the envs and divergence in the IaC etc resulting in more complexity and engineering work. (Of course you're probably…

If you’re building your own infrastructure in a data center then sure you absolutely want to test your redundancy. But with AWS it’s a checkbox. It’s transparent to you and your applications. The infrastructure to host in multiple AZs is already in place. The only real issue with MultiAZ is the failover in RDS depending on the database you use could be seconds or 10s of seconds.

IME not so in the real world, you'll have accidental state in your distributed system outside the DBs. You'll have some stuff that actually always runs in one AZ in normal circumstances and your integration partner in another org has whitelisted only its IP. Etc etc, everything that is untested will find ways to conspire to rot. Especially if you haven't learned by seeing these bugs so you can avoid them.

Also you won't have a clear experience and understanding of what happens in the failover and you'll won't know to avoid failover breaking mistakes in your VPC configs, security groups, frontend-backend shared state etc. (And by "you" I mean "your dev team", it's not enough that one guy gets it).

Also^2 if you read the news about all the outages it's very common for failover systems to fail generally, not just AWS - the general engineering wisdom is: always test your failovers. And there's no substitute for end-to-end testing it, instead of individually testing each layer/module. (Bad: "we can skip testing db failover", good: "let's test that the whole system works when there's a az failure")

Post reply on HN