Live data from Hacker News

AWS costs every programmer should know (2019)

david-codes.hatanian.com

151–160 of 163 posts

Re: AWS costs every programmer should know (2019)

#151
post #138

Earlier quoted context omitted.

What sort of application are you building where you need a thousand instances of a spring application? Like even hosted on a low powered hardware, Java services can easily deal with order of 100 requests/second. This is home turf, this is what Java is really good at. That 100rps may be lowballing it. So you're what, processing 100k+ API requests per second? I can't imagine that would translate toward any less than a…

What sort of app code are you writing where you can keep data, authentication and user data all on the desktop? Just because you can write a docker-compose.yml (or do without) that describes a system that holds together, doesn't mean the system you've produced stands the chance at SOC2 or whatever standard you hold dear. Having just finished a 27001 audit, I promise, "we rely on AWS for X" went a lot farther than "we…

Strong "forgotten how to count that low" vibes from this one.

Re: AWS costs every programmer should know (2019)

#153
post #41

AWS is overrated. Most of the engineers and managers choose it because it's a safe choice for them. But in the long term the costs rise and you can't get away from it. I would even argue it's only for those who are afraid of command line.

What would you use instead?

Re: AWS costs every programmer should know (2019)

#154
post #150
post #19

Earlier quoted context omitted.

There's no need to act condescending or call people names ("silly"). A 15 year track record and a public commitment aren't things you can hand wave away. I'd also point out there's no substance behind your "game theoretic" jargon.

The parent put it too strongly but it's equally naive to assume that track records and public commitments mean anything to a privately-held company, with a commitment to profit shareholders, run by Jeff Bezos.

I can’t believe this needs to be spelled out.

Of course AWS needs to make money.

Of course they make more money in the short term if they raise prices.

Of course this affects their long term revenue and profitability because customers are afraid that prices of any of the products they use could rise.

Say whatever you want about Amazon, but they have never been a company that cares about the short term. They care about profits for the next 20 years, not Q3 2022.

You’re calling me/my views naive? Seriously? Just after I excoriated someone for name calling?

Re: AWS costs every programmer should know (2019)

#155
post #4

It's interesting how AWS can keep so high prices on these. But it's just the beginning, the real money comes from when they convince you to run over a dozen vms/containers (all needing storage etc of course). You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. You can then get a pat on the h…

As an "onPrem" & "Big Data" guy I've been astonished at how fast the bill can run up. We are in the middle of a cloud adoption where the big proponent started with "its just money, devs time costs more.. do what you gotta do" to "hey so we need to consolidate these 3 DBs, and X is on vacation so lets spin this service down, and lets not worry about multi AZ/region yet" .. all before we even have a PROD launch, lol. I…

[deleted]

Re: AWS costs every programmer should know (2019)

#156
post #4

It's interesting how AWS can keep so high prices on these. But it's just the beginning, the real money comes from when they convince you to run over a dozen vms/containers (all needing storage etc of course). You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. You can then get a pat on the h…

As an "onPrem" & "Big Data" guy I've been astonished at how fast the bill can run up. We are in the middle of a cloud adoption where the big proponent started with "its just money, devs time costs more.. do what you gotta do" to "hey so we need to consolidate these 3 DBs, and X is on vacation so lets spin this service down, and lets not worry about multi AZ/region yet" .. all before we even have a PROD launch, lol. I…

> cost of an RDS instance

If RDS is expensive, you can always spin up your own DB deployment, you can use an EC2 instance, or you can just run it via ECS or EKS.

You buy those beefy servers, and where do you put them? you need a data center, where you need data center grade internet lines, you need staff to maintain all that.

> Similarly on the ops/infra folks cost save, for every on-prem infra guy they think they can RIF, they've hired a cloud infra guy who for sure costs more.

I disagree, you will need less cloud infra people, who will likely cost more, however everything will end up being more reliable than having a legion manually maintain everything.

> Likewise for the redundancy/backups/etc being "easy/free" in the cloud, we've already lost data in object store because of poorly execute combination of configuration & actions when try to do some permission changes/copy migration. It was completely unintentional and not noticed for weeks. No one actually executed a literal rm command at the time. Just because the object store advertises zero loss / versioning / etc, you really do still need to do backups.

Did you ever lose data by properly putting something to S3 or Glacier and not doing anything with it?

> However for internal apps at a firm where the usage is largely scaled with staffing levels, the amount of fixed compute makes it hard to argue every app and every use case belongs in the cloud.

It's actually easy to argue: - the world is not going to stay stagnant, if I need 1 more server tomorrow, I can get it in minutes on the Cloud, use it and get rid of it whenever I no longer have a need for it. - the on-prem costs are downplayed, people usually just talk about the cost of the metal and forget the upkeep and extra staff required - the on-prem reliability can be very questionable, hardware failures can cause disasters - cloud stacks can be easier to maintain and hand over, you have less hardware and less operating systems to worry about

Re: AWS costs every programmer should know (2019)

#157

Earlier quoted context omitted.

About 5 or 6 years ago we had an alert in the middle of the night that our RDS instance dieded. It failed over in about 15 seconds (SQL Server so it’s a bit slow compared to PostgreSQL) but the MultiAZ worked as advertised. The downside is AWS never told us why it occurred.

I’ve seen a few AWS instance hardware failures, they happen with some regularity. You can handle single instance failure without being multi AZ. Testing an actual AZ failure, as in the whole AZ going offline or getting partitioned from the other AZs, is pretty much impossible.

AZs are connected via normal user visible networks, you can just break those. They even provide examples, https://github.com/awslabs/aws-well-architected-labs/tree/ma...

Those are basic (don't cover flapping or glacial-speed slowdown degradation modes, some services only, etc) but a starting point at least that can be extended.

Re: AWS costs every programmer should know (2019)

#158
post #4

It's interesting how AWS can keep so high prices on these. But it's just the beginning, the real money comes from when they convince you to run over a dozen vms/containers (all needing storage etc of course). You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. You can then get a pat on the h…

> You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x.

Don't believe this has anything to do with the cloud, if you want the same thing with your own data center, it's going to be 18x there as well.

> You can then get a pat on the head ("pass AWS well-architected review").

Or you can be pragmatic, the cloud will allow you to do whatever you want, nobody is stopping you.

> Then they innovate "serverless" stuff

Serverless does not replace everything, there are some things it's good for, but not everything, if you use it for something that it's not useful for then that's an issue with your judgement, not with the fact that it exists as an option.

> And don't even get me started on how much work by this time has gone to build the infra-as-code to manage the rube golderg machine, you'll (seriously) have more lines of CDK code than app logic (that was slower to develop & harder to debug than your actual app code per line).

I have thousands of lines of YAML and MD that I can give to anyone who is skilled enough to copy and paste and that person can 100% replicate the environment without me helping them once.

If a component fails, I can just rebuild it very quickly and 100% accurately by just re-importing the Cloud Formation code, if I need an environment that is 100% same as the one that I already have, I just need to re-run a few scripts and I have it.

What credible alternative can you propose that is better than infrastructure as code?

> They've probably never costed / self administered a server in production use, or if they have, they know to keep quiet.

I could make a similar snarky remark about sys-admins not understanding that the world is not static, however I would instead focus on the fact that acquisition of hardware is nowhere near the total cost of the infrastructure.

According to a 3 second Google search, sys-admins in the US can cost around 75k-140k. Sys-admins can also slow down development by having to perform more manual actions and having to be more over protective, that adds additional costs, because you will still have to pay your idle developers while they are waiting for the sys-admin to action the ticket where they requested write access to their own home folder.

Re: AWS costs every programmer should know (2019)

#159
post #158
post #4

It's interesting how AWS can keep so high prices on these. But it's just the beginning, the real money comes from when they convince you to run over a dozen vms/containers (all needing storage etc of course). You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. You can then get a pat on the h…

> You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. Don't believe this has anything to do with the cloud, if you want the same thing with your own data center, it's going to be 18x there as well. > You can then get a pat on the head ("pass AWS well-architected review"). Or you can be pragm…

You could spend less than 18 x AWS cost, because AWS compute is so expensive, depending what hosting you chose. A self owned DC is isn't the best option for most. You might also skip the overkill (for most cases) AZ redundancy because it wasn't pushed on you. You might even go with a much more managed platform, like the Heroku like ones. Depends on what you build and for who.

I'm not knocking IaC, but unnecessarily huge amounts of IaC to manage unnecessarily complex AWS service infra after you can't afford the monolith + db model and get roped to dynamodb, lambda@edge, API gateway, step functions, etc all requiring IAM roles, security groups, observability tooling, cicd & version control complexity, config services etc etc all the downsides you read about in microservice horror stories. And you can't even ssh in and strace or tcpdump the stuff like you could with your own microservices, they're black boxes.

I also don't want separate sys admins, at least the bad kind you describe. But having your own servers, or knowing what they cost, doesn't mean you would.

Re: AWS costs every programmer should know (2019)

#160
post #114
post #12

Earlier quoted context omitted.

If you're not testing az failovers you're probably just wasting money, like untested backups... But it's true that most people don't know this and aren't testing it because they don't know that az failovers don't automatically just work(tm). And the second downside of course would be asymmetry between the envs and divergence in the IaC etc resulting in more complexity and engineering work. (Of course you're probably…

Are AZ failovers not done automatically? Where can I read more about that?

They are from user pov just separate networks where you can deploy separate copies of services, replicated db cluster nodes &whatnot. Each service handles (knock on wood) an az becoming unreachable/slow/crazy independently. Which can become fun with service interdependencies, and a mix of your self implemented services + AWS provided ones.

There's high level description at https://aws.amazon.com/about-aws/global-infrastructure/regio...

Post reply on HN