Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS costs every programmer should know (2019)

david-codes.hatanian.com

Post reply on HN
Print view View original

51–60 of 163 posts

Re: AWS costs every programmer should know (2019)

#51

Post by dijit »

moduspol wrote: post #46
yakshaving_jgt wrote: post #43

Earlier quoted context omitted.

Self-hosting and infrastructure-as-code aren't mutually exclusive.

This doesn't feel like a meaningful distinction. You can go from an empty cloud account to a fully-provisioned application stack in minutes with the right infra-as-code. Provisioning infrastructure at that level is mutually exclusive with self-hosting. But true, even with self-hosting, you can use infra-as-code at various levels (Ansible, OpenStack, self-hosted Kubernetes). It's just not really comparable to having y…

Who cares when you can accidentally bankrupt your company with the wrong keystroke?

Time to spin up is nice for prototypes, it can be very freeing.

The mistake that people make is assuming AWS can host things better than they can, since they’re only mere mortals: it leads to a sort of learned helplessness where you assume the cost of a thing is it’s true value and not an egregious markup.

My biggest gripe with these cloud providers (except maybe GCP) is that they promise less ops work and it can be true in the beginning; but over time the ops work becomes basically the same burden except esoteric to the provider.

It’s IBM mainframes again, with specialists on IBM pushing more IBM because it’s their bread.

But overall, if you know your problem then time to create an instance isn’t very valuable, believe it or not: the majority of workloads are not excessively elastic, at least the upper bound is not unlimited like many people seem to claim.

Apart from that !cloud != self-hosted. There’s plenty of hardware providers that can get you a dozen machines in under an hour; even discounting virtual host providers like vultr and Tulsa.

Re: AWS costs every programmer should know (2019)

#52

Post by russellendicott »

fulafel wrote: post #20
philliphaydon wrote: post #18

Earlier quoted context omitted.

If you’re building your own infrastructure in a data center then sure you absolutely want to test your redundancy. But with AWS it’s a checkbox. It’s transparent to you and your applications. The infrastructure to host in multiple AZs is already in place. The only real issue with MultiAZ is the failover in RDS depending on the database you use could be seconds or 10s of seconds.

IME not so in the real world, you'll have accidental state in your distributed system outside the DBs. You'll have some stuff that actually always runs in one AZ in normal circumstances and your integration partner in another org has whitelisted only its IP. Etc etc, everything that is untested will find ways to conspire to rot. Especially if you haven't learned by seeing these bugs so you can avoid them. Also you wo…

Dealing with this now for a client. Can't test Redshift AZ relocation feature because there's no way to simulate AZ failure. Only safe bet is full multi region with DNS switcheroo.

Re: AWS costs every programmer should know (2019)

#53

Post by maccard »

nathants wrote: post #35
arinlen wrote: post #28

Earlier quoted context omitted.

> scale to zero whenever possible (lambda) AWS pricing is great because you aren't price-gouged if you don't use it? And your statement isn't really true. If you use AWS Lambdas do implement something like an HTTP endpoint, you still need to pay for stuff like AWS KMS and API Gateway even if no client hits you with a request. - use minimal infra (lambda managing ec2 with route53 health check) Above free tier AWS Lamb…

aws pricing is literally a catalog. what and how much you buy is kind of the whole thing. a subset of the catalog is good. aws core primitives are: - flexible - reliable - robust if your service succeeds, you will definitely want to move some components out of aws, but not all of it. aws can always remain the control plane. if you don’t need what aws offers, don’t use it. they offer it at a price, and it is what it i…

I don't agree about avoiding the services. If you're mega successful down the line you can likely afford to do the migration when it's expensive, but the services are what save you_time_, and initially money too. I can spin up an RDS server that is secure and production ready in less than 5 minutes that quite frankly I don't need to touch for a very very long time. If we need more power, I can restart the instance with a bigger type, or if we've overspecced I can use a smaller one. Compared to running your own instances on ec2 there's a few hours of poking around to do things like password handling, backups, logs, not to mention you're now responsible for managing updates and the likes. It's a false economy to avoid them IMO.

That's not to say you should immediately pivot to all in AWS all the time but like any engineering be aware of the price you're paying to build Vs buy

Re: AWS costs every programmer should know (2019)

#54

Post by moonchrome »

philliphaydon wrote: post #18
fulafel wrote: post #12

Earlier quoted context omitted.

If you're not testing az failovers you're probably just wasting money, like untested backups... But it's true that most people don't know this and aren't testing it because they don't know that az failovers don't automatically just work(tm). And the second downside of course would be asymmetry between the envs and divergence in the IaC etc resulting in more complexity and engineering work. (Of course you're probably…

If you’re building your own infrastructure in a data center then sure you absolutely want to test your redundancy. But with AWS it’s a checkbox. It’s transparent to you and your applications. The infrastructure to host in multiple AZs is already in place. The only real issue with MultiAZ is the failover in RDS depending on the database you use could be seconds or 10s of seconds.

>But with AWS it’s a checkbox. It’s transparent to you and your applications. The infrastructure to host in multiple AZs is already in place. The only real issue with MultiAZ is the failover in RDS depending on the database you use could be seconds or 10s of seconds.

Have you actually seen this work on your project in practice ? Like a region go down and another region picks up automatically and it kept working just by switching a checkbox ?

Re: AWS costs every programmer should know (2019)

#55

Post by russellendicott »

fulafel wrote: post #16
blackoil wrote: post #11

Earlier quoted context omitted.

Hire a better engineer/architect.

Indeed, this is often the next step. The new architect must have lots of AWS certifications. With the new architeture, the long term cost savings will be exponential, compared to the current projected cost curve! This can optionally combine with the Kubernetes scenario.

Exactly, then you need at least 1 architect per app. I wrote an article about this problem: http://blog.bytester.net/posts/cloud-talent/

Re: AWS costs every programmer should know (2019)

#56

Post by dinedal »

nathants wrote: post #35
arinlen wrote: post #28

Earlier quoted context omitted.

> scale to zero whenever possible (lambda) AWS pricing is great because you aren't price-gouged if you don't use it? And your statement isn't really true. If you use AWS Lambdas do implement something like an HTTP endpoint, you still need to pay for stuff like AWS KMS and API Gateway even if no client hits you with a request. - use minimal infra (lambda managing ec2 with route53 health check) Above free tier AWS Lamb…

aws pricing is literally a catalog. what and how much you buy is kind of the whole thing. a subset of the catalog is good. aws core primitives are: - flexible - reliable - robust if your service succeeds, you will definitely want to move some components out of aws, but not all of it. aws can always remain the control plane. if you don’t need what aws offers, don’t use it. they offer it at a price, and it is what it i…

> if your service succeeds, you will definitely want to move some components out of aws, but not all of it. aws can always remain the control plane.

Do you factor in the costs of dev time in a cross cloud environment at all? Nevermind all the AWS specific tooling and code you write, which needs to be thrown away and re-done to accomplish this?

Don't get me wrong, I love AWS when I'm not paying the bill, it makes my job of shipping features quickly so much easier. It's good and it works well. But it's a little crazy to think it's "cheap" and to ignore the costs of vendor lock in.

Re: AWS costs every programmer should know (2019)

#57

Post by nathants »

maccard wrote: post #53
nathants wrote: post #35

Earlier quoted context omitted.

aws pricing is literally a catalog. what and how much you buy is kind of the whole thing. a subset of the catalog is good. aws core primitives are: - flexible - reliable - robust if your service succeeds, you will definitely want to move some components out of aws, but not all of it. aws can always remain the control plane. if you don’t need what aws offers, don’t use it. they offer it at a price, and it is what it i…

I don't agree about avoiding the services. If you're mega successful down the line you can likely afford to do the migration when it's expensive, but the services are what save you_time_, and initially money too. I can spin up an RDS server that is secure and production ready in less than 5 minutes that quite frankly I don't need to touch for a very very long time. If we need more power, I can restart the instance wi…

definitely. some of the services are great. they are just a lot more expensive. in the context of making aws cheaper, avoiding services is the play.

Re: AWS costs every programmer should know (2019)

#58

Post by cutthegrass2 »

raffraffraff wrote: post #37
philliphaydon wrote: post #10

Earlier quoted context omitted.

> You need to be triply redundant on 3 availability zones, (3x) both with the RDS db cluster and app containers (2x) . And then have separate dev/staging/prod envs (3x). That's 18x. Why would you care about the redundancy on staging / dev? Just making up things to inflate AWS costs now.

Even if you don't want it sometimes you're forced to run multiple AZs (eg: EKS requires 2x). But that 18x figure is nuts. VPCs, AZs, subnets, IAM etc are free. The cost comes from what you deploy into them. So separate environments don't have to be as expensive as production. You can scale them to zero, use smaller computer instances, run self-managed versions of expensive stuff (like DBs) or simply run small single-…

Whilst the VPC itself is free, they get you on the NATGW - which you probably need along with the VPC in most cases.

Re: AWS costs every programmer should know (2019)

#59

Post by moduspol »

yakshaving_jgt wrote: post #50
moduspol wrote: post #46

Earlier quoted context omitted.

This doesn't feel like a meaningful distinction. You can go from an empty cloud account to a fully-provisioned application stack in minutes with the right infra-as-code. Provisioning infrastructure at that level is mutually exclusive with self-hosting. But true, even with self-hosting, you can use infra-as-code at various levels (Ansible, OpenStack, self-hosted Kubernetes). It's just not really comparable to having y…

Perhaps I am confused or we are talking about different things and consequently talking past each other, but I can literally open a new AWS/GCP/Hetzner/DO account, plug the credentials into my local configuration (in code), and then run a command with NixOps to provision an entire network of machines with custom specifications, and to automatically install all the software I need on those machines. Perhaps you aren't…

It sounds like we're using different definitions of the word "self-hosted." If you have your own on-premises lab or colo rental, you're not using AWS/GCP/Hetzner/DO, and you've got a lot of undifferentiated heavy lifting before your NixOps kick in (including maintenance going forward).

If your point is that you can avoid a serverless architecture, still use cloud, and still use infra as code: of course you can. We've got to be disagreeing on what "self-hosted" means. OP criticized the cost and complexity of deploying EC2 instances and RDS databases across AZs, so presumably infra-as-code wouldn't help him here. OP didn't present an alternative solution, but reading between the lines is to not use cloud infrastructure (e.g. on-prem or colo).

Re: AWS costs every programmer should know (2019)

#60

Post by nathants »

dinedal wrote: post #56
nathants wrote: post #35

Earlier quoted context omitted.

aws pricing is literally a catalog. what and how much you buy is kind of the whole thing. a subset of the catalog is good. aws core primitives are: - flexible - reliable - robust if your service succeeds, you will definitely want to move some components out of aws, but not all of it. aws can always remain the control plane. if you don’t need what aws offers, don’t use it. they offer it at a price, and it is what it i…

> if your service succeeds, you will definitely want to move some components out of aws, but not all of it. aws can always remain the control plane. Do you factor in the costs of dev time in a cross cloud environment at all? Nevermind all the AWS specific tooling and code you write, which needs to be thrown away and re-done to accomplish this? Don't get me wrong, I love AWS when I'm not paying the bill, it makes my j…

vendor lockin is fine. it’s like having a great employee, but not leveraging them fully because bus factor and cogs. you should be sad if a great employee leaves, or your vendor shuts down. choose both carefully, and trust them.

my only experience expanding out of aws is to cloudflare for egress. i’m sure it can be rough, but so can anything solved with dev time.

expanding out of aws with individual tightly scoped components should be easier than general cloud migration. just bandwidth egress. just cheaper cpus for background processing. just whatever.

keep the complicated stuff on main cloud where it belongs.

Post reply on HN