Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Slashing data transfer costs in AWS

bitsand.cloud

Post reply on HN
Print view View original

161–170 of 268 posts

Re: Slashing data transfer costs in AWS

#161

Post by dzikimarian »

overstay8930 wrote: post #152
chongli wrote: post #138

Earlier quoted context omitted.

Both on prem and cloud require people familiar with them and cloud-engineers are in no way cheaper. I think the real story is a bit sordid: office politics. On-prem and cloud are different skillsets. Companies that have been around for a while can end up with both on-prem and cloud experts who end up competing with each other, often on separate teams. Throw in some slick consultants from Amazon who are able to bend t…

Cloud engineers can do the job of 4-5 on-prem people. Our AWS devs don't need to be BGP or ZFS experts, they just need to be AWS experts.

Well our on-prem team doesn't need AWS pricing calculation and optymization expert, so there's that :-)

Re: Slashing data transfer costs in AWS

#162

Post by kevincox »

danielklnstein wrote: post #20
quickthrower2 wrote: post #19

This is a loophole. Hitting some loss leader at AWS, but if everyone only buys the $1 hotdog and nothing else then the $1 hotdog gets removed.

I'm not sure how this could be removed - the fundamentals behind it are basic building blocks of S3. Maybe raising the cost of transient storage? e.g. If you have to pay for a minimum of a day's storage - but even if that was the case this would still be cost-effective, and at any rate it seems very unnatural for AWS to charge on such granularity. + I would guess that S3 is orders of magnitude more profitable for AWS…

It would be fairly easy to change the pricing policy. GCP did something similar for cross-region https://cloud.google.com/storage/pricing-announce#network. This is pretty severe because it seems to affect all reads. However I can imagine an alternate implementation where the source AZ is tracked when data is written and egress fees are charged when the data is read (as if the data was always stored in the source AZ). This could even be done more complexly such as only charging the first time data is read in another AZ. Once you read once it is free as-if it is now cached in that new AZ forever. Another option would just be raising the minimum storage duration so that it basically costs all or most of what the data transfer would.

It would definitely piss a lot of people off as it is adding to their bill, but it could likely be done in a way that makes exploiting this for just data transfer not worth it without adding huge costs to most "real" use cases.

Re: Slashing data transfer costs in AWS

#163

Post by project2501a »

overstay8930 wrote: post #134
lijok wrote: post #109

Earlier quoted context omitted.

No key person risk management -> no risk register -> no management. Your friends company will fail regardless of poor sysadmin decision making or not. They need to hire competent management ASAP.

This is basically the logic of people who say the cloud is too expensive, you have to ignore so many things to make being on premise logical. Basically you are lying to yourself if you think you can run a datacenter cheaper and better than Amazon or Microsoft can, because if you can you are just making huge sacrifices somewhere (usually time, which is why reddit sysadmins complain about how much work they have while…

you must be management, cuz

1. you think it's the sysadmin's fault

2. there are no competent sysadmins out there

Re: Slashing data transfer costs in AWS

#164

Post by dzikimarian »

falserum wrote: post #137
dzikimarian wrote: post #113

Earlier quoted context omitted.

Apparently we're doing the impossible for over 12 years now. Who knew? Some people act like it's some kind of black magic. It's not. We've some customers in our DC and some on AWS for various reasons. AWS isn't less problematic. AWS is about 10x more expensive. Both on prem and cloud require people familiar with them and cloud-engineers are in no way cheaper. Only meaningful problem is that on-prem requires some up f…

Cloud clearly makes sense for: - small business with at least some reliability expectation, and little to none IT expertise - huge workload requirement volatility - having someone else to blame - solution is already working in cloud, with teams being very comfortable there, and perceiving on-prem as “enemy” (analogy: forcing devs to rewrite stuff from haskell to java) - that extra cost is small budget line for you On…

Depending how small is small business, but I would probably go with VPS.

Re: Slashing data transfer costs in AWS

#165

Post by marcus0x62 »

steveBK123 wrote: post #80
marcus0x62 wrote: post #68

Earlier quoted context omitted.

I've never seen an IT team that couldn't spin up a VM in minutes. I have seen a bunch of teams that weren't allowed to because of ludicrous "change control" practices. Fire the managers that create this state of affairs, not the devops folks, regardless of whether you "go cloud" or not.

Teams are what they DO, not what they CAN DO.

Ok, but I’m not sure what that has to do with what I posted.

Re: Slashing data transfer costs in AWS

#166

Post by jonatron »

declan_roberts wrote: post #122
jonatron wrote: post #16

If you're a heavy bandwidth user it's worth looking at Leaseweb, PhoenixNAP, Hetzner, OVH, and others who have ridiculously cheaper bandwidth pricing. I remember a bizarre situation where the AWS sales guys wouldn't budge on bandwidth pricing even though the company wouldn't be viable at the standard prices.

That’s very unusual, I think. Transfer cost seem to be something most people can negotiate.

I hadn't really thought about it much, but googling it looks like there's a discount programme for a committed spend of around $1M/year. For a small company, that's a lot of money, and it was an unusually large amount of bandwidth for the size of company. I suppose it makes sense now I know they're interested in companies spending that sort of money.

Re: Slashing data transfer costs in AWS

#167

Post by lillecarl »

overstay8930 wrote: post #160
lillecarl wrote: post #153

Earlier quoted context omitted.

Incorrect, there's automation for on-prem or smaller providers as well, just because you're unable to doesn't mean it's the truth. How do you think AWS or Cloudflare is built?

You're arguing semantics, you are on massive copium if you think that automation is useful to anyone. Go ahead and set up cryptographic attestation (or just try and interface with a TPM ffs) for your apps on Hetzner to decrypt customer data and see how impossible it is.

It's useful to a lot of people, not everyone has the same requirements. I'm running SecureBoot and LUKS encryption on my Laptop. If there's a TPM2 interface in whatever you can use the same thing. Add an immutable Linux distro and Kubernetes and you'll cover pretty many use cases. Not everyone has the same requirements, $BIGCLOUD makes things easier, but you also pay top dollar for it. AWS funds Amazon

Re: Slashing data transfer costs in AWS

#168

Post by robertlagrant »

rescbr wrote: post #157
steveBK123 wrote: post #78

Earlier quoted context omitted.

You CAN, but of course that's not what AWS steers you to. Once your org has gotten to that step, it's been so steered by AWS staff it's hard to imagine suddenly finding sense and building with open standard stuff. Very few AWS shops I have encountered avoid the siren call of various AWS-only or AWS-specific services, which they then become heavily ingrained in.. Generally I do think its mostly about transforming CapE…

I was one of those AWS people that worked with Financial Services customers. We (at least my team) were always pushing for a minimum of modernization when architecting migration projects - even a simple move to containers, managed by whatever orchestrator they want to run. It helps enormously on costs at least by just taking off so many overhead and overprovisioned VMs from the roster of migration candidates. More of…

Yep I did a (tiny small by your scale) lift and shift+ - basically take a thing that ran as a regular process with a database attached, and containerised (and pen tested, but that's another story) the thing and plugged it into a cloud-managed database. It worked great.

Re: Slashing data transfer costs in AWS

#169

Post by rescbr »

jonatron wrote: post #166
declan_roberts wrote: post #122

Earlier quoted context omitted.

That’s very unusual, I think. Transfer cost seem to be something most people can negotiate.

I hadn't really thought about it much, but googling it looks like there's a discount programme for a committed spend of around $1M/year. For a small company, that's a lot of money, and it was an unusually large amount of bandwidth for the size of company. I suppose it makes sense now I know they're interested in companies spending that sort of money.

The trick is to use CloudFront if possible, even if not caching, just passing through requests.

Standard discounts start at 10 TB, which is not that much.

If not using HTTP, then it's a no starter.

Re: Slashing data transfer costs in AWS

#170

Post by jjav »

ttul wrote: post #107
qvrjuec wrote: post #104

Earlier quoted context omitted.

>someone who will not treat their datacenter like a home lab What does this mean? They steal company resources for themselves, or just configure things incompetently?

Incompetence. Take my friend’s company for instance. They were frustrated paying $60K/mo to Amazon so their brilliant sysadmin bought $600K of servers and moved them into a cheap colo. Over Christmas, everything died, and the brilliant sysadmin was on holiday. Nobody could get things going again for many days and so their entire SaaS business was failing. They lost a lot of business and trust as a result. The sysadmi…

> The sysadmin is now gone and they are back on AWS.

This story has nothing to do with AWS or on-prem.

It's a story about incompetent management allowing a single human point of failure. If they don't change that, they'll have the same problem wherever they go.

Post reply on HN