I've worked at at least one company where we found using Direct Connect to a data center somewhere and egressing via that data center instead of amazon saved us significant bandwidth costs.
AWS Customers Rack Up Hefty Bills for Moving Data
81–90 of 252 posts
Re: AWS Customers Rack Up Hefty Bills for Moving Data
#82Earlier quoted context omitted.
The point is to make it easy to move data in and hard to move it out.
Someone should set up a big fat pipe right outside of Amazon data centers with free unlimited transfers, get data on behalf of customers copied to hard drives from AWS, and then attach those hard drives to the Uber pipe. I bet that service could work for a short & glorious moment in time.
That would still require egress.
Re: AWS Customers Rack Up Hefty Bills for Moving Data
#83I've actually been working on a library to help mitigate cloud storage lock-in. The idea is to treat cloud storage providers like disks are treated in RAID. For example, you have 3 separate cloud providers. Cloud providers 1 and 2 have every other byte of data striped across them. Cloud provider 3 has parity data. To pull a file you can only need 2 of the 3 cloud providers. If you don't like how a cloud storage provi…
Are you doing this to replace a CDN? There are already 3rd party CDNs like CloudFlare. If you are doing it as a replacement for traffic within an AWS region and availability zone, it seems like you will be both more expensive and have much higher latency. Or is the application something else entirely?
It's something else entirely. It's a mixed cloud approach combining the storage offerings of Azure, Google Cloud, and S3 providers. The idea is not to trust one cloud provider to provide fair pricing and proper redundancy. Right now I'm mirroring RAID 0,1, and 3. Applying RAID 3 to the cloud is going to give you higher latency and more processor and memory usage because the file has to be reassembled on the client machine. However, if you apply RAID 1 to the cloud your latency is similar because each cloud provider has the full file. In the case of RAID 1 the library will upload a full copy to each cloud provider and will download files by trying providers until one succeeds. If you only use two providers your pricing is usually the same because geo-redundancy in storage providers is often twice the cost and you're getting geo-redundancy built in by having multiple providers in different regions. RAID 3 is actually cheaper because you have geo-redundancy, but you're only storing 1.5 times as much data.
Re: AWS Customers Rack Up Hefty Bills for Moving Data
#84Earlier quoted context omitted.
Disclaimer: I worked on an AWS service team. This is, oddly enough, similar to a debate people have about consumers TV or Internet: should pricing be "unlimited" or "a la carte"? AWS is combining all your networking charges into one lump "outgoing data transfer" fee. So it's heavily marked up in comparison to what they're paying for the outgoing data transfer, and you're not sure how much is profit vs. whether it's g…
> If there's a compelling case for doing it differently, someone should do it and see if it works. Cloudflare doesn't charge for bandwidth. I always throw cloudflare on top of anything I do, not because I really need a CDN or anything, but because the bandwidth cost would bankrupt me otherwise. The ceo of cloudflare gave the rationale on why they don't charge: > There’s a fixed cost of setting up those peering arrang…
[deleted]
Re: AWS Customers Rack Up Hefty Bills for Moving Data
#85Is there a plausible explanation why egress fees from cloud providers costs around $0.1/GB? "Traditional" server providers such as Hetzner are able to offer bandwidth at orders of magnitude lower price (eg. $1.1/TB). I understand that cloud providers may have better interconnects or better uptimes, but that doesn't justify the magnitudes higher pricing.
> Is there a plausible explanation why egress fees from cloud providers costs around $0.1/GB?
Isn't DigitalOcean a cloud provider ($0.01/GB)?. Isn't Oracle a cloud provider (first 10 TB free, $ 0.0085 after)? Isn't OVH a cloud provider (free bandwidth)?
Re: AWS Customers Rack Up Hefty Bills for Moving Data
#86Earlier quoted context omitted.
Someone should set up a big fat pipe right outside of Amazon data centers with free unlimited transfers, get data on behalf of customers copied to hard drives from AWS, and then attach those hard drives to the Uber pipe. I bet that service could work for a short & glorious moment in time.
That would still require egress.
I was going to say it’d be an improvement but I can’t actually make sense of the AWS Snowball proving
Re: AWS Customers Rack Up Hefty Bills for Moving Data
#87I've worked at at least one company where we found using Direct Connect to a data center somewhere and egressing via that data center instead of amazon saved us significant bandwidth costs.
amazon still charges for traffic through direct connect(50%), dont they?
Re: AWS Customers Rack Up Hefty Bills for Moving Data
#88Earlier quoted context omitted.
The point is to make it easy to move data in and hard to move it out.
Someone should set up a big fat pipe right outside of Amazon data centers with free unlimited transfers, get data on behalf of customers copied to hard drives from AWS, and then attach those hard drives to the Uber pipe. I bet that service could work for a short & glorious moment in time.
Amazon beat you to it: https://aws.amazon.com/snowball/