Live data from Hacker News

Why cloud bandwidth is so obscenely expensive and what you can do about that

kerkour.com

111–120 of 164 posts

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#111
post #34

One thing to keep in mind - if you accidentally run up a bill because of a mistake, there’s a good chance you can reach out to support and they will credit your account. $5k may be a lot to an individual, but the cloud provider’s costs for that service are significantly less and they shouldn’t mind forgiving the charge. Case in point, when the author deleted his account and switched providers, the old host didn’t cha…

That’s still a lot of hopes and prayers you do not end in financial ruin. Instead, the cloud providers could offer prepaid credits/billing maximums to let individuals sleep at night. I would love to run my side project off of AWS to gain the experience, but no way I feel comfortable with a potentially unlimited liability because I configured something poorly. My joke project can take the uptime hit if it means I know…

At least AWS sort of provides tooling to approximate this. I have some alerts setup in CloudWatch for anomalous Network Out, estimated charges exceeding a threshold, and a raw Network Out limit that if exceeded shuts instances down.

Overtime, bandwidth costs have gone down for AWS. What's finally causing me to think about jumping ship (surprised OP didn't mention OVH's unmetered stuff) is their upcoming pricing per month for IPv4 addresses. The alternative is proxying through CloudFlare, which I don't particularly want to do.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#112
post #51

I wonder is There also is Hetzner and their dedicated servers. You pay for a server with a 1Gb/s connection and all egress is free. Unfortunately, I can't recommend them as each time I've tried to create an account, their abuse detection systems banned my account even before I had the occasion to enter a payment method. related to I've myself won the jackpot in the past on Netlify where I forgot to put a sleep in a w…

[dead]

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#113
post #101

Cloudflare R2 (their S3 clone) has free egress. We are in the middle of migrating about 5TB of fairly active data from Google Cloud Storage to R2. For years now, most of our cloud bill has been egress from GCS. Activity spiked recently and doing something about it became urgent. R2 has an S3-compatible API (we're using the Amazon-provided S3 Java SDK). Storage is 25% cheaper than GCS standard. Making our code multi-p…

General question, can we just call object storage object storage? If there's going to be another provider after already remembering S3, B2, now R2, on top of the various aliases for products like virtual servers, shared hosting, VLANs, and other things people are presenting as new products with the most random of abbreviations, I'm really going to lose track.

we need something like [this](https://www.azureperiodictable.com/), but across all cloud providers

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#114
post #65

I’ve just never seen bandwidth be a significant part of the bill for any sass I’ve built/worked for. I’m super curious about this. What kind of sass businesses use huge amounts of bandwdith but don’t provide enough commercial value to the customers to make it viable? Can folks with first-hand experience please share here?

Why does it need to be sass?

My problem was hosting too many photos on the landing pages. Like literally like 60 photos on 4 different landing pages.

The page looked fantastic and people would email me that I have the best site on the internet. But if I ever went viral, I'd always go down. I have since changed so the pages only have like 20 pictures now.

I only now became profitable, and it has little to do with anything other than waiting for word of mouth to take over.

This might seem like a crappy business, but now its a few thousand per year passive profit. Who knows what the future holds. Its year 7 and the growth is exponential despite me ending all marketing.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#115
post #51

I wonder is There also is Hetzner and their dedicated servers. You pay for a server with a 1Gb/s connection and all egress is free. Unfortunately, I can't recommend them as each time I've tried to create an account, their abuse detection systems banned my account even before I had the occasion to enter a payment method. related to I've myself won the jackpot in the past on Netlify where I forgot to put a sleep in a w…

> Is there some kind of blacklist if you do something like that?

No; but oddly enough, the people who "do things like that" generally tend to come from certain countries — I guess countries with cultures that don't place much weight on the concept of "incurring a debt of honor" by consuming someone else's resources without them ever knowing about it.

So most systems don't generally need a big, manually curated and ever-growing blacklist; they just need to block registrations from IP addresses / ASNs of ISPs headquartered in these countries; and/or block payment attempts from credit cards issued by banks headquartered in these countries. That immediately stops 90% of such abuse.

And of the remaining 10%, half of it is still people from those same countries — just using foreign IPs through residential-botnet VPNs, and stolen credit cards they purchased on scammer forums. Blocking these is a bit of an art, but it's possible: there's always patterns to the requests themselves, often because those same scammers try to solve all their problems with money, and so have also purchased scam-site kits to run on the hosting they acquire — things like cryptocurrency "drainers." If you're a VPS hosting provider, you can just detect these by the SHAs of the files; if you're a dedicated hosting provider with no access to customers' disks, it's still pretty easy to pick these out by the outbound signature of the network traffic they generate — as they almost always rely on making requests to particular third-party SaaS information systems, that you can turn into an IDS detection fingerprint.

(I'm personally in a different position in this ecosystem — my company operates one of the public informational SaaS services that these scam-site kits like to use. From my company's perspective, these scammers are perfectly normal paying customers, not intent to scam us... but we don't want these people as customers, so we still detect this fraudware by the fingerprint it makes in our API request logs, and permaban the users who deploy such kits by every fingerprinting metric we can.)

---

Though, on another note, I suppose you could call the observational "IP reputation" metrics gathered by providers like https://www.ipqualityscore.com/ something like a blacklist — and I'm sure hosting providers like Hetzner check your "IP score" before letting you register. But these aren't blacklists in the sense of being manually curated.

Instead, what these providers curate is something like a distributed version of an SSHGuard blocklist: a bunch of the provider's own "observer nodes", all over the world, observe what IPs are hitting them with DDoSes and other botnet-like activities, and consider these IPs temporarily compromised for as long as that activity persists (because any device infected by a botnet can potentially be repurposed as a part of a residential-proxy VPN network — and that means that any traffic observed to come from such a device, can't be trusted to be originating from that device.)

IIRC, these providers will also "go undercover" to buy access to both commercial and residential-botnet VPNs; cycle through them to find out what all the available exit-node IP addresses are from a client's perspective — and then mark all these as compromised as well.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#116

Cloudflare R2 (their S3 clone) has free egress. We are in the middle of migrating about 5TB of fairly active data from Google Cloud Storage to R2. For years now, most of our cloud bill has been egress from GCS. Activity spiked recently and doing something about it became urgent. R2 has an S3-compatible API (we're using the Amazon-provided S3 Java SDK). Storage is 25% cheaper than GCS standard. Making our code multi-p…

You kinda pay in reduced performance. For example, civitai uses r2 for delivering the images, and it frequently doesn't load at all for me or very slowly. When your users have to use a VPN to move their traffic to another Cloudflare data center so the site actually loads, something is wrong...

Heh, this entire threat is "Fast, cheap, reliable... Pick two" where everyone wonders why all three are so much more expensive.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#117
post #51

I wonder is There also is Hetzner and their dedicated servers. You pay for a server with a 1Gb/s connection and all egress is free. Unfortunately, I can't recommend them as each time I've tried to create an account, their abuse detection systems banned my account even before I had the occasion to enter a payment method. related to I've myself won the jackpot in the past on Netlify where I forgot to put a sleep in a w…

Hetzner default process requires contacting a human before your account is ready for any use.

Looks like they have some exceptions, so some people manage to just make an account and use it. And looks like there's some requirement on German law that forces their hands. But I don't think the OP got flagged for anything.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#118
post #111

Earlier quoted context omitted.

That’s still a lot of hopes and prayers you do not end in financial ruin. Instead, the cloud providers could offer prepaid credits/billing maximums to let individuals sleep at night. I would love to run my side project off of AWS to gain the experience, but no way I feel comfortable with a potentially unlimited liability because I configured something poorly. My joke project can take the uptime hit if it means I know…

At least AWS sort of provides tooling to approximate this. I have some alerts setup in CloudWatch for anomalous Network Out, estimated charges exceeding a threshold, and a raw Network Out limit that if exceeded shuts instances down. Overtime, bandwidth costs have gone down for AWS. What's finally causing me to think about jumping ship (surprised OP didn't mention OVH's unmetered stuff) is their upcoming pricing per m…

It's still absurd having to hand-craft that type of protection against racking up enormous bills, especially with cloud providers now actively promoting their various "free" tiers to hobbyists.

It reminds me of EU roaming before the regulatory cap, when a single video call close to the border could cost you thousands of Euros.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#119
post #111

Earlier quoted context omitted.

That’s still a lot of hopes and prayers you do not end in financial ruin. Instead, the cloud providers could offer prepaid credits/billing maximums to let individuals sleep at night. I would love to run my side project off of AWS to gain the experience, but no way I feel comfortable with a potentially unlimited liability because I configured something poorly. My joke project can take the uptime hit if it means I know…

At least AWS sort of provides tooling to approximate this. I have some alerts setup in CloudWatch for anomalous Network Out, estimated charges exceeding a threshold, and a raw Network Out limit that if exceeded shuts instances down. Overtime, bandwidth costs have gone down for AWS. What's finally causing me to think about jumping ship (surprised OP didn't mention OVH's unmetered stuff) is their upcoming pricing per m…

Yeah, again "sorta" "approximates".

AWS is famous for making those alerts hard to set and error-prone. Honestly, I don't understand why people (instead of large companies) use them at all.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#120
post #105

Earlier quoted context omitted.

> Cloud sells egress bandwidth at an enormous markup. Meanwhile ingress is free. If this were a technical limitation ingress would be billed too because there's nothing asymmetric about the Internet. Symmetry is the entire problem. 1) to maintain settlement-free peering, you need traffic-symmetry. Amazon — on their own - transmits far more data than they receive. They benefit far more on this account from ingress tra…

Assuming that's all true, it still doesn't explain the magnitude of the markup. VPS providers can do $0.01/gig outbound almost across the board. This includes some very large ones, and because bandwidth is cheap there people tend to host a lot of high bandwidth things there. GCP, AWS, etc. are roughly 10X more expensive than this. Are they 10X better? The cost is very high. I can't stress this enough. A terabyte outb…

Note: at no point in my comment did I say I thought the specific dollar amount of the bandwidth markup was reasonable or unreasonable. I pointed out reasons why differential bandwidth pricing for ingress/egress exists at the major cloud providers, when you claimed there were no technical reasons.

Also note: VPS providers generally do not exist at the scale where they could or would pursue settlement-free peering. They buy transit from one (or, hopefully, more) upstream providers who have to worry about the first point I made. The VPS providers still have to worry about point #2, but they can do that by courting different kinds of customers, without the baggage (in the context of running a network) of a big legacy content distribution business that both Amazon and Google have.

Post reply on HN