Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why cloud bandwidth is so obscenely expensive and what you can do about that

kerkour.com

Post reply on HN
Print view View original

71–80 of 164 posts

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#71

Post by sroussey »

toast0 wrote: post #32
zeroCalories wrote: post #9

Earlier quoted context omitted.

??? This is obviously true. The bandwidth available in a geographic location is, practically speaking, limited. Roads are cheap to pave, but if there was congestion 24/7 and you paved all you could, then you would need to start rationing road usage. For a business the best way to ration is to charge more. Solving this problem requires a massive amount of engineering work, which would also justify the increased cost.

Bandwidth in a geographic location is certainly limited... there's only so many fiber strands available, and fiber transceivers only go up to a certain speed, +/- wave division; there's costs of transceivers and what not. But in most places where you'll find datacenters, the limit is rather high. It's also usually possible to run more fiber. If you take a look at the Seattle Internet Exchange [1] which is pretty publ…

There is so much dark fiber out there. Bothering to activate it is a different story.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#72

Post by dilyevsky »

AaronM wrote: post #4
varsketiz wrote: post #2

Serious Q for big Cloud spenders - how much have you negiotiated egress down from the list price?

Im not sure you can. At least at the scale we use AWS its a flat % off the top based on our spend

Ime you can when you apply some creative negotiation coupled with a multi-million $ spends. Usually to their peered (“partner”) networks though not generic transit

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#73

Post by jauntywundrkind »

Syonyk wrote: post #41
jauntywundrkind wrote: post #33

Earlier quoted context omitted.

If you want bandwidth, vps are quite cheap. 100Mbps averages to a a little over 32TB/mo. There there's tons of low CPU low ram systems with 32TB/mo transfer for cheap. Now under $10/mo: https://lowendbox.com/blog/hosteons-they-survived-dedipath-a... People should rack their own rpi5 or nucs, at least. There's various providers for both. And consider moving up from there. The remaining folks who will accept & rack a m…

Sure, I'll use a $5/mo DO Droplet (maybe $6/mo now for the new ones?) and their 1TB transfer pool if I just need raw bandwidth, but, tbh, not a lot of stuff is just pure bandwidth needs. And "capable cloud boxes for various purposes" start adding up. If I just wanted to shovel out a static website, sure. But I can do that a variety of other ways too. Once you start running "real" things, CPU and RAM use start climbin…

The point of this article was trying to liberate bandwidth, and I was highlighting how even common answers like your Droplet are far from the value-end of what's available. I showed an 18x better price point than what you would have gone to, on transfer alone (the 8GB memory is also a colossal advantage vs 0.5/1gb on a droplet).

I think for a lot of people, a nuc with dual m.2 drives is probably a much more affordable & realistic entry point that would probably survive even your fairly demanding gameserving use cases. And cost less than half as much.

Overall I'm very interested in a future where we do have scale out edge caches, where we do just need modest resources to maintain wide-scale connectivity. Many personal services are not at this scale or not this architecturally complex, but being able to have efficient scalable personal systems would be an interesting ends to move towards. Matrix would be a good example of a service that would be nice to have good fan-out for.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#74

Post by crazygringo »

Obscurity4340 wrote: post #59

Can anyone take a crack at why the hell you need a company and its computer farm 3000 miles away to merey sync your calendar, CalDav or whatever it is between two devices that are oerfectly capable of taking to each other or with a local-first mediating process or tool? I don't get what is about syncing things (particularly the basics like Contacts, Calendar/Reminders that requires you to send the data thousands of m…

Here's the answer: because it's not written for just 2 devices. It's written to sync however many devices you want -- 3 or 4 or 5 or whatever. Both your phones, your tablet, your laptop, and your watch.

And it has to sync when your devices might never actually be turned on at the same time. And you want it to sync when you're traveling, and you don't want to open your home firewall for security reasons.

So it requires a single, centralized, always-on server as the source of truth. And given the speed of light, whether that server is 3 miles or 3,000 miles away is largely irrelevant.

And writing additional code to sync directly when devices are on the same LAN and turned on at the same time, as a special case, is not only entirely redundant -- but you still have to sync with the server, since you generally want the information backed up as well in case you lose your devices, or they're stolen, or damaged. Using your examples of Contacts and Calendar, those are often some of the most important things in someone's life to keep synced remotely.

Does that answer your question?

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#75

Post by bombcar »

roncesvalles wrote: post #49
swatcoder wrote: post #35

Earlier quoted context omitted.

Heathen! Not everyone can afford that PhD in enterprise class hardware management you must have earned to pull this off. And besides, what are you going to do when you wake up tomorrow and you're the next FAANG? You'll never scale to 2B DAU on that box and we all need to be ready for that, just in case. I knew a guy who knew a guy who founded a FAANG, so it could happen to you too. You have to be ready! /s

But seriously, for scaling up, just have a plan for how to move things to the cloud quickly. Containerization and building everything on Kubernetes help.

The easiest plan for scaling up is "don't buy the literal biggest server out there" because if you at all MAKE money when you're scaling up, you'll be able to do a hardware swap to a massive device, which will buy you tons of time to work out what to do next.

Here's Let's Encrypt doing just that: https://letsencrypt.org/2021/01/21/next-gen-database-servers...

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#76

Post by bombcar »

Obscurity4340 wrote: post #59

Can anyone take a crack at why the hell you need a company and its computer farm 3000 miles away to merey sync your calendar, CalDav or whatever it is between two devices that are oerfectly capable of taking to each other or with a local-first mediating process or tool? I don't get what is about syncing things (particularly the basics like Contacts, Calendar/Reminders that requires you to send the data thousands of m…

Because it's simpler for them to develop, because they make the server the source of truth, and then they can sell it to you.

And for most people, it works fine, but when you hit edge cases you're dead in the water.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#77

Post by stefan_ »

zeroCalories wrote: post #9
onlyrealcuzzo wrote: post #6

TL;DR - author claims the hardest part of running a cloud PaaS is networking, thus the cloud providers charge you a ton for egress to outsource the optimization problem to you. It's not clear to me this claim is true - they're in the business of solving exactly this optimization problem - and they have, they're just capturing maximum profit. If it was really the hardest / most costly - then other providers wouldn't b…

??? This is obviously true. The bandwidth available in a geographic location is, practically speaking, limited. Roads are cheap to pave, but if there was congestion 24/7 and you paved all you could, then you would need to start rationing road usage. For a business the best way to ration is to charge more. Solving this problem requires a massive amount of engineering work, which would also justify the increased cost.

Bandwidth pricing works the same as product pricing, you are falling for the classic engineer trap in thinking the sum of the parts somehow determines the price. Cloud bandwidth prices are many many magnitudes more than what it "costs" to transfer.

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#78

Post by Syonyk »

theLiminator wrote: post #68
Syonyk wrote: post #20

What you can do about it is go back to the early 2000s and rack up your own boxes in a colo, with some reasonable unmetered bandwidth allocations, and skip the whole "cloud" nonsense. My personal box is a 6C/12T Xeon from a few years back, 128GB ECC RAM, and about 24TB usable disk plus 2TB NVMe, for running a lot of other VMs that I run actual services in (remote backup, my web presence, communications, game servers,…

I think that's great for hosting your own stuff. Just personally not something I'd want to do once I really need high availability (power outages/network outages). But I suspect it might actually be a great way for compute bound startups to start for cheap.

So...

Why do you need such high availability that a rare power outage, network outage, or equipment failure is such a big deal?

I ask having spent a lot of my college years (and some beyond) trying to run "enterprise grade uptime on a college student budget" - with the appropriate frustration. I've come around to a more casual, "As long as I can reasonably get it back up in a day or two, meh..." way of thinking about various services.

It's important to figure out what you need and what you want - a few friends and I once had a stack of rather compute-heavy boxes in a rack, and we ran most of them without a UPS. The provided UPS wasn't enough amps for what we were running, so we just backed up the core stuff and left the crunchers subject to outages. Stuff could restart easily enough without a big deal. It was annoying, but not a big deal.

I'd argue that most startups don't need a multi-region, globally backed up infrastructure. And that they'd be a lot better off with less, cashflow-wise. But I guess that's not "Built to scale to 10B users" that VC fund expect to see these days, so... jump through the hoops provided?

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#79

Post by bombcar »

Syonyk wrote: post #20

What you can do about it is go back to the early 2000s and rack up your own boxes in a colo, with some reasonable unmetered bandwidth allocations, and skip the whole "cloud" nonsense. My personal box is a 6C/12T Xeon from a few years back, 128GB ECC RAM, and about 24TB usable disk plus 2TB NVMe, for running a lot of other VMs that I run actual services in (remote backup, my web presence, communications, game servers,…

The nice key about this is "local datacenter" - it's nice to be able to drive over to your hardware when needed.

I've found some nice colo deals but I'm a bit hesitant at being too far away and requiring remote hands, though I should work out exactly how much I'm paying for power ...

Re: Why cloud bandwidth is so obscenely expensive and what you can do about that

#80

Post by _jal »

swatcoder wrote: post #48
proteal wrote: post #34

One thing to keep in mind - if you accidentally run up a bill because of a mistake, there’s a good chance you can reach out to support and they will credit your account. $5k may be a lot to an individual, but the cloud provider’s costs for that service are significantly less and they shouldn’t mind forgiving the charge. Case in point, when the author deleted his account and switched providers, the old host didn’t cha…

That's been true during recent flush years, where money was cheap and nobody really cared, but there are signs all over the place that the attitude is changing. Everything that used to be free is being monetized and everything that was monetized is seeing increased prices. We should expect that these sort of implicit affordances are being trimmed as well, or at least that they're likely to be soon.

Yeah, I rather suspect they'll be much less forgiving of this sort of thing in the future.

All that takes is a lean quarter and some MBA to coin a term for rookie mistake-windfalls, and they will come to feel entitled to revenue from it.

Post reply on HN