I would be surprised if Azure doesn't waive or reduce this bill dramatically. Something similar happened to me with AWS. I had a simple file upload service where files would expire if they hadn't been accessed in 24 hours. Someone started using it to upload music and videos. I ended up with a high bandwidth bill on Amazon S3. I reached out and explained what happened, they waived the costs entirely (to the tune of $5000).
I got pwned by my cloud costs
271–280 of 657 posts
Re: I got pwned by my cloud costs
#272Earlier quoted context omitted.
Probably would run just fine on a <= 4 euro/month virtual machine too. Of course it doesn't quite scale to zero like APIGW,lambda,etc. but on the other hand you can be fairly confident to not pay more if your pet project suddenly lands on the front page of HN.
No it wouldn't, because I infrequently need burst (bigger lambdas). As for costs, setting up billing and usage alarms on AWS is absolutely trivial. Finally, using stuff like S3 or dynamo for storage gives me a peace of mind I will never have when managing my own servers.
Out of curiosity, what are the lambdas doing that requires more than 2-4 GB of RAM?
Re: I got pwned by my cloud costs
#273Earlier quoted context omitted.
I think the case is for big companies that have a hard time attracting IT talent. Like, not in an even remotely IT-related industry, and their headquarters is in a city with no significant tech community. Places that scream "working here will not improve your resume". I'm a major cloud skeptic, but there's a certain class of giant enterprisey companies that are never going to be able to attract good IT talent, and if…
I believe renting dedicated servers is often overlooked. You pay someone else to install hardware, ensure network connections and be on-site for hardware swaps etc but still have the maximum degree of flexibility. Even larger companies can work well with that model, traffic also tends to be cheap enough that you can spread across different vendors to avoid lock-in. And in that case, your sysadmins can sit wherever th…
Many providers will now also bring up dedicated servers for you so fast, or offer you API based provisioning of single-tenant VMs or similar that it's really rare that the difference relative to cloud providers becomes much of an issue.
I used to spin up dedicated servers and then put an overlay network + a simple set of tools to spin up containers on them years before Kubernetes etc. was a thing, and we'd have a "global" (we had VMs in Asia, dedicated servers in Germany and colocated own servers in the UK) unified deployment mechanism that let me spin up containers wherever with a one-liner. Having a few extra dedicated servers with spare capacity standing by still made the whole system far cheaper than e.g. AWS, even if you attributed my entire salary towards it (I spent nowhere near all my time keeping that running).
It's easy enough to find consultants that can set up systems like this that abstracts away the dedicated hosting providers so you can mix and match and move with ease - especially today with options like Kubernetes.
If I was to go back to doing consulting I'd probably look at finding a way of packaging this kind of offering up behind lots of marketing speak and offer some sort of "abstract" hybrid private cloud layer on top of a choice of dedicated hosting providers to make that kind of hosting palatable to execs who refuse to believe the cost saving potential because they've never dived into the actual numbers (oh, the amount of time I've spent building out spreadsheets with precise cost models that'd get promptly ignored because someones had heard from a friend that company X swore vendor Y was cheap and believed it blindly)
Re: I got pwned by my cloud costs
#274Can somebody explain to me why I wouldn't just rent a 40 EUR dedicated server from Hetzner with unlimited traffic and gigabit uplink? His 600GB/day is way less than what you get over a gigabit link within a day. Sure, sudden bursts would perhaps "throttle" at a gigabit, but according to his article that was only the cloudflare proxy anyhow, so no pain in having that take a few seconds longer. As far as I am concerned…
>Can somebody explain to me why I wouldn't just rent a 40 EUR dedicated server from Hetzner [...] , I just don't understand why people use cloud services.
This recurring question of "why AWS/Azure instead of Hetzner/OVH ?" keeps happening because people are incorrectly comparing higher-level PaaS to lower-level IaaS without realizing it.
PaaS != IaaS are not equivalent. IaaS is not a direct drop-in replacement for PaaS to save money if the workload is using PaaS features that IaaS does not include.
The author Troy Hunt is using the higher-level Azure services like Table Storage (like AWS DynamoDB/SimpleDB) and Azure Functions (like AWS Lambda), and others. E.g. One of the article's hyperlinks talks about using Azure Functions.[1]
If he used Hetzner, he'd have to reinvent the Azure services stack with open-source projects (some of which are buggy and immature) and expend extra sysadmin/programming work for something that's not as integrated. The Azure/AWS stack includes many desirable housekeeping tools such as provisioning, monitoring, routing, etc which he'd also have to re-invent.
TLDR: People choose Azure/AWS because it has more features out of the box. You just have to figure out on a case-by-case basis if the PaaS value-add makes financial sense for your particular workload.
EDIT to downvoters: if Hetzner actually has built-in equivalents to AWS Lambda and DynamoDB, please reply with a correction because I don't want to spread misinformation.
[1] https://www.troyhunt.com/serverless-to-the-max-doing-big-thi...
Re: I got pwned by my cloud costs
#275Earlier quoted context omitted.
I think it is an irresponsible fad that people use cloud services for hobby projects (and despite its wide popularity I'm calling HIBP a hobby project since he's running it on the side for free) unless they have solid cloud ops experience from their day job. Cloud providers love it when people do this and are famously easy to talk to when you get an unexpected invoice high enough to require remortgaging your house to…
I don't know what the alternative is. Run a home server and pay an ISP $$$ for unusually high upload bandwidth/throughput? 99/100 times running it in the cloud is going to be cheaper, easier, and more resilient. Of course, the delayed sticker shock is a problem.. I think Google cloud actually lets you create a budget that turns services off if they go over, so there's a solution here if you run a hobby project that y…
I've variously paid $5-$30 a month for a VPS/dedicated server to host all my random side projects over almost two decades, including websites for other people, email, etc; there's probably two dozen or more sites running on my Hetzner dedicated server, with storage and CPU and RAM to more than spare. And not once had to worry about extra fees or weird billing issues. Bandwidth has grown from 100Mbps to 1Gbps and I've never had traffic issues.
Re: I got pwned by my cloud costs
#276Earlier quoted context omitted.
> As far as I am concerned, I just don't understand why people use cloud services. For companies the benefits are the abiltiy to get new servers at a click of a button and get rid of a server. For example, asking the ops team to setup a snapshot of a database for a few hours while I do something is super useful. There is also the ability to use autoscale and other stuff to automagically scale your system to handle tr…
Hetzner dedicated server * 3 + k3s + vnet + longhorn + metallb = basically the cloud. I can snapshot a database disk with a click of a button and restore the snapshot with yet another few clicks. I have 1.5 TB of highly available disk space, 40 cores of full CPU power, 160 GB of RAM, & dynamically provisioned IPs for metallb. For only $130USD a month. For the same price in Azure, I had 6 CPU cores & 8 GB RAM.
You could do that.
But let's say need 4x vCPU: 72, Memory (GiB): 144 for 4 hours. Or you need that 12 hours a day but for the rest of the time you need 2 cpu and 4 GiB of memory.
You need to handle traffic spikes such as TV traffic.
Yes you could self-host a cloud env but you can't scale your resources the way you can with cloud.
Re: I got pwned by my cloud costs
#277Can somebody explain to me why I wouldn't just rent a 40 EUR dedicated server from Hetzner with unlimited traffic and gigabit uplink? His 600GB/day is way less than what you get over a gigabit link within a day. Sure, sudden bursts would perhaps "throttle" at a gigabit, but according to his article that was only the cloudflare proxy anyhow, so no pain in having that take a few seconds longer. As far as I am concerned…
He should and did use torrents.
Re: I got pwned by my cloud costs
#278As soon as I saw "17GB file" i thought "that's what torrents are for". Otherwise one mistake and... Well this happens. Or someone maliciously bypasses CF cache e.g. by parameters. Cloud just is not suitable for any kind of volume egress. It's a death trap. Like going on vacation with data roaming enabled.
Or Hetzner server auction to get a cheap 20/30€ machine with unlimited traffic at 1Gbps. Setup time is max 1h even if you do it manually, with cloudflare Tunnel it's also really easy to lock down everything with a firewall and have minimal exposure to threats.
> Setup time is max 1h even if you do it manually
- Patching - Remediation, Monitoring, day0 response
- Security Information and Event Management - exports, alerts, OS configuration
- OS/Application Hardening - Encryption, Password/keys rotation, CIS/other baselines, Drift Management
- Backup - Encryption, (don't forget your passwords/keys are changing), retention, data protection compliance, monitoring, alerting, test days
- High Availability - replication, synchronisation, monitoring, alerts, test days
This is just the tip of the ice berg, if you operate in an environment where Insurance, Reputation, Regulatory Compliance, etc.. are important, then it's easy to see why PAAS solutions are desirable.
Re: I got pwned by my cloud costs
#279Ah the old cloud provider switcheroo. Yip this is the way they make money. They make it easy to setup some gigantic hugely scalable website then hit you with a gigantic scaled up bill. AWS would do this as well. Team I'm in at the moment is in the early stages of cloud adoption but the company in total has fell hook line and sinker for AWS. When I mentioned the cost there is always an excuse. The main one being that…
our department now spend > £1 million per year to AWS in hosting costs. A 20% reduction in those fees could pay for a few sysadmin(s). You can hire a "few" sysadmins for 200k/year?
£200k / year, in the UK? That's about 2-5 depending on experience.
Re: I got pwned by my cloud costs
#280Can somebody explain to me why I wouldn't just rent a 40 EUR dedicated server from Hetzner with unlimited traffic and gigabit uplink? His 600GB/day is way less than what you get over a gigabit link within a day. Sure, sudden bursts would perhaps "throttle" at a gigabit, but according to his article that was only the cloudflare proxy anyhow, so no pain in having that take a few seconds longer. As far as I am concerned…
- Patching - Remediation, Monitoring, day0 response
- Security Information and Event Management - exports, alerts, OS configuration
- OS/Application Hardening - Encryption, Password/keys rotation, CIS/other baselines, Drift Management
- Backup - Encryption, (don't forget your passwords/keys are changing), retention, data protection compliance, monitoring, alerting, test days
- High Availability - replication, synchronisation, monitoring, alerts, test days
This is just the tip of the ice berg, if you operate in an environment where Insurance, Reputation, Regulatory Compliance, certification, etc.. are important, then it's easy to see why PAAS solutions are desirable.