dupe of https://news.ycombinator.com/item?id=35094407
In general, it has been the position of moderators that reposts are generally fine with an interval of 6 months-1 year.
Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
21–30 of 66 posts
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#22Not directly related, but how much energy is being wasted by thousands of companies scraping the internet continuously and storing roughly the same information as everyone else, and then storing that in their own datacenters? I understand the commercial reasons for it, but this all seems very inefficient.
The semantic web was a beautiful idea at one point
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#23if you give the heart of your business (data) to an alien company you make TWO collosal mistakes:
1) you transfer the core of your business to somebody else 2) you can be blackmailed service and costs wise
outsourcing in general is a deadly management fashion.
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#24Really misleading numbers in a lot of ways. Notably, they dismiss the risk of inflation, ignore longer term maintenance, choose really poor cloud analogs for their architecture, and ignore cost-saving options like spot instances and pre-pay discounts.
All that said, yes, cloud is often more expensive for simple applications with stable 24/7 workloads that don’t evolve over time. Do the research and choose the right infrastructure platform for your business.
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#25The new mantra with the cloud champions in my company is that cloud was never meant to save money. It’s a premium experience that’s about saving time.
This did not sound right so I dug into the emails our leadership sent us between 6-3 years ago upping our “cloud transformation”. And yup, saving money was a part of it.
It’s only over the last year or so where it’s become obvious we didn’t save any money and in fact spent a lot more that it’s become about functionality and quality and not about cost.
The cloud may beat on Prem on functionality. For example, global colocation is much easier with the cloud. But don’t f’ing gaslight me and tell me the cloud providers hadn’t been selling cost as a benefit and even the primary benefit for the first 10 years or so at least.
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#26Not directly related, but how much energy is being wasted by thousands of companies scraping the internet continuously and storing roughly the same information as everyone else, and then storing that in their own datacenters? I understand the commercial reasons for it, but this all seems very inefficient.
Nowhere near what is currently being wasted on LLMs generating the content those scrapers will soon be copying to their datacenters, or the cryptomining propping up Bitcoin.
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#27They crawl all the time, their instances could go down and no problem, there are still hundreds doing the same task. They consume waaaay too much traffic for the cloud to make sense financially. Hybrid approach is best in cases like this. Use the cloud for client facing interfaces and rent dedicated servers for the spiders. edit: even better, build your own data center instead of renting.
In a much smaller scale, I'm working on a web crawler as well, and renting a dedicated server at Hetzner with unlimited traffic is cheaper than any VPS, or cloud offering. 8 cores, 32 GB RAM, 2x 500 GB SSD for ~€40/month — it's an older CPU but web crawlers don't spend too much time crunching numbers anyway.
What crawling framework you using?
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#28Yes, but this is truly an exceptional case. Their workloads are basically scraping (crawling) at a massive scale. Just like Google does, it makes more sense to have cheap throw-away hardware for this use case. There are no permission issues or ACLs. There’s no need to auto scale and the traffic is very predictable. There is no serious need to orchestrate deployments. I imagine it’s mostly just workers reading URLs fr…
[deleted]
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#29>by not going to the cloud ...in the worst way imaginable Doing a direct lift and shift with 1:1 replacement of instances is, intentionally, prohibitibely expensive, so you stop and think .
Right, now factor in the cost of architecting software not because that’s the best architecture for your use case but to save money on the specific infrastructure choice you’ve made.
Re: Ahrefs Saved US$400M in 3 Years by Not Going to the Cloud (2023)
#30The new mantra with the cloud champions in my company is that cloud was never meant to save money. It’s a premium experience that’s about saving time. This did not sound right so I dug into the emails our leadership sent us between 6-3 years ago upping our “cloud transformation”. And yup, saving money was a part of it. It’s only over the last year or so where it’s become obvious we didn’t save any money and in fact s…
Indeed. Cost optimization is one of the “6 Pillars of the AWS Well-Architected Framework” as promoted by AWS themselves:
“The Cost Optimization pillar includes the ability to run systems to deliver business value at the lowest price point.”
https://docs.aws.amazon.com/wellarchitected/latest/framework...