When I was looking at AWS for personal use, I first thought it was oddly expensive even when factoring in not having to buy the hardware. When I looked at just what the electricity cost to run it myself would be, I think that addition alone turned out AWS was actually cheaper. This is without factoring in cooling / dedicated space / maintenance.
AWS doesn't make sense for scientific computing
71–80 of 281 posts
Re: AWS doesn't make sense for scientific computing
#72Database analyst for a large communication company here. I have similar doubts about AWS for certain kinds of intensive business analysis. Not API based transactions, but back-office analysis where complex multi-join queries are run in sequence against tables with 10s of millions of records. We do some of this with SQL servers running right on the desktop (and one still uses Excel with VLOOKUP). We have a pilot proje…
Do you have disaster recovery / high availability requirements? SQL server on a desktop has a lot of single points of failure.
Re: AWS doesn't make sense for scientific computing
#73Earlier quoted context omitted.
I remember the first time a server caught fire in the closet we kept the rack in. Backups were kept on a server right below the one on fire. But, y'know, we saved money.
Don’t worry, we do incremental backups during weekdays and a full backup on Sunday. We use 2 tapes only, so one is always outside of the building. But you know, we saved money.
[deleted]
Re: AWS doesn't make sense for scientific computing
#74Buying your own fleet of dedicated servers seems like a smart move in the short term, but then five years from now you’ll get someone on the team insisting that they need the latest greatest GPU to run their jobs. Cloud providers give you the option of using newer chipsets without having to re-purchase your entire server fleet every five years.
Re: AWS doesn't make sense for scientific computing
#75Earlier quoted context omitted.
I remember the first time a server caught fire in the closet we kept the rack in. Backups were kept on a server right below the one on fire. But, y'know, we saved money.
Don’t worry, we do incremental backups during weekdays and a full backup on Sunday. We use 2 tapes only, so one is always outside of the building. But you know, we saved money.
[deleted]
Re: AWS doesn't make sense for scientific computing
#76Astronomy is moving more and more to cloud computing:
Re: AWS doesn't make sense for scientific computing
#77This has been my exact field of work for a few years now; in general I have found that:
When people claim it is 10x more expensive to use public cloud, they have no earthly idea what it actually costs to run a HPC service, a data centre, or do any of the associated maintenance.
When the claim is 3x more expensive in the cloud, they do know those things but are making a bad faith comparison because their job involves running an on-premises cluster and they are scared of losing their toys.
When the claim is 0-50% more to run in the cloud, someone is doing the math properly and aiming for a fair comparison.
When the claim is that cloud is cheaper than on-prem, you are probably talking to a cloud vendor account manager whose colleagues are wincing at the fact that they just torched their credibility.
Re: AWS doesn't make sense for scientific computing
#78This has been my exact field of work for a few years now; in general I have found that: When people claim it is 10x more expensive to use public cloud, they have no earthly idea what it actually costs to run a HPC service, a data centre, or do any of the associated maintenance. When the claim is 3x more expensive in the cloud, they do know those things but are making a bad faith comparison because their job involves…
Author here! I think running an HPC service that has a steady queue in AWS can be more than 3x as expensive.
What type of HPC do you work in? Maybe I'm over-indexing on computational biology.
Re: AWS doesn't make sense for scientific computing
#79Having had the responsibility of providing HPC for a literal buildings full of scientists, I can say that it may be true that you can get computation cheaper with owned hardware, than in a cloud. Certainly pay as you go, individual project at a time processing will look that way to the scientist. But I can also say with confidence that the contest is far closer than they think. Scientists who make this argument almos…
This nails so much of the discussion that should be had. When using any cloud service provider, you aren't just paying for the machines/hardware you use - you are paying for people to take care of a bunch of headaches of having to maintain this hardware. It's incredibly easy to overlook this aspect of costs and really easy to oversimplify what's involved if you don't know how these things actually work.
Re: AWS doesn't make sense for scientific computing
#80Earlier quoted context omitted.
I remember the first time a server caught fire in the closet we kept the rack in. Backups were kept on a server right below the one on fire. But, y'know, we saved money.
Don’t worry, we do incremental backups during weekdays and a full backup on Sunday. We use 2 tapes only, so one is always outside of the building. But you know, we saved money.
A bit off topic, but I gotta say you guys are a riot!
If there was a comedy tour for IT/Programmer types, I'd pay to see you guys in it.
Best thing about your stuff is that it's literally all funny precisely because it's all true.