81–90 of 281 posts
Our EM facility generates 10 TB of raw data per day, and once you start computing it, that increases by 30%-50% depending on what you do with it. Plus, moving between network storage and local scratch for computational steps basically never ends and keeps multiple 10 Gbe links saturated 100% of the time.
Having had the responsibility of providing HPC for a literal buildings full of scientists, I can say that it may be true that you can get computation cheaper with owned hardware, than in a cloud. Certainly pay as you go, individual project at a time processing will look that way to the scientist. But I can also say with confidence that the contest is far closer than they think. Scientists who make this argument almos…
The article points out that this is mostly not necessary for scientific computing.
The killer we've seen is data egress costs. Crunching the numbers for some of our pipelines, we'd actually be paying more to get the data out of AWS than to compute it.
This has been my exact field of work for a few years now; in general I have found that: When people claim it is 10x more expensive to use public cloud, they have no earthly idea what it actually costs to run a HPC service, a data centre, or do any of the associated maintenance. When the claim is 3x more expensive in the cloud, they do know those things but are making a bad faith comparison because their job involves…
Author here! I think running an HPC service that has a steady queue in AWS can be more than 3x as expensive. What type of HPC do you work in? Maybe I'm over-indexing on computational biology.
Could whatever actions have to be performed on the data also be performed in AWS?
Also while briefly looking into this I found that AWS has an egress waiver for researchers and educational instiutions: https://aws.amazon.com/blogs/publicsector/data-egress-waiver...
Earlier quoted context omitted.
I think it really depends on the task. Where HIPAA violation is a real threat, the equation changes. And just for CYA purposes those projects can get pushed to a cloud. Which does not necessarily involve any attempts to make them any more secure, but this is a different topic. That said, many scientists are operating on premise hardware like this: some servers in a shared rack and an el-cheapo storage solutions with…
> And it works just fine for them. Until it doesn't because there's a fire or huge power surge or whatever. That's the point -- there's a lot of risk they're not taking into account, and by focusing on the "it works just fine for them", you're cherry picking the ones that didn't suffer disaster.
I've never worked in this space, but I'm curious about the need for massive egress. What's driving the need to bring all that data back to the institution? Could whatever actions have to be performed on the data also be performed in AWS? Also while briefly looking into this I found that AWS has an egress waiver for researchers and educational instiutions: https://aws.amazon.com/blogs/publicsector/data-egress-waiver..…
The other is for reproducibility - typically you need to preserve lots of in-between steps for peer review and proving that you aren't making things up. Some intermediary data is wiped out, but usually only if it can be quickly and easily regenerated.
Having had the responsibility of providing HPC for a literal buildings full of scientists, I can say that it may be true that you can get computation cheaper with owned hardware, than in a cloud. Certainly pay as you go, individual project at a time processing will look that way to the scientist. But I can also say with confidence that the contest is far closer than they think. Scientists who make this argument almos…
Earlier quoted context omitted.
Any particular you could recommend for GPU?
Lambda GPU Cloud has the cheapest A100s of that group. https://lambdalabs.com/service/gpu-cloud Lambda A100s - $1.10 / hr Paperspace A100s - $3.09 / hr Genesis A100s - no A100s but their 3090 (1/2 the speed of 100) is - $1.30 / hr for half the speed
Having had the responsibility of providing HPC for a literal buildings full of scientists, I can say that it may be true that you can get computation cheaper with owned hardware, than in a cloud. Certainly pay as you go, individual project at a time processing will look that way to the scientist. But I can also say with confidence that the contest is far closer than they think. Scientists who make this argument almos…
Classic big workstations are way more capable than people think - but at the same time it's hard to justify buying one machine per user unless your department is swimming in money. Also, academic budgets tend to come in fixed chunks, and university IT departments may not have your particular group as a priority - so often it's just better to invest once in a standalone server tower that you can set up to do exactly what you need it to than try to get IT to support your needs or the accounting department to pay recurring AWS bills.