[deleted]
AWS doesn't make sense for scientific computing
41–50 of 281 posts
Re: AWS doesn't make sense for scientific computing
#42Is genomic code typically distributed-memory parallel? I'm under the impression that it is more like batch processing, not a ton of node-to-node communication but you want lots of bandwidth and storage.
If you are doing a big distributed-memory numerical simulation, on the other hand, you probably want infiniband I guess.
AWS seems like an OK fit for the former, maybe not great for the latter...
Re: AWS doesn't make sense for scientific computing
#43Cloud worked really well for me when I was in school. A lot of the time, I would only need a beefy computer for a few hours at a time (often due to high memory usage) and you can/could rent out spot instances for very cheap. There are about 730 hours per month so the cost calculus is very different for a student/researcher who needs fast turnaround times (high performance), but only for a short period of time.
However, I know not all HPC/scientific computing works that way and some workloads are much more continuous.
Re: AWS doesn't make sense for scientific computing
#44Is genomic code typically distributed-memory parallel? I'm under the impression that it is more like batch processing, not a ton of node-to-node communication but you want lots of bandwidth and storage. If you are doing a big distributed-memory numerical simulation, on the other hand, you probably want infiniband I guess. AWS seems like an OK fit for the former, maybe not great for the latter...
The fastest way to do a lot of genomics stuff is with FPGA accelerators, which also aren't used by most of the other tenants in a multi-tenant scientific computing center. The cloud is perfect for that kind of work.
Re: AWS doesn't make sense for scientific computing
#45Its much more complex than described. The author is making a brilliant argument for getting a secondhand workstation and shoving under their desk. If you are doing multi machine batch style processing, then you won't be using ondemand, you'd use the spot pricing. The missing argument in that part is storage costs. Managing a high speed, highly available synchronous file system that can do a sustained 50gb/sec is hard…
Isn't 50GB sec like 5 NVMe Gen 5 SSDs + 1 or 2 for redundancy? Actually, you are right. Consumer SSDs I've seen only do about 1.5GB/s sustained.
Even (high-end) consumer SSDs can saturate a PCIe gen 4 x4 link if you are doing sequential reads. Non-sequential hurts on even enterprise SSDs.
Re: AWS doesn't make sense for scientific computing
#46The killer we've seen is data egress costs. Crunching the numbers for some of our pipelines, we'd actually be paying more to get the data out of AWS than to compute it.
Re: AWS doesn't make sense for scientific computing
#47Having had the responsibility of providing HPC for a literal buildings full of scientists, I can say that it may be true that you can get computation cheaper with owned hardware, than in a cloud. Certainly pay as you go, individual project at a time processing will look that way to the scientist. But I can also say with confidence that the contest is far closer than they think. Scientists who make this argument almos…
I think it really depends on the task. Where HIPAA violation is a real threat, the equation changes. And just for CYA purposes those projects can get pushed to a cloud. Which does not necessarily involve any attempts to make them any more secure, but this is a different topic. That said, many scientists are operating on premise hardware like this: some servers in a shared rack and an el-cheapo storage solutions with…
> And it works just fine for them.
Until it doesn't because there's a fire or huge power surge or whatever.
That's the point -- there's a lot of risk they're not taking into account, and by focusing on the "it works just fine for them", you're cherry picking the ones that didn't suffer disaster.
Re: AWS doesn't make sense for scientific computing
#48Cloud worked really well for me when I was in school. A lot of the time, I would only need a beefy computer for a few hours at a time (often due to high memory usage) and you can/could rent out spot instances for very cheap. There are about 730 hours per month so the cost calculus is very different for a student/researcher who needs fast turnaround times (high performance), but only for a short period of time. Howeve…
Thats how my department uses the cloud--we have an image we store up at AWS geared towards a couple of tasks and we spin up a big instance when we need it, run the task, pull out the results, then stop the machine. Total cost sub-100 dollars. If we had to go the HPC group we'd have to fight with them to get the environment configured, get access to the system, get payment setup, teach the faculty to use the environment, etc. Its just a pain for very little gain.
Re: AWS doesn't make sense for scientific computing
#49> Most scientific computing runs on queues. These queues can be months long for the biggest supercomputers – that's the API equivalent of storing your inbound API requests, and then responding to them months later
Makes sense if the jobs are all low urgency.
We have a similar problem in trading so we have a composite solution with non-cloud simulation hardware and additional AWS hardware. That's because we have the high utilization solution combined with high urgency.
Re: AWS doesn't make sense for scientific computing
#50> Most scientific computing runs on queues. These queues can be months long for the biggest supercomputers
That sounds very much like an argument for a cloud. Instead of waiting months to do your processing, you spin up what you need, then tear it down when you are done.