AWS doesn't make sense for scientific computing • Hacker News Board

Re: AWS doesn't make sense for scientific computing

#51

Post by sabalaba » Fri, Oct 07, 2022, 5:11 PM UTC

lostmsu wrote: post #25

zatarc wrote: post #11

Why does no one consider colocation services anymore? And why do people only know Hetzner, OVH and Linode as alternatives to the big cloud providers? There are so many good and inexpensive server hosting providers, some with decades of experience.

Any particular you could recommend for GPU?

Lambda GPU Cloud has the cheapest A100s of that group. https://lambdalabs.com/service/gpu-cloud

Lambda A100s - $1.10 / hr Paperspace A100s - $3.09 / hr Genesis A100s - no A100s but their 3090 (1/2 the speed of 100) is - $1.30 / hr for half the speed

Re: AWS doesn't make sense for scientific computing

#52

Post by lebovic » Fri, Oct 07, 2022, 5:12 PM UTC

pclmulqdq wrote: post #36

Even as a big cloud detractor, I have to disagree with this. A lot of scientific computing doesn't need a persistent data center, since you are running a ton of simulations that only take a week or so, and scientific computing centers at big universities are a big expense that isn't always well-utilized. Also, when they are full, jobs can wait weeks to run. These computing centers have fairly high overhead, too, alth…

Author here. I agree with your points! I use AWS for a computational biology company I'm working on. A lot of scientific computing can spin up and down within a couple hours on AWS and benefits from fast turnaround. Most academic HPCs (by # of clusters) are slower than a mega-cluster on AWS, not well utilized, and have a lot of bureaucratic process.

That said, most of scientific computing (by % of total compute) happens in a different context. There's often a physical machine within the organization that's creating data (e.g. a DNA sequencer, particle accelerator, etc), and a well-maintained HPC cluster that analyzes that data. The researchers have already waited months for their data, so another couple weeks in a queue doesn't impact their cycle.

For that context, AWS doesn't really make sense. I do think there's room for a cloud provider that's geared towards an HPC use-case, and doesn't have the app-inspired limits (e.g data transfer) like AWS, GCP, and Azure.

Re: AWS doesn't make sense for scientific computing

#53

Post by thesausageking » Fri, Oct 07, 2022, 5:12 PM UTC

I'm suspicious of the author's actual experience.

The fact that scientific computing has a different pattern than the typical web app is actually a good thing. If you can architect large batch jobs to use spot instances, it's 50-80% cheaper.

Also this bit: "you can keep your servers at 100% utilization by maintaining a queue of requested jobs" isn't true in practice. The pattern of research is the work normally comes in waves. You'll want to train a new model or run a number of large simulations. And then there will be periods of tweaking and work on other parts. And then more need for a lot of training. Yes, you can always find work to put on a cluster to keep it >90% utilization, but if it can be elastic (and has compute has budget attached to it), it will rise and fall.

Re: AWS doesn't make sense for scientific computing

#54

Post by wenc » Fri, Oct 07, 2022, 5:14 PM UTC

Calculating costs based on sticker price is sometimes misleading because there’s another variable: negotiated pricing, which can be much much lower than sticker prices, depending on your negotiating leverage. Different companies pay different prices for the same product.

If you’ve ever worked at a big company or university (any place where you spend at scale), you’ll know you rarely pay sticker price. Software licensing is particularly elastic because it’s almost zero marginal cost. Raw cloud costs are largely a function of energy usage and amortized hardware costs — there’s a certain minimum you can’t go under but there remains a huge margin that is open to be negotiated on.

Startups/individuals rarely even think about this because they rarely qualify. But big orgs with large spends do. You can get negotiated cloud pricing.

Re: AWS doesn't make sense for scientific computing

#55

Post by eastbound » Fri, Oct 07, 2022, 5:15 PM UTC

0xbadcafebee wrote: post #38

W-Stool wrote: post #32

Earlier quoted context omitted.

Let me echo this as someone who once was responsible for HPC computing in a research intensive public university. Most career academics have NO IDEA how much enterprise computing infrastructure costs. If a 1 terabyte USB hard drive is $40 at Costco we (university IT) must be getting a much better deal than that. Take this argument and apply it to any aspect of HPC computing and that's what you're fighting against. Th…

I remember the first time a server caught fire in the closet we kept the rack in. Backups were kept on a server right below the one on fire. But, y'know, we saved money.

Don’t worry, we do incremental backups during weekdays and a full backup on Sunday. We use 2 tapes only, so one is always outside of the building. But you know, we saved money.

Re: AWS doesn't make sense for scientific computing

#56

Post by rirze » Fri, Oct 07, 2022, 5:15 PM UTC

captainmuon wrote: post #40

A former colleague did his PHD in particle physics with a novel technique (matrix element method). I can't really explain it, but it is extremely CPU intensive. That working group did it on CERN's resources, and they had to borrow quotas from a bunch of other people. For fun they calculated how much it would have cost on AWS and came up with something ridiculous like 3 million euros.

I would imagine CERN's resources are essentially a data center comparable to a small cloud provider's resources.

Re: AWS doesn't make sense for scientific computing

#57

Post by withinboredom » Fri, Oct 07, 2022, 5:18 PM UTC

crazygringo wrote: post #47

ptero wrote: post #33

Earlier quoted context omitted.

I think it really depends on the task. Where HIPAA violation is a real threat, the equation changes. And just for CYA purposes those projects can get pushed to a cloud. Which does not necessarily involve any attempts to make them any more secure, but this is a different topic. That said, many scientists are operating on premise hardware like this: some servers in a shared rack and an el-cheapo storage solutions with…

> And it works just fine for them. Until it doesn't because there's a fire or huge power surge or whatever. That's the point -- there's a lot of risk they're not taking into account, and by focusing on the "it works just fine for them", you're cherry picking the ones that didn't suffer disaster.

Ummm. I’ve def been unable to do anything for entire days because our AWS region went down and we had to rebuild the database from scratch. AWS goes down, you twiddle your thumbs and the people you report to are going to be asking why, for how long, etc. and you can’t give them an answer until AWS comes back to see how fubar things are.

When your own hardware rack goes down. You know the problem, how much it costs to fix it, and when it will come back up; usually within a few hours (or minutes) of it going down.

Do things catch fire, yes. But I think you’re over-estimating how often. In my entire life, I’ve had a single SATA connector catch fire and it just melted plastic before going out.

Re: AWS doesn't make sense for scientific computing

#58

Post by withinboredom » Fri, Oct 07, 2022, 5:20 PM UTC

thayne wrote: post #50

> Most scientific computing runs on queues. These queues can be months long for the biggest supercomputers That sounds very much like an argument for a cloud. Instead of waiting months to do your processing, you spin up what you need, then tear it down when you are done.

The queue then just turns into the bank account. The queue doesn’t magically go away.

Re: AWS doesn't make sense for scientific computing

#59

Post by zatarc » Fri, Oct 07, 2022, 5:25 PM UTC

lostmsu wrote: post #25

zatarc wrote: post #11

Why does no one consider colocation services anymore? And why do people only know Hetzner, OVH and Linode as alternatives to the big cloud providers? There are so many good and inexpensive server hosting providers, some with decades of experience.

Any particular you could recommend for GPU?

I'm not in a position to recommend or not a particular provider for gpu-equipped servers, simply because I've never had the need for gpus.

My first thought was related to colocation services. From what I understand, a lot of people avoid on-premise/in-house solutions because they don't want to deal with server rooms, redundant power, redundant networks, etc.

So people go to the cloud and pay horrendous prices there.

Why not take a middle path? Build your own custom server with your perferred hardware and put in a colocation

Re: AWS doesn't make sense for scientific computing

#60

Post by wistlo » Fri, Oct 07, 2022, 5:26 PM UTC

Database analyst for a large communication company here.

I have similar doubts about AWS for certain kinds of intensive business analysis. Not API based transactions, but back-office analysis where complex multi-join queries are run in sequence against tables with 10s of millions of records.

We do some of this with SQL servers running right on the desktop (and one still uses Excel with VLOOKUP). We have a pilot project to try these tasks in a new Azure instance. I look forward to seeing how it performs, and at what cost.