Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS doesn't make sense for scientific computing

noahlebovic.com

Post reply on HN
Print view View original

101–110 of 281 posts

Re: AWS doesn't make sense for scientific computing

#101

Post by racking7 »

wenc wrote: post #54

Calculating costs based on sticker price is sometimes misleading because there’s another variable: negotiated pricing, which can be much much lower than sticker prices, depending on your negotiating leverage. Different companies pay different prices for the same product. If you’ve ever worked at a big company or university (any place where you spend at scale), you’ll know you rarely pay sticker price. Software licens…

This is definitely true for cloud retail prices. However, this becomes not true in cases I've seen when there is an existing discount. Reserved instances, for example.

Re: AWS doesn't make sense for scientific computing

#102

Post by horsawlarway »

crazygringo wrote: post #47
ptero wrote: post #33

Earlier quoted context omitted.

I think it really depends on the task. Where HIPAA violation is a real threat, the equation changes. And just for CYA purposes those projects can get pushed to a cloud. Which does not necessarily involve any attempts to make them any more secure, but this is a different topic. That said, many scientists are operating on premise hardware like this: some servers in a shared rack and an el-cheapo storage solutions with…

> And it works just fine for them. Until it doesn't because there's a fire or huge power surge or whatever. That's the point -- there's a lot of risk they're not taking into account, and by focusing on the "it works just fine for them", you're cherry picking the ones that didn't suffer disaster.

I'd counter by saying I think you're over-estimating how valuable mitigating that risk is to this crowd.

I'd further say that you're probably over-estimating how valuable mitigating that risk is to anyone, although there are a few limited set of customers that genuinely do care.

There are few places I can think of that would benefit more by avoiding cloud costs than scientific computing...

They often have limited budgets that are driven by grants, not derived by providing online services (computer going down does not impact bottom line).

They have real computation needs that mean hardware is unlikely to sit idle.

There is no compelling reason to "scale" in the way that a company might need to in order to handle additional unexpected load from customers or hit marketing campaigns.

Basically... the only meaningful offering from the cloud is likely preventing data loss, and this can be done fairly well with a simple backup strategy.

Again - they aren't a business where losing a few hours/days of customer data is potentially business ending.

---

And to be blunt - I can make the same risk avoidance claims about a lot of things that would simply get me laughed out of the room.

"The lead researcher shouldn't be allowed in a car because it might crash!"

"The lab work must be done in a bomb shelter in case of war or tornados!"

"No one on the team can eat red meat because it increases the risk of heart attack!"

and on and on and on... Simply saying "There's risk" is not sufficient - you must still make a compelling argument that the cost of avoiding that risk is justified, and you're not doing that.

Re: AWS doesn't make sense for scientific computing

#103

Post by [deleted] »

killingtime74 wrote: post #98
mangecoeur wrote: post #90

Earlier quoted context omitted.

I've been running a group server (basically a shared workstation) for 5 years and it's been great. Way cheaper than cloud, no worrying about national rules on where data can be stored, no waiting in a SLURM batch queue, Jupyter notebooks on tap for everyone. A single $~6k outlay (we don't need GPUs which helps). Classic big workstations are way more capable than people think - but at the same time it's hard to justif…

Aren't you talking about 1 server when this is talking about HPC?

[deleted]

Re: AWS doesn't make sense for scientific computing

#104

Post by mattkrause »

julienchastang wrote: post #21

I’ve also been skeptical of the commercial cloud for scientific computing workflows. I don’t think this cost benefit analysis mentions it, but the commercial cloud makes even less sense when you take into account brick and mortar considerations. In other words, if your company/institution has already paid for the machine rooms, sys admins, networks, the physical buildings, the commercial cloud is even less appealing.…

It's often not that they're "not comfortable"; it's that we're often flat-out not allowed to.

Re: AWS doesn't make sense for scientific computing

#106

Post by adolph »

withinboredom wrote: post #58
thayne wrote: post #50

> Most scientific computing runs on queues. These queues can be months long for the biggest supercomputers That sounds very much like an argument for a cloud. Instead of waiting months to do your processing, you spin up what you need, then tear it down when you are done.

The queue then just turns into the bank account. The queue doesn’t magically go away.

Classic iron triangle, pick two:

  * cheap
  * fast
  * available
https://en.wikipedia.org/wiki/Project_management_triangle

Re: AWS doesn't make sense for scientific computing

#107

Post by noobermin »

crazygringo wrote: post #47
ptero wrote: post #33

Earlier quoted context omitted.

I think it really depends on the task. Where HIPAA violation is a real threat, the equation changes. And just for CYA purposes those projects can get pushed to a cloud. Which does not necessarily involve any attempts to make them any more secure, but this is a different topic. That said, many scientists are operating on premise hardware like this: some servers in a shared rack and an el-cheapo storage solutions with…

> And it works just fine for them. Until it doesn't because there's a fire or huge power surge or whatever. That's the point -- there's a lot of risk they're not taking into account, and by focusing on the "it works just fine for them", you're cherry picking the ones that didn't suffer disaster.

May be consider that your use case and the average scientist's use case isn't the same? What works for you won't work for them and vise versa? What you consider a risk, I wouldn't?

Consider the following, I have never considered applying meltdown or spectre mitigations if it makes my code run slower because I plain don't care, assuming anyone even peeks at what my simulations doing, whoopdeedo, I don't care. I won't do that on my laptop I use to buy shit off amazon with, but the workstation I have control of? I don't care. I DO care if my simulation will take 10 days instead of a week.

My use case isn't yours because my needs aren't yours. Not everything maps across domains.

Re: AWS doesn't make sense for scientific computing

#108

Post by secabeen »

walnutclosefarm wrote: post #24

Having had the responsibility of providing HPC for a literal buildings full of scientists, I can say that it may be true that you can get computation cheaper with owned hardware, than in a cloud. Certainly pay as you go, individual project at a time processing will look that way to the scientist. But I can also say with confidence that the contest is far closer than they think. Scientists who make this argument almos…

They leave out major costs because they don't pay those costs. Power, Cooling, Real Estate are all significant drivers of AWS costs. Researchers don't pay those costs directly. The university does, sure, but to the researcher, that means those costs are pre-paid. Going to AWS means you're essentially paying for those costs twice. plus all the profit margin and availability that AWS provides that you also don't need.

Re: AWS doesn't make sense for scientific computing

#109

Post by secabeen »

ptero wrote: post #33
walnutclosefarm wrote: post #24

Having had the responsibility of providing HPC for a literal buildings full of scientists, I can say that it may be true that you can get computation cheaper with owned hardware, than in a cloud. Certainly pay as you go, individual project at a time processing will look that way to the scientist. But I can also say with confidence that the contest is far closer than they think. Scientists who make this argument almos…

I think it really depends on the task. Where HIPAA violation is a real threat, the equation changes. And just for CYA purposes those projects can get pushed to a cloud. Which does not necessarily involve any attempts to make them any more secure, but this is a different topic. That said, many scientists are operating on premise hardware like this: some servers in a shared rack and an el-cheapo storage solutions with…

Thankfully, only a small part of the academic research enterprise involves human subjects, HIPAA, and all that. Neither fruit flies nor quarks have privacy rights.

Re: AWS doesn't make sense for scientific computing

#110

Post by boldlybold »

bhewes wrote: post #83
fwip wrote: post #46

The killer we've seen is data egress costs. Crunching the numbers for some of our pipelines, we'd actually be paying more to get the data out of AWS than to compute it.

Data movement has become the number one cost in system builds energy wise.

As in, the networking equipment consumes the most energy? Given the 30x markup on AWS egress I'm inclined to say it's more about incentives and marketing, but I'd love to learn otherwise.
Post reply on HN