UC Berkeley launches SkyPilot to help navigate soaring cloud costs

"Reduce your costs", they said. "Don't worry about those expensive hardware costs", or management. "Only pay for what you need", they said.

Until everyone was hooked and the tooling was everywhere.

Even just putting the pricing for hardware/services aside, the bandwidth charges alone are quite amazing. Most have it at $80 per TB per month. That's $80 a month for a 3mbit connection. [3mbps * 3600 seconds an hour * 24 hours * 30 days ~= 1TB]. Even if you pretended it was 6mbps for double the redundancy, that's still $80 for a 6mbit connection. I understand there's more to it than just transfer amount/speed, but this is in a place where bandwidth is supposed to be cheaper, as it's "purchased" in wholesale, yet a 1GBit upload connection could transfer 330TB in a month, which would cost over $23,000 at Microsoft/Google/Amazon.

I am not saying the cloud doesn't make sense. I love me some Firebase. CDN's are game changers. And GPU's per hour is the only way some scientists/researchers can afford to do work [although it does contribute to the NVIDIA's focusing on manufacturing high price/low volume].

That said, it's not for everyone. A cheap VPS can do a lot.

Specifically for a university or an institution with a lot of cheap/free bandwidth, on-site would not only be cheaper, but also have lower latency for its campus users and more control wise.

Also, if done right, there's a possibility of teaching the students relevant real-world solutions using the knowledge gained/within the university.

https://docs-research-it.berkeley.edu/services/high-performa...

University HPC clusters are typically managed and owned departmentally. My former employer had two clusters in two different departments. I worked directly with a few other universities who also had departmental HPC clusters, and I’ve read a boatload of HPC docs from different unis and labs. Sharing clusters for many types of workloads seems to happen more at the regional level (like Archer cluster in Edinburgh, or PNNL in WA).

Ours is a fairly small cluster of maybe 60 or 80 job nodes. All compute networking travels over Infiniband—-I think we bought HDR for the new cluster, but price may held us to EDR—-which is probably why the cluster was so expensive to build (around $2M). Our storage cluster was another $1-$2M project.

All that is to preface this: cloud usage doesn’t mean they’re only using cloud resources. Our goal, for instance, was to build a hybrid cluster (and their continued goal, as far as I know). The first step was to offload low priority work with low resource requirements and wall times, and comparatively long timelines to the cheapest possible, compliant cloud provider (some of our researchers have specific data sharing and privacy requirements).

Let’s say a job is submitted on Monday morning at 9 am. It needs 2 CPUs and 4 GB of RAM, and a wall time of five minutes. The researcher can wait until next Monday at 10 am for the results. There’s at least a chance that running this job in the on-prem cluster is less cost efficient than offloading the job to another resource, whether the cost is direct (i.e. 5 minutes on-prem costs $0.10, and the cloud costs $0.05 or whatever real values would look like).

Ideally, we would have some method by which you could compare the cost to run a job in the cluster vs another resource, and it would automatically offload jobs of up to $X to the cloud, whether as a very low priority queue, or as overflow in times of full utilization. There are several other conditions that would need to be met, as well, but you get the idea (just for example, one consideration is if the work be interrupted. If the job has to run start to finish, the cloud is likely not suitable for that work).

Also universities don’t have the same cost models as companies. Their labs have less money - except those institutions with large endowments - but they all have access to near limitless “slave” labor in the form of grad students and post docs.

-- (likely) Tim Ferris.

I sat out of the cloud hoopla and it cost me opportunities - Now, I am dragging my feet into it to be employable.. but the Anti-Cloud (River?) is going to be a concept. 39 signals already did a blog post and change in strategy.

I look forward to the economically lean years of 2023 and 2024 and HN front page articles about how moving away from Cloud and owning your own server is saving 'millions'.