I'm the developer of kdeps.com, and I really like Google Cloud Run, been using it since beta version. Kdeps outputs Dockerized full-stack AI agent apps that runs open-source LLMs locally, and my project works so well with GCR.
Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
21–30 of 191 posts
Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#22Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#23All the cruft of a big cloud provider, AND the joy of uncapped yolo billing that has the potential to drain your creditcard overnight. No thanks, I'll personally stick with Modal and vast.ai
thats what billing limits are for
Unless something changed gcp only does billing alerts, not billing limits
Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#24All the cruft of a big cloud provider, AND the joy of uncapped yolo billing that has the potential to drain your creditcard overnight. No thanks, I'll personally stick with Modal and vast.ai
I dunno, the scale to zero and pay per second features seemed super useful to me after forgetting to shut down some training instances with AWS. Also the fast startup ability, if it actually works as well as they say, would be amazing for a lot of the type of workloads that I have.
Agreed, but runpod or modal offer the same. Happy to use big cloud for a client if they pay the bills, but for personal quests... too scary.
Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#25Im tired of using AI in cloud services. I want user friendly locally owned AI hardware.
Right now nothing is consumer friendly. I can’t get a packaged deal of some locally running ChatGPT quality UI or voice command system in an all in one package. Like what Macs did for PCs I want the same for AI.
Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#26Im tired of using AI in cloud services. I want user friendly locally owned AI hardware. Right now nothing is consumer friendly. I can’t get a packaged deal of some locally running ChatGPT quality UI or voice command system in an all in one package. Like what Macs did for PCs I want the same for AI.
Oracle just announced they are spending $40 billion on GPU hardware. All cloud providers have an AI offering, and there are AI-specific cloud providers. I don't think retail is invited.
Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#27The pricing doesn't look that compelling, here are the hourly rate comparisons vs runpod.io vs vast.ai: 1x L4 24GB: google: $0.71; runpod.io: $0.43, spot: $0.22 4x L4 24GB: google: $4.00; runpod.io: $1.72, spot: $0.88 1x A100 80GB: google: $5.07; runpod.io: $1.64, spot: $0.82; vast.ai $0.880, spot: $0.501 1x H100 80GB: google: $11.06; runpod.io: $2.79, spot: $1.65; vast.ai $1.535, spot: $0.473 8x H200 141GB: google:…
I think the Google prices are billed per-second so under 20min you are better on Google?
Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#28All the cruft of a big cloud provider, AND the joy of uncapped yolo billing that has the potential to drain your creditcard overnight. No thanks, I'll personally stick with Modal and vast.ai
Not providing a cap on spending is a major flaw of GCP for individuals / small projects.
With Cloud Run, AFAIK, spending can effectively be capped by: limiting concurrency, plus limiting the max number of instances it can scale to. (But this is not as good as GCP having a proper cap.)
Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#29Earlier quoted context omitted.
thats what billing limits are for
Those, on gcp, are just alerts, not hard limits, no?
Yeah. I think you can hack together a function which pulls the plug automatically if a billing alert fires, but IIRC the alerts can take a few hours to respond, so extreme runaway usage could still result in a bad time.
Re: Cloud Run GPUs, now GA, makes running AI workloads easier for everyone
#30Reason Cloud Run is so nice compared to other providers is that it has autoscaling, with scaling to 0. Meaning it can cost basically 0 if it's not being used. Also can set a cap on the scaling, e.g. 5 instances max, which caps the max cost of the service too. - Note, I only have experience with the CPU version of Cloud Run, (which is very reliable / easy).
Even regular Cloud Run can take a lot of time to boot (~3 to 30 seconds), so this can be a problem when scaling to 0
I’m looking at logs for a service I run on cloud run right now which scales to zero. Boot times are approximate 200ms for a Dart backend.