181–190 of 191 posts
i wonder what all this hype-driven overcapacity will be used for by future generations. once this bubble pops we are going to have some serious albeit high-latency hardware
> overcapacity I’m Not sure that word means what you think it means. There is a pretty severe shortage of GPU capacity in the industry right now.
I've been using this for daily/weekly ETL tasks which saves quite a lot of money vs having an instance on all the time but it's been clunky. The main issue is despite there being a 60 minute timeout available the API will just straight up not return a response code if your request takes > ~5 minutes in most cases so you gotta make sure you can poll where the datas being stored and let the client time out
The pricing doesn't look that compelling, here are the hourly rate comparisons vs runpod.io vs vast.ai: 1x L4 24GB: google: $0.71; runpod.io: $0.43, spot: $0.22 4x L4 24GB: google: $4.00; runpod.io: $1.72, spot: $0.88 1x A100 80GB: google: $5.07; runpod.io: $1.64, spot: $0.82; vast.ai $0.880, spot: $0.501 1x H100 80GB: google: $11.06; runpod.io: $2.79, spot: $1.65; vast.ai $1.535, spot: $0.473 8x H200 141GB: google:…
> Google's pricing also assumes you're running it 24/7 for an entire month What makes you think that? Cloud Run [pricing page]( https://cloud.google.com/run/pricing ) explicitly says : "charge you only for the resources you use, rounded up to the nearest 100 millisecond" Also, Cloud Run's [autoscalling]( https://cloud.google.com/run/docs/about-instance-autoscaling ) is in effect, scaling down idle instances after a m…
If you wanted to show hourly pricing, you would show that first, then calculate the monthly price from the hourly rate. I've no idea if the monthly cost includes sustained usage discount and what the hourly cost is for just running it for an hour.
Earlier quoted context omitted.
> Google's pricing also assumes you're running it 24/7 for an entire month What makes you think that? Cloud Run [pricing page]( https://cloud.google.com/run/pricing ) explicitly says : "charge you only for the resources you use, rounded up to the nearest 100 millisecond" Also, Cloud Run's [autoscalling]( https://cloud.google.com/run/docs/about-instance-autoscaling ) is in effect, scaling down idle instances after a m…
Because the pricing when creating an instance shows me the cost for the entire month, then works out the average hourly price based on that. This is just creating a GPU VM instance, I don't see how to see the cost of different NVidia GPUs without it. If you wanted to show hourly pricing, you would show that first, then calculate the monthly price from the hourly rate. I've no idea if the monthly cost includes sustain…
Are you referring to the GCP pricing calculator?
> This is just creating a GPU VM instance
Maybe you are referring to the Compute Engine VM creation page? Cloud Run is a different GCP service.
The Cloud Run Service creation UI doesn't show the cost.
Earlier quoted context omitted.
Has this changed? When I looked pre-ga the requirements were you need to pay for the CPU 24x7 to attach a GPU so that is not really scaling to zero unless this requirement has changed...
Speaking from my experience, it does scale to zero except you pay for 15 mins after the last request. So if you get all your requests in a 2 hours window then that's great. It will scale to zero for rest of the 22 hours. However, if you get at least one request every 15 mins then you will pay for 24 hours and it is ~3X more expensive then equivalent VM on Google Cloud.
Earlier quoted context omitted.
You're thinking of gvisor. But no, the "gen2" runtime is a microvm ala firecracker and performs a lot better as a result.
https://github.com/cloud-hypervisor/cloud-hypervisor or something else?
Earlier quoted context omitted.
Setting max instances effectively caps your spend right?
Yes. CR has had this feature since day 1, people just don't bother to read the docs and would rather write long blog posts blaming their cloud provider for manufacturing the gun they shot themself in the foot with.
Earlier quoted context omitted.
https://github.com/claceio/clace is project I am building which gives a Cloud Run type deployment experience on your own VMs. For each app, it supports scale down to zero containers (scaling up beyond one is being built). The authorization and auditing features are designed for internal tools, any app can be deployed otherwise.
Have a look at Knative
Clace is built for the use case of deploying internal tools, so it comes out of the box with CI/CD, auditing, OAuth etc. With Kubernetes, you need to glue together ArgoCD, an IDP etc to get the same.