Is this just about creating new pools? I haven't noticed an issue with our existing pools scaling.
You were able to add more Nodes to you're pool? Are you using any auto scaling?
Google Kubernetes Engine's third consecutive day of service disruption
11–20 of 419 posts
Re: Google Kubernetes Engine's third consecutive day of service disruption
#12Earlier quoted context omitted.
What operations? Status just shows node pool creation.
Can not create a new Clusters or Node Pool and can not resize exiting Node Pools, as far as users are reporting it's happening in all regions too. Error message when creating a new Cluster: Deploy error: Not all instances running in IGM after 35m7.509000994s. Expect 1. Current errors: [ZONE_RESOURCE_POOL_EXHAUSTED]: Instance 'gke-cluster-3-pool-1-41b0abf8-73d7' creation failed: The zone 'projects/url-shortner-218503/…
Hi, just curious: why are you in us-west2?
Re: Google Kubernetes Engine's third consecutive day of service disruption
#13Earlier quoted context omitted.
Can not create a new Clusters or Node Pool and can not resize exiting Node Pools, as far as users are reporting it's happening in all regions too. Error message when creating a new Cluster: Deploy error: Not all instances running in IGM after 35m7.509000994s. Expect 1. Current errors: [ZONE_RESOURCE_POOL_EXHAUSTED]: Instance 'gke-cluster-3-pool-1-41b0abf8-73d7' creation failed: The zone 'projects/url-shortner-218503/…
Hi, just curious: why are you in us-west2?
Just the latest region I tried, been trying to spin up Clusters around the country all day.
Re: Google Kubernetes Engine's third consecutive day of service disruption
#14This has been going on longer than three days. We have been dealing with this exact issue since at least Monday (11/5) morning in us-central1.
Re: Google Kubernetes Engine's third consecutive day of service disruption
#15Status page is inaccurate as issues doesn't only affect the web UI, the same operations are not functioning via the CLI.
I just tried turning up my 1-node test cluster via terraform, and it worked fine. I would have thought the gcloud CLI would be using the same API.
I did this in the australia-southeast1-a zone.
Re: Google Kubernetes Engine's third consecutive day of service disruption
#16Been trying to spin up vm instances all day, had to try every single zone just to get one up. Not only is this incredibly harmful to a technology business dependent on this infra, it wasnt obvious to me what the issue was until I tried creating instances. Nothing says, hey resources are constrained here, try this one. Just about ready to bite the bullet and move to aws.
Re: Google Kubernetes Engine's third consecutive day of service disruption
#17I use preemptible machines in autodialing and for first time did not have any machines available for multiple hours yesterday. I am wondering whether this falls under the normal preemptible behaviour or this service degradation.
Re: Google Kubernetes Engine's third consecutive day of service disruption
#18Been trying to spin up vm instances all day, had to try every single zone just to get one up. Not only is this incredibly harmful to a technology business dependent on this infra, it wasnt obvious to me what the issue was until I tried creating instances. Nothing says, hey resources are constrained here, try this one. Just about ready to bite the bullet and move to aws.
Same here. We have spent 2 days trying to create instances and migrate images just to figure out later they can't start.
Right when I convinced our project to get migrated from AWS...
Re: Google Kubernetes Engine's third consecutive day of service disruption
#19This has been going on longer than three days. We have been dealing with this exact issue since at least Monday (11/5) morning in us-central1.
same here. using gcloud, not web console
Re: Google Kubernetes Engine's third consecutive day of service disruption
#20Status page is inaccurate as issues doesn't only affect the web UI, the same operations are not functioning via the CLI.
Yeah almost all regions and zones for any compute instance have been exhausted since about 1pm PST on Friday. I finally got one up last night on us-east1, but my older cluster is basically SOL until it's fixed on us-west1. It went down for an upgrade and never came back up because of the same resource issue.