I've been creating dataproc clusters successfully over the past couple of weeks using the following gcloud command:
gcloud dataproc --region us-east1 clusters create test1 --subnet
default --zone us-east1-c --master-machine-type n1-standard-4
--master-boot-disk-size 250 --num-workers 10 --worker-machine-type n1-standard-4 --worker-boot-disk-size 200 --num-worker-local-ssds 1
--image-version 1.2 --scopes 'https://www.googleapis.com/auth/cloud-platform' --project MyProject
--initialization-actions gs://MyBucket/MyScript.sh
But today I'm getting the following error when I try to create dataproc cluster from either gcloud cli or the GCP web console:
ERROR: (gcloud.dataproc.clusters.create) Operation
[projects/MyProject/regions/us-east1/operations/SOMELONGIDHERE]
failed: Invalid value for field
'resource.disks[1].initializeParams.labels': ''. Cannot specify
initializeParams.labels for local SSD..
I tried changing the cluster name and the zone (not region), without any success.
Thanks in advance
There was an issue on Google's end that was corrected.
It should be working now.
Related
Here's my problem :
I would like to connect to a gcp instance. When I run the Google Cloud SDK shell as an administrator with the command :
gcloud compute ssh my_instance --zone=europe-west1-b -- -L=8081:locahost:8081
..I get this error : ERROR (gcloud.compute.ssh) [..../putty.exe] exited with return code [1]
My instance is running with the metadata enable-oslogin as TRUE, as the project.
Do you have an idea of what is the problem ?
When using -- in the command, you are passing SSH flags after the dashes and not gcloud command flags. To explain, gcloud compute ssh is a thin wrapper around the ssh(1) command that takes care of authentication and the translation of the instance name into an IP address.
In this case, -- is equivalent to --ssh-flag as per this SDK reference. It seems that putty is outputting an error that is not passed into the command line (SDK shell). The actual error should be visible in the dialog window before putty exits.
I have tried the command myself on Windows and the exact error was unknown option "L=8081:localhost:8081". The SSH flag is not accepted as you have an = sign there (typo).
According to linuxcommand.org manual, the flag should be in this format:
-L [bind_address:]port:host:hostport
Hence, you should run the command like this:
gcloud compute ssh my_instance --zone=europe-west1-b -- -L 8081:locahost:8081
Note also that you may have to create a firewall rule to allow Ingress to the instance on port 8081.
I try to create an autoscaled container cluster on GKE.
When I use the "--enable-autoscaling" option (like the documentation indicates here : https://cloud.google.com/container-engine/docs/clusters/operations#create_a_cluster_with_autoscaling) :
$ gcloud container clusters create mycluster --zone $GOOGLE_ZONE --num-nodes=3 --enable-autoscaling --min-nodes=2 --max-nodes=5
but the MIG (Managed Instanced Group) is not displayed as 'autoscaled' as shown by both the web interface and the result of the following command :
$ gcloud compute instance-groups managed list
NAME SIZE TARGET_SIZE AUTOSCALED
gke-mycluster... 3 3 no
Why ?
Then, I tried the other way indicated in the kubernetes docs (http://kubernetes.io/docs/admin/cluster-management/#cluster-autoscaling) but got an error caused by the '=true' apparently :
$ gcloud container clusters create mytestcluster --zone=$GOOGLE_ZONE --enable-autoscaling=true --min-nodes=2 --max-nodes=5 --num-nodes=3
usage: gcloud container clusters update NAME [optional flags]
ERROR: (gcloud.container.clusters.update) argument --enable-autoscaling: ignored explicit argument 'true'
Is the doc wrong on this ?
Here is my gcloud version results :
$ gcloud version
Google Cloud SDK 120.0.0
beta 2016.01.12
bq 2.0.24
bq-nix 2.0.24
core 2016.07.29
core-nix 2016.03.28
gcloud
gsutil 4.20
gsutil-nix 4.18
kubectl
kubectl-linux-x86_64 1.3.3
Last precision : the autoscaler seems 'on' in the description on the cluster :
$ gcloud container clusters describe mycluster | grep auto -A 3
- autoscaling:
enabled: true
maxNodeCount: 5
minNodeCount: 2
Any idea to explain this behaviour please ?
Kubernetes cluster autoscaling does not use the Managed Instance Group autoscaler. It runs a cluster-autoscaler controller on the Kubernetes master that uses Kubernetes-specific signals to scale your nodes. The code is in the autoscaler repo if you want more info.
I've also sent out a PR to fix the invalid flag usage in the autoscaling docs. Thanks for catching that!
I am trying to follow this tutorial https://tensorflow.github.io/serving/serving_inception
But I see this
$ gcloud container clusters create inception-serving-cluster --num-nodes 5
ERROR: (gcloud.container.clusters.create) ResponseError: code=403, message=Required "container.clusters.create" permission for "projects/tensorflow-serving".
I did not see an option to add permissions to the project anywhere. How do I do this using the CLI or the UI?
EDIT:
I do have the project created
EDIT:
Just saw that it works fine from the cloud shell
Update: Your project's name is tensorflow-serving-1360, so you should be running gcloud container clusters create inception-serving-cluster --num-nodes 5 --project=tensorflow-serving-1360.
The project tensorflow-serving is not owned by you. It is the example project name used in the linked tutorial, but you need to replace it with the name of your own project as described in the line at the beginning of Part 2:
Here we assume you have created and logged in a gcloud project named
tensorflow-serving
(Tested on 2019.04.07)
Firstly, check the list of auth accounts:
gcloud auth list
Next set the active account:
gcloud config set account <email_address_from_above_output>
Then, specify the parameter for create cluster commamd:
gcloud container clusters create <cluster_name> --num-nodes=2 --project=<PROJECT_ID>
e.g.
gcloud container clusters create prod-myapp-cluster --num-nodes=2 --project=myapp-20394823094
Expected output:
kubeconfig entry generated for prod-myapp-cluster.
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
prod-myapp-cluster asia-south1-a 1.11.7-gke.12 35.5xx.2xx.1xx n1-standard-1 1.11.7-gke.12 2 RUNNING
Get your project name or create a project if you have created on already at console.cloud.google.com
Enable Kubernetes engine API on the console
run this code on your command prompt
gcloud container clusters create bd-serving-cluster --num-nodes 5 -project=tensorflow-serving-264611 \
--zone=us-central1-f
replace 'bd' with the name of your serving cluster and 'tensorflow-serving-264611' with the project name you created in step 1 and you can choose your preferred zone or use the default 'us-central1-f'
Creating an instance using gcloud does not seem to work:
google-cloud> gcloud compute instances create minecraft-instance --image ubuntu-14-10 --tags minecraft
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.create) Unable to fetch a list of zones. Specifying [--zone] may fix this issue:
- Project marked for deletion.
Adding the zone name fails differently:
google-cloud> gcloud compute instances create minecraft-instance --image ubuntu-14-10 --zone us-central1-a --tags minecraft
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.create) Failed to find image for alias [ubuntu-14-10] in public image project [ubuntu-os-cloud].
- Project marked for deletion.
Providing a different image name fails too:
google-cloud> gcloud compute instances create minecraft-instance --image ubuntu-1410-utopic --zone us-central1-a --tags minecraft
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.create) Could not fetch image resource:
- Project marked for deletion.
What is the exact command to create an instance using gcloud?
Did you authenticate before and set the default project?
gcloud auth login
gcloud config set project PROJECT
The base setup of gcloud is in the Google Cloud documentation.
Or did you delete your project?
Project marked for deletion.
You have several things going on, one of which is reading the docs:
https://cloud.google.com/compute/docs/gcloud-compute/#creating
You syntax should be:
gcloud compute instances create minecraftinstance \
--image ubuntu-14-10 \
--zone [SOME-ZONE-ID] \
--machine-type [SOME-MACHINE-TYPE]
Where SOME-ZONE-ID is a geographic zone to create the instance in, found by running:
gcloud compute zones list
SOME-MACHINE-TYPE is the machince type to create. Valid types are found by running:
gcloud compute machine-types list
But specifically, you seem to be creating an instance in a Project that has been deleted:
- Project marked for deletion.
Also, you need to authenticate and set a default project:
gcloud auth
and
gcloud config set project [ID]
Billable resources can not be created for projects which has been flagged for deletion. For a project to be deletable, billing must be disabled first, and so, instances can not be created. As for the error messages, it seems gcloud command is not handling this situation correctly and replying bogus error codes instead.
The only compulsory arguments to gcloud compute instances create are the name, the zone and the project. A valid working project must be set either by using --project PROJECT flag to gcloud commands, or by using gcloud config set project PROJECT before. Similarly, to choose the zone you can either use the --zone ZONE flag or the gcloud config set compute/zone ZONE command before.
Enabling billing on your current project and undeleting it will work too. To figure out which project and zone the gcloud command is running in by default, use this:
gcloud config list
In my case I had to specify --image-project that got me going:
gcloud compute instances create core --image ubuntu-1604-xenial-v20180126 --machine-type f1-micro --zone us-east4-a --image-project ubuntu-os-cloud
My Case,Create a managed instance group using the instance template:
gcloud compute instance-groups managed create nginx-group \
--base-instance-name nginx \
--size 2 \
--template nginx-template \
--target-pool nginx-pool \
--zone us-central1-c
You have to specify the --image-project --image-family
Refer https://cloud.google.com/compute/docs/images#os-compute-support.
I'm trying to delete the access config for one of my Google Compute Engine instances, and as described in some of the documentation, the access config for my instance is named "External NAT" rather than the default "external-nat". When I try to run:
gcloud compute instances delete-access-config my-instance-name --access-config-name="External NAT"
I get the following error:
ERROR: (gcloud.compute.instances.delete-access-config) unrecognized arguments: NAT
I'm assuming the error of the space in "External NAT". Seems like this should be a simple fix but I can't figure it out. Any help would be much appreciated!
You added "=" when in fact it is not needed. It worked as follows :
$ gcloud compute instances delete-access-config test-instance --access-config-name "External NAT"
Output:
Updated [https://www.googleapis.com/compute/v1/projects/test-project/zones/europe-west1-c/instances/test-instance].
gcloud compute instances delete-access-config test-instance --access-config-name="External NAT" --network-interface="nic0" --zone="us-east1-b"