My problem is to create 5k instances and retrieve there public IP addresses.
Specifically for zone us-west1-a I can create a group of 50 instances by the following:
gcloud compute instance-groups managed create test --base-instance-name morning --size 50 --template benchmark-template-micro --zone us-west1-a
Questions:
How to specify the start-script to run each created instances? I can't find them here.
How to get the public IP addresses of those created instances?
the startup-script can be assigned to the template for the instance used; see here.
one can obtain information with gcloud compute instance-groups managed describe.
while there are no public IP addresses unless you'd assign external IP addresses.
As mentioned by Martin, the startup-script is configured in the instance template.
Unfortunately, there is no API that lists the ip addresses of the instances in the group. There are however APIs (and gcloud commands) to get the list of instances and the ip addresses of instances. Here is an example to fetch this information from the command line:
gcloud compute instance-groups list-instances $INSTANCE_GROUP --uri \
| xargs -I '{}' gcloud compute instances describe '{}' \
--flatten networkInterfaces[].accessConfigs[] \
--format 'csv[no-heading](name,networkInterfaces.accessConfigs.natIP)'
To speed this up, you may want to use xarg's -P flag to parallelize the instance describe requests.
Since all instances in the group have the same prefix. You can also just do a list search by prefix. Although, this may pull in another that uses the same prefix even if not part of the instance group:
gcloud compute instances list --filter='name ~ ${PREFIX}*' \
--flatten networkInterfaces[].accessConfigs[] \
--format 'csv[no-heading](name,networkInterfaces.accessConfigs.natIP)'
Related
I have setup a compute instance called to run cronjobs on Google Compute engine using a service account with the following roles:
Custom Compute Image User + Deletion rights
Compute Admin
Compute Instance Admin (beta)
Kubernetes Engine Developer
Logs Writer
Logs Viewer
Pub/Sub Editor
Source Repository Reader
Storage Admin
Unfortunately, when I ssh into this cronjob runner instance and then run:
sudo gcloud compute --project {REDACTED} instances create e-latest \
--zone {REDACTED} --machine-type n1-highmem-8 --subnet default \
--maintenance-policy TERMINATE \
--scopes https://www.googleapis.com/auth/cloud-platform \
--boot-disk-size 200 \
--boot-disk-type pd-standard --boot-disk-device-name e-latest \
--image {REDACTED} --image-project {REDACTED} \
--service-account NAME_OF_SERVICE_ACCOUNT \
--accelerator type=nvidia-tesla-p100,count=1 --min-cpu-platform Automatic
I get the following error:
The user does not have access to service account {NAME_OF_SERVICE_ACCOUNT}. User: {NAME_OF_SERVICE_ACCOUNT} . Ask a project owner to grant you the iam.serviceAccountUser role on the service account.
Is there some other privilege besides compute instance admin that I need to be able to create instances with my instance?
Further notes: (1) when I try to not specify --service-account the error is the same except that the service account my user doesn't have access to is the default '51958873628-compute#developer.gserviceaccount.com'.
(2) adding/removing sudo doesn't change anything
Creating an instance that uses a service account requires you have the compute.instances.setServiceAccount permission on that service account. To make this work, grant the iam.serviceAccountUser role to your service account (either on the entire project or on the specific service account you want to be able to create instances with).
Find out who you are first
if you are using Web UI: what email address did you use to login?
if you are using local gcloud or terraform: find the json file that contains your credentials for gcloud (often named similarly to myproject*.json) and see if it contains the email: grep client_email myproject*.json
GCP IAM change
Go to https://console.cloud.google.com
Go to IAM
Find your email address
Member -> Edit -> Add Another Role -> type in the role name Service Account User -> Add
(You can narrow it down with a Condition, but lets keep it simple for a while).
Make sure that NAME_OF_SERVICE_ACCOUNT is service account from current project.
If you change project ID, and don't change NAME_OF_SERVICE_ACCOUNT, then you will encounter this error.
This can be checked on Google Console -> IAM & Admin -> IAM.
Then look for service name ....-compute#developer.gserviceaccount.com and check if numbers at the beginning are correct. Each project will have different numbers in this service name.
I have a bunch of GCE instances and I want to run the same shell command on all of them. Is it possible to do something like gcloud compute ssh --command="ls -al" my-instance1 my-instance2 my-instance3?
You can use gcloud compute instances list --format='value[separator=","](name,zone)' to get a list like:
my-instance1,my-zone1
my-instance2,my-zone2
my-instance3,my-zone3
Then you can use bash Substring Removal to extract the parts before and after the comma.
var="before,after"
before="${var%,*}"
after="${var#*,}"
Put it all in a loop and add trailing '&' to run things in the background:
for instance in $(gcloud compute instances list --format='value[separator=","](name,zone)'); do
name="${instance%,*}";
zone="${instance#*,}";
gcloud compute ssh $name --zone=$zone --command="ls -al" &
done
To add to Mike's answer, here is how to do the same without Substring Removal:
for i z in $(gcloud compute instances list --format='value(name, zone)'); do
gcloud compute ssh $i --command="ls -lah" --zone=$z;
done
Hi I am trying list compute instances in a specific network, and subnetwork, and can't seem to get the filtering right. For example, I have a network named "prod-net" with a subnetwork named "app-central". When I run the search I just get "Listed 0 items".
~ gcloud compute instances list --filter='network:prod-net'
Listed 0 items.
Any suggestions?
The --filter flag doesn't operate on the table data, but rather the underlying rich resource object. To see this object, run gcloud compute instances list --format=json.
What you're looking for in this case is:
$ gcloud compute instances list --filter='networkInterfaces.network=prod-net'
(I switched the : to = because the former means "contains" and the latter means an exact match. See gcloud topic filters for more).
You can indeed filter GCE instances by subnetwork using gcloud.
You need to filter by networkInterfaces.subnetwork and the literal value to compare with, is the full subnet resource url, not just the subnet-name.
The "resource url" for your subnet can be obtained by:
gcloud compute networks subnets list <YOUR_SUBNET_NAME> --format=flattened
Example:
$ gcloud compute networks subnets list sg-zk-1 --project my-gcp-project --format=flattened
---
creationTimestamp: 2017-04-20T02:22:17.853-07:00
gatewayAddress: 10.9.19.33
id: 6783412628763296550
ipCidrRange: 10.9.19.32/28
kind: compute#subnetwork
name: sg-zk-1
network: valkyrie
privateIpGoogleAccess: True
region: asia-southeast1
selfLink: https://www.googleapis.com/compute/v1/projects/my-gcp-project/regions/asia-southeast1/subnetworks/sg-zk-1
In the above example, the subnet-name is sg-zk-1.
The corresponding resource URL for the subnet is the value of the selfLink which is https://www.googleapis.com/compute/v1/projects/my-gcp-project/regions/asia-southeast1/subnetworks/sg-zk-1.
Now that I have the subnet_url I can filter the instances belonging to it:
$ subnet_url="https://www.googleapis.com/compute/v1/projects/my-gcp-project/regions/asia-southeast1/subnetworks/sg-zk-1"
$ gcloud compute instances list --filter="networkInterfaces.subnetwork=${subnet_url}"
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
sg-zookeeper-4 asia-southeast1-b n1-standard-2 10.9.19.37 RUNNING
sg-zookeeper-5 asia-southeast1-b n1-standard-2 10.9.19.38 RUNNING
sg-zookeeper-1 asia-southeast1-a n1-standard-2 10.9.19.34 RUNNING
sg-zookeeper-2 asia-southeast1-a n1-standard-2 10.9.19.35 RUNNING
sg-zookeeper-3 asia-southeast1-a n1-standard-2 10.9.19.36 RUNNING
Creating an instance using gcloud does not seem to work:
google-cloud> gcloud compute instances create minecraft-instance --image ubuntu-14-10 --tags minecraft
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.create) Unable to fetch a list of zones. Specifying [--zone] may fix this issue:
- Project marked for deletion.
Adding the zone name fails differently:
google-cloud> gcloud compute instances create minecraft-instance --image ubuntu-14-10 --zone us-central1-a --tags minecraft
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.create) Failed to find image for alias [ubuntu-14-10] in public image project [ubuntu-os-cloud].
- Project marked for deletion.
Providing a different image name fails too:
google-cloud> gcloud compute instances create minecraft-instance --image ubuntu-1410-utopic --zone us-central1-a --tags minecraft
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS
ERROR: (gcloud.compute.instances.create) Could not fetch image resource:
- Project marked for deletion.
What is the exact command to create an instance using gcloud?
Did you authenticate before and set the default project?
gcloud auth login
gcloud config set project PROJECT
The base setup of gcloud is in the Google Cloud documentation.
Or did you delete your project?
Project marked for deletion.
You have several things going on, one of which is reading the docs:
https://cloud.google.com/compute/docs/gcloud-compute/#creating
You syntax should be:
gcloud compute instances create minecraftinstance \
--image ubuntu-14-10 \
--zone [SOME-ZONE-ID] \
--machine-type [SOME-MACHINE-TYPE]
Where SOME-ZONE-ID is a geographic zone to create the instance in, found by running:
gcloud compute zones list
SOME-MACHINE-TYPE is the machince type to create. Valid types are found by running:
gcloud compute machine-types list
But specifically, you seem to be creating an instance in a Project that has been deleted:
- Project marked for deletion.
Also, you need to authenticate and set a default project:
gcloud auth
and
gcloud config set project [ID]
Billable resources can not be created for projects which has been flagged for deletion. For a project to be deletable, billing must be disabled first, and so, instances can not be created. As for the error messages, it seems gcloud command is not handling this situation correctly and replying bogus error codes instead.
The only compulsory arguments to gcloud compute instances create are the name, the zone and the project. A valid working project must be set either by using --project PROJECT flag to gcloud commands, or by using gcloud config set project PROJECT before. Similarly, to choose the zone you can either use the --zone ZONE flag or the gcloud config set compute/zone ZONE command before.
Enabling billing on your current project and undeleting it will work too. To figure out which project and zone the gcloud command is running in by default, use this:
gcloud config list
In my case I had to specify --image-project that got me going:
gcloud compute instances create core --image ubuntu-1604-xenial-v20180126 --machine-type f1-micro --zone us-east4-a --image-project ubuntu-os-cloud
My Case,Create a managed instance group using the instance template:
gcloud compute instance-groups managed create nginx-group \
--base-instance-name nginx \
--size 2 \
--template nginx-template \
--target-pool nginx-pool \
--zone us-central1-c
You have to specify the --image-project --image-family
Refer https://cloud.google.com/compute/docs/images#os-compute-support.
This is a question in two parts:
Do GCE instances have a stable public DNS name? The default DNS name for instance with public IP a.b.c.d seems to be d.c.b.a.bc.googleusercontent.com
If yes, what's the best way to obtain this information? Here's the hack I've been using thus far:
EXTERNAL_IP=$(curl -s http://bot.whatismyipaddress.com/)
EXTERNAL_DNS=$(dig +short -x ${EXTERNAL_IP})
reverse lookup is okay to do, for IP address you would probably prefer using gcutil
https://developers.google.com/compute/docs/gcutil/tips
EXTERNAL_IP=$(gcutil getinstance --format=csv --zone=[your_zone] [your_instance] | grep external-ip | cut -d "," -f 2)
GCE instances don't currently have a public DNS name for their external IP address. But there is now a gcloud compute config-ssh (docs) command that's a pretty good substitute.
This will insert Host blocks into your ~/.ssh/config file that contain the IP address and configuration for the host key.
Although this only helps with SSH (and SSH-based applications like Mosh and git+ssh), it does have a few advantages over DNS:
There is no caching/propagation delay as you might have with DNS
It pre-populates the right host key, and the host key is checked the right way even if the ephemeral IP address changes.
Example:
$ gcloud compute config-ssh
...
$ ssh myhost.us-west1-b.surly-koala-232