Azure CLI command should be Parameters based, so that user can enter his any values to parameters and deployment should be proceed.any one know how to set this parameter.
example- I want to create one azure resource group and the value is user-defined.
az group create --name "user-defined" --location "East US 2"
In Windows, you could use Read-Host to promote user to input the parameter value.
$groupName = Read-Host -Prompt 'Input your group name'
az group create --name $groupName --location "East US 2"
In Linux, you could use read -p to promote user to input the parameter value.
read -p "Enter your groupname: " myname
az group create --name $myname --location "East US 2"
Related
I have the following step in the GitHub workflow that creates an organization in an AWS account and sets AWS Organization Root ID. The problem is that this step fails in Github workflow with exit code 254, but works if I run it via local command line. It's failing because the AWS CLI is returning an error An error occurred (AWSOrganizationsNotInUseException) when calling the DescribeOrganization operation: Your account is not a member of an organization. which expected and the script is supposed to handle it. I'm redirecting all output to stdout, so I'm really scratching my head here trying to figure out why the script is exiting instead of handling failures.
- name: Setup Organization
id: org
run: |
response=$(aws organizations describe-organization 2>&1)
if [ $? -ne 0 ]; then
if echo ${response} | grep -q "AWSOrganizationsNotInUseException"; then
echo "Creating organization..."
aws organizations create-organization --feature-set "ALL"
fi
fi
echo "::set-output name=ROOT_ID::$(aws organizations list-roots --query 'Roots[0].[Id]' --output text)"
shell: bash
Looks like Github Actions executes bash scripts in fail-fast mode (bash --noprofile --norc -e -o pipefail {0}) by default. I had to turn it off to take full control by passing bash {0} to the shell parameter. The following step works in the workflow now in case anyone is struggling with a similar issue.
- name: Setup Organization
id: org
run: |
response=$(aws organizations describe-organization 2>&1)
if [ $? -ne 0 ]; then
if echo ${response} | grep -q "AWSOrganizationsNotInUseException"; then
echo "Creating organization..."
aws organizations create-organization --feature-set "ALL"
fi
fi
echo "::set-output name=ROOT_ID::$(aws organizations list-roots --query 'Roots[0].[Id]' --output text)"
shell: bash {0}
I have two arrays with numbers that are already stored in variables:
$SLOT_IDS = [1,2,3,4,5]
$PR_IDS = [3,4]
I would like to find which numbers are in array 1 but not array 2. So in this case it would be
$OBSOLETE_SLOT_IDS = [1,2,5]
and then I would like to run a command foreach of those numbers and insert it into a placeholder:
So this command:
az webapp deployment slot delete -g group --name webapp --slot pr-<PLACEHOLDER>
Should be run three times:
az webapp deployment slot delete -g group --name webapp --slot pr-1
az webapp deployment slot delete -g group --name webapp --slot pr-2
az webapp deployment slot delete -g group --name webapp --slot pr-5
I know that should look something like this (it is required that it is inline):
for i in $OBSOLETE_SLOT_IDS; do az webapp deployment slot delete -g group --name webapp --slot pr-$i; done
So my questions:
How can I calculte $OBSOLETE_SLOT_IDS from the other two variables with an inline command
What is the correct version of the for loop
comment: seems that the variables do not contain actual arrays. They are basically the return values of some curl calls that I stored in variables:
A shorter approach that uses jq to get the difference of the two arrays:
#!/usr/bin/env bash
slot_ids="[1,2,3,4,5]"
pr_ids="[3,4]"
while read -r id; do
az webapp deployment slot delete -g group --name webapp --slot "pr-$id"
done < <(jq -n --argjson a "$slot_ids" --argjson b "$pr_ids" '$a - $b | .[]')
jq -r '.[]' will transform your array to a stream with one number per line -- which is the format that standard UNIX tools expect to work with.
Once we have the numbers in a sorted form, we can use comm to compare the two streams. -3 tells comm to ignore contents present in both streams, and -2 tells it to ignore content present only in the second stream, so comm -23 prints only files unique to the first stream.
Using readarray (added in bash 4.0) then lets us read that content into an array, which we can iterate over in our for loop.
#!/usr/bin/env bash
slot_ids='[1,2,3,4,5]'
pr_ids='[3,4]'
readarray -t filtered_slot_ids <(
comm -23 \
<(jq -r '.[]' <<<"$slot_ids" | sort) \
<(jq -r '.[]' <<<"$pr_ids" | sort))
for i in "${filtered_slot_ids[#]}"; do
az webapp deployment slot delete -g group --name webapp --slot "pr-$i"
done
How do I make my cli script wait until a resource is provisioned before attempting the next operation?
For example I am creating a waf policy and then attempting to assign that waf policy to an app gateway
The issue is that the waf policy is still being created
# WAF Policy
az network application-gateway waf-policy create \
--name $wafPolicyName
--resource-group $resourceGroupName
# App Gateway - wont allow to create without a private IP
az network application-gateway create \
--name $appGatewayName \
--resource-group $resourceGroupName \
--waf-policy $wafPolicyName
This result is an error:- Another operation on this or dependent resource is in progress
How do I make it wait?
Try using this: az network application-gateway waf-policy wait --name $wafPolicyName --resource-group $resourceGroupName --created
Here is a link about how to use az network application-gateway waf-policy wait.
The wait command works perfectly on my side:
I have a Docker container that performs a single large computation. This computation requires lots of memory and takes about 12 hours to run.
I can create a Google Compute Engine VM of the appropriate size and use the "Deploy a container image to this VM instance" option to run this job perfectly. However once the job is finished the container quits but the VM is still running (and charging).
How can I make the VM exit/stop/delete when the container exits?
When the VM is in its zombie mode only the stackdriver containers are left running:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
bfa2feb03180 gcr.io/stackdriver-agents/stackdriver-logging-agent:0.2-1.5.33-1-1 "/entrypoint.sh /u..." 17 hours ago Up 17 hours stackdriver-logging-agent
161439a487c2 gcr.io/stackdriver-agents/stackdriver-metadata-agent:0.2-0.0.17-2 "/bin/sh -c /opt/s..." 17 hours ago Up 17 hours 8000/tcp stackdriver-metadata-agent
I create the VM like this:
gcloud beta compute --project=abc instances create-with-container vm-name \
--zone=us-central1-c --machine-type=custom-1-65536-ext \
--network=default --network-tier=PREMIUM --metadata=google-logging-enabled=true \
--maintenance-policy=MIGRATE \
--service-account=xyz \
--scopes=https://www.googleapis.com/auth/cloud-platform \
--image=cos-stable-69-10895-71-0 --image-project=cos-cloud --boot-disk-size=10GB \
--boot-disk-type=pd-standard --boot-disk-device-name=vm-name \
--container-image=gcr.io/abc/my-image --container-restart-policy=on-failure \
--container-command=python3 \
--container-arg="a" --container-arg="b" --container-arg="c" \
--labels=container-vm=cos-stable-69-10895-71-0
When you create the VM, you'll need to give it write access to compute so you can delete the instance from within. You should also set container environment variables like gce_zone and gce_project_id at this time. You'll need them to delete the instance.
gcloud beta compute instances create-with-container {NAME} \
--container-env=gce_zone={ZONE},gce_project_id={PROJECT_ID} \
--service-account={SERVICE_ACCOUNT} \
--scopes=https://www.googleapis.com/auth/compute,...
...
Then within the container, whenever YOU determine your task is finished:
request an api token (im using curl for simplicity and DEFAULT gce service account)
curl "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" -H "Metadata-Flavor: Google"
This will respond with json that looks like
{
"access_token": "foobarbaz...",
"expires_in": 1234,
"token_type": "Bearer"
}
Take that access token and hit the instances.delete api endpoint (notice the environment variables)
curl -XDELETE -H 'Authorization: Bearer {TOKEN}' https://www.googleapis.com/compute/v1/projects/$gce_project_id/zones/$gce_zone/instances/$HOSTNAME
Having grappled with the problem for some time, here's a full solution that works pretty well.
This solution doesn't use the "start machine with a container image" option. Instead it uses a startup script, which is more flexible. You still use a Container-Optimized OS instance.
Create a startup script:
#!/usr/bin/env bash
# get image name and container parameters from the metadata
IMAGE_NAME=$(curl http://metadata.google.internal/computeMetadata/v1/instance/attributes/image_name -H "Metadata-Flavor: Google")
CONTAINER_PARAM=$(curl http://metadata.google.internal/computeMetadata/v1/instance/attributes/container_param -H "Metadata-Flavor: Google")
# This is needed if you are using a private images in GCP Container Registry
# (possibly also for the gcp log driver?)
sudo HOME=/home/root /usr/bin/docker-credential-gcr configure-docker
# Run! The logs will go to stack driver
sudo HOME=/home/root docker run --log-driver=gcplogs ${IMAGE_NAME} ${CONTAINER_PARAM}
# Get the zone
zoneMetadata=$(curl "http://metadata.google.internal/computeMetadata/v1/instance/zone" -H "Metadata-Flavor:Google")
# Split on / and get the 4th element to get the actual zone name
IFS=$'/'
zoneMetadataSplit=($zoneMetadata)
ZONE="${zoneMetadataSplit[3]}"
# Run compute delete on the current instance. Need to run in a container
# because COS machines don't come with gcloud installed
docker run --entrypoint "gcloud" google/cloud-sdk:alpine compute instances delete ${HOSTNAME} --delete-disks=all --zone=${ZONE}
Put the script somewhere public. For example put it on Cloud Storage and create a public URL. You can't use a gs:// URI for a COS startup script.
Start an instance using a startup-script-url, and passing the image name and parameters, e.g.:
gcloud compute --project=PROJECT_NAME instances create INSTANCE_NAME \
--zone=ZONE --machine-type=TYPE \
--metadata=image_name=IMAGE_NAME,\
container_param="PARAM1 PARAM2 PARAM3",\
startup-script-url=PUBLIC_SCRIPT_URL \
--maintenance-policy=MIGRATE --service-account=SERVICE_ACCUNT \
--scopes=https://www.googleapis.com/auth/cloud-platform --image-family=cos-stable \
--image-project=cos-cloud --boot-disk-size=10GB --boot-disk-device-name=DISK_NAME
(You probably want to limit the scopes, the example uses full access for simplicity)
I wrote a self-contained Python function based on Vincent's answer.
def kill_vm():
"""
If we are running inside a GCE VM, kill it.
"""
# based on https://stackoverflow.com/q/52748332/321772
import json
import logging
import requests
# get the token
r = json.loads(
requests.get("http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token",
headers={"Metadata-Flavor": "Google"})
.text)
token = r["access_token"]
# get instance metadata
# based on https://cloud.google.com/compute/docs/storing-retrieving-metadata
project_id = requests.get("http://metadata.google.internal/computeMetadata/v1/project/project-id",
headers={"Metadata-Flavor": "Google"}).text
name = requests.get("http://metadata.google.internal/computeMetadata/v1/instance/name",
headers={"Metadata-Flavor": "Google"}).text
zone_long = requests.get("http://metadata.google.internal/computeMetadata/v1/instance/zone",
headers={"Metadata-Flavor": "Google"}).text
zone = zone_long.split("/")[-1]
# shut ourselves down
logging.info("Calling API to delete this VM, {zone}/{name}".format(zone=zone, name=name))
requests.delete("https://www.googleapis.com/compute/v1/projects/{project_id}/zones/{zone}/instances/{name}"
.format(project_id=project_id, zone=zone, name=name),
headers={"Authorization": "Bearer {token}".format(token=token)})
A simple atexit hook gets me my desired behavior:
import atexit
atexit.register(kill_vm)
Another solution is to not use GCE and instead use AI Platform's custom job service, which automatically shuts down the VM after the Docker container exits.
gcloud ai-platform jobs submit training $JOB_NAME \
--region $REGION \
--master-image-uri $IMAGE_URI
You can specify --master-machine-type.
See the GCP documentation on custom containers.
The simplest way, from within the container, once it's finished:
ZONE=`gcloud compute instances list --filter="name=($HOSTNAME)" --format 'csv[no-heading](zone)'`
gcloud compute instances delete $HOSTNAME --zone=$ZONE -q
-q skips the interactive confirmation
$HOSTNAME is already exported
Just use curl and the local metadata server (no need for Python scripts or gcloud). Add the following to the end of your Docker Entrypoint script, so it's run when the container finishes:
# Note: inside the container the name is exposed as $HOSTNAME
INSTANCE_NAME=$(curl -sq "http://metadata.google.internal/computeMetadata/v1/instance/name" -H "Metadata-Flavor: Google")
INSTANCE_ZONE=$(curl -sq "http://metadata.google.internal/computeMetadata/v1/instance/zone" -H "Metadata-Flavor: Google")
echo "Terminating instance [${INSTANCE_NAME}] in zone [${INSTANCE_ZONE}}"
TOKEN=$(curl -sq "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token" -H "Metadata-Flavor: Google" | jq -r '.access_token')
curl -X DELETE -H "Authorization: Bearer ${TOKEN}" https://www.googleapis.com/compute/v1/$INSTANCE_ZONE/instances/$INSTANCE_NAME
For security sake, and Principle of Least Privilege, you can run the VM with a custom service account, and give that service account a role, with this permission (a custom role is best).
compute.instances.delete
The OpenShift command line tool (oc) offers a command to add a role to groups of users. The syntax is:
oc adm policy add-role-to-group ROLE GROUP [GROUP ...] [options]
In a script I found such command with option "-n" but there's no way I can find in the oc reference documentation a description of this or other allowed options.
Worse: it seems developers of the oc tool are trying to kid you, as the image shows.
I'm using oc version:
oc v3.2.1.13-1-gc2a90e1
kubernetes v1.2.0-36-g4a3f9c5
By running the command oc adm options you can see that the -n option is for the following:
-n, --namespace='': If present, the namespace scope for this CLI request.
-n is simply for passing a project name, so your command will run against this project.
e.g. oc add-role-to-group admin groupx -n projectx
will assign admin role to groupx on projectx