I need to replace a variable that has the extra argument passed to it via CLI.
./deploy.yml -e 'jira_ticket=REL-78'
I can't get it to strip out 'deploylist/REL-78' because I'm passing in 'jira_ticket' If i hardcode the varible ('REL-78') it works perfectly.
- name: Set fact
set_fact: deploy_list"{{ item | replace('deploylist/{{ jira_ticket }}/', '')"
with_items: ' {{ modules_to_deploy.value }}'
register: deploy_list_result
ok: [127.0.0.1] => (item=deploylist/REL-78/api)
ok: [127.0.0.1] => (item=deploylist/REL-78/ariaapi)
ok: [127.0.0.1] => (item=deploylist/REL-78/ariaquery)
ok: [127.0.0.1] => (item=deploylist/REL-78/ariaserver)
ok: [127.0.0.1] => (item=deploylist/REL-78/dashboardidp)
ok: [127.0.0.1] => (item=deploylist/REL-78/oracle)
ok: [127.0.0.1] => (item=deploylist/REL-78/uisp)
ok: [127.0.0.1] => (item=deploylist/REL-78/ui)
How about this... (I also added = and closing }})
- name: Set fact
set_fact: deploy_list="{{ item | replace('deploylist/'+jira_ticket+'/', '') }}"
with_items: modules_to_deploy.value
register: deploy_list_result
Note that you don't have to stringify an argument for with_items.
Related
I have a playbook to get all disks letter configured on my server and I need a task to verify if extra var letter is on the list.
For example I need to check if "F" is on the json data below.
Could you please help me on the best best syntax?
Thanks
{
"disks_drives_letter": [
[
"C"
],
[
"D"
],
[
"E"
],
[]
]
}
You can use setup module to get your host information like disks. For more information about the setup module https://docs.ansible.com/ansible/latest/collections/ansible/builtin/setup_module.html
Example of playbook:
- hosts: localhost
tasks:
gather_facts: false
vars:
my_disk_drives: ['sda', 'sdb']
tasks:
- name: Collect host hardware information
setup:
gather_subset:
- hardware
- name: Output if disk exist
debug:
msg: "{{ item }} exists"
loop: "{{ my_disk_drives }}"
when: item in hostvars[inventory_hostname].ansible_devices.keys() | list
- name: Output if disks does not exist
debug:
msg: "{{ item }} does not exist"
loop: "{{ my_disk_drives }}"
when: not item in hostvars[inventory_hostname].ansible_devices.keys() | list
Output:
TASK [Output if disk exist]
ok: [localhost] => (item=sda) => {
"msg": "sda exists"
}
skipping: [localhost] => (item=sdb)
TASK [Output if disks does not exist]
skipping: [localhost] => (item=sda)
ok: [localhost] => (item=sdb) => {
"msg": "sdb does not exist"
}
Use filters intersect and difference, and declare the lists
my_disks_exist: "{{ ansible_devices.keys()|intersect(my_disks) }}"
my_disks_not_exist: "{{ my_disks|difference(my_disks_exist) }}"
Example of a complete playbook for testing
- hosts: localhost
vars:
my_disks: [sda, sdb, sdc]
my_disks_exist: "{{ ansible_devices.keys()|intersect(my_disks) }}"
my_disks_not_exist: "{{ my_disks|difference(my_disks_exist) }}"
tasks:
- setup:
gather_subset: devices
- debug:
var: ansible_devices.keys()
- debug:
var: my_disks_exist
- debug:
var: my_disks_not_exist
| flatten help me thanks #vladimir-botka
- name: Get all disks letter from the disks infos
set_fact:
disks_drives_letters: "{{ win_disk_facts | json_query(query) | flatten }}"
- name: Check if disk_letter is used on server fail:
msg: "The disk letter already exist on the VM" when: '"{{ drive_letter }}" in "{{ disks_drives_letters}}"'
I have the following requirement.
loop over a json data
Use the above data to run tasks in parallel
Example: My json data is as follows
{
"xyx":
{
"hostname": "xyz",
"username": "root",
"password": "welcome1",
"password2": "welcome2",
"version": "19.17",
"toversion" : "19.18",
"netconf": "bond",
"baseversion": "18.8"
},
"abc": {
"hostname": "abc",
"username": "root",
"password": "welcome1",
"password2": "welcome2",
"version": "19.17",
"toversion" : "19.18",
"netconf": "bond",
"baseversion": "18.8"
}
}
I need to run the following task in both hostnames abc and xyz in parallel
- name: Image a system
command: python3 image.pyc "{{ hostname }}" {{ version }}
Can someone please help achieve this ?
I tried to loop over the json but not sure how to use it in the tasks to run in parallel.
Given the dictionary in the variable images. Either use the filter dict2items
- name: Image a system
command: "python3 image.pyc {{ item.value.hostname }} {{ item.value.version }}"
loop: "{{ images|dict2items }}"
(not tested)
, or get the values from the dictionary
- name: Image a system
command: "python3 image.pyc {{ item.hostname }} {{ item.version }}"
loop: "{{ images.values()|list }}"
(not tested)
Optionally, you can create a dictionary for this purpose. Declare the below variable in vars as appropriate
hostname_version: "{{ images.values()|list|
items2dict(key_name='hostname',
value_name='version') }}"
gives
hostname_version:
abc: '19.17'
xyz: '19.17'
Then, the task below should give the same results
- name: Image a system
command: "python3 image.pyc {{ item.key }} {{ item.value }}"
loop: "{{ hostname_version|dict2items }}"
(not tested)
Example of a complete playbook for testing
shell> cat pb.yml
- hosts: localhost
vars:
images:
abc:
baseversion: '18.8'
hostname: abc
netconf: bond
password: welcome1
password2: welcome2
toversion: '19.18'
username: root
version: '19.17'
xyx:
baseversion: '18.8'
hostname: xyz
netconf: bond
password: welcome1
password2: welcome2
toversion: '19.18'
username: root
version: '19.17'
hostname_version: "{{ images.values()|list|
items2dict(key_name='hostname',
value_name='version') }}"
tasks:
- debug:
msg: "python3 image.pyc {{ item.value.hostname }} {{ item.value.version }}"
loop: "{{ images|dict2items }}"
loop_control:
label: "{{ item.key }}"
- debug:
msg: "python3 image.pyc {{ item.hostname }} {{ item.version }}"
loop: "{{ images.values()|list }}"
loop_control:
label: "{{ item.hostname }}"
- debug:
var: hostname_version
- debug:
msg: "python3 image.pyc {{ item.key }} {{ item.value }}"
loop: "{{ hostname_version|dict2items }}"
gives
shell> ansible-playbook pb.yml
PLAY [localhost] **********************************************************************************
TASK [debug] **************************************************************************************
ok: [localhost] => (item=abc) =>
msg: python3 image.pyc abc 19.17
ok: [localhost] => (item=xyx) =>
msg: python3 image.pyc xyz 19.17
TASK [debug] **************************************************************************************
ok: [localhost] => (item=abc) =>
msg: python3 image.pyc abc 19.17
ok: [localhost] => (item=xyz) =>
msg: python3 image.pyc xyz 19.17
TASK [debug] **************************************************************************************
ok: [localhost] =>
hostname_version:
abc: '19.17'
xyz: '19.17'
TASK [debug] **************************************************************************************
ok: [localhost] => (item={'key': 'abc', 'value': '19.17'}) =>
msg: python3 image.pyc abc 19.17
ok: [localhost] => (item={'key': 'xyz', 'value': '19.17'}) =>
msg: python3 image.pyc xyz 19.17
PLAY RECAP ****************************************************************************************
localhost: ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Q: "How do I run the tasks in parallel?"
A: Run the tasks in the loop asynchronously. See Asynchronous actions and polling. For example, to test the timing, the playbook below runs the command sleep 3
shell> cat pb.yml
- hosts: localhost
vars:
images:
abc:
baseversion: '18.8'
hostname: abc
netconf: bond
password: welcome1
password2: welcome2
toversion: '19.18'
username: root
version: '19.17'
xyx:
baseversion: '18.8'
hostname: xyz
netconf: bond
password: welcome1
password2: welcome2
toversion: '19.18'
username: root
version: '19.17'
hostname_version: "{{ images.values()|list|
items2dict(key_name='hostname',
value_name='version') }}"
tasks:
- command:
cmd: "sleep 3"
#cmd: "echo python3 image.pyc {{ item.key }} {{ item.value }}"
loop: "{{ hostname_version|dict2items }}"
register: async_out
async: 5
poll: 0
- debug:
var: async_out
when: debug|d(false)|bool
- async_status:
jid: "{{ item.ansible_job_id }}"
loop: "{{ async_out.results }}"
loop_control:
label: "{{ item.item.key }}"
register: async_poll
until: async_poll.finished
retries: 5
- debug:
var: async_poll
when: debug|d(false)|bool
- debug:
msg: |
{% for result in async_poll.results %}
{{ result.item.item.key }} {{ result.start }} {{ result.end }}
{% endfor %}
gives
shell> ansible-playbook pb.yml
PLAY [localhost] **********************************************************************************
TASK [command] ************************************************************************************
changed: [localhost] => (item={'key': 'abc', 'value': '19.17'})
changed: [localhost] => (item={'key': 'xyz', 'value': '19.17'})
TASK [debug] **************************************************************************************
skipping: [localhost]
TASK [async_status] *******************************************************************************
FAILED - RETRYING: [localhost]: async_status (5 retries left).
changed: [localhost] => (item=abc)
changed: [localhost] => (item=xyz)
TASK [debug] **************************************************************************************
skipping: [localhost]
TASK [debug] **************************************************************************************
ok: [localhost] =>
msg: |-
abc 2023-01-24 13:17:48.526694 2023-01-24 13:17:51.530734
xyz 2023-01-24 13:17:48.831822 2023-01-24 13:17:51.836860
PLAY RECAP ****************************************************************************************
localhost: ok=3 changed=2 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0
From the last debug you see that:
The second command started 0.3s after the first one
The first command finished 2.7s after the second one started
This is the "best effort" parallelism you can get in Ansible.
I'm trying to read from a list of CSV files (fileglob) and the files have a standard naming convention. Just wanted to read the CSV files using read_csv and register/set_fact them to the filename, which should be the variable name.
The files are
apples.csv
pears.csv
grapes.csv
What I've tried, (its close but not correct)
- name: "Read into dynamic variables"
read_csv:
path: "{{ item }}"
with_fileglob:
- "/tmp/fruits/*.csv"
fruit_name: "{{ item | basename | regex_replace('.csv') }}"
register: "fruit_{{ fruit_name }}"
So ideally want to get the contents of each CSV to be part of variable, e.g. fruit_apples which can be later re-used in other plays.
For example, given the files
shell> tree fruits/
fruits/
├── apples.csv
├── grapes.csv
└── pears.csv
0 directories, 3 files
shell> cat fruits/apples.csv
red,big,20
green,small,10
shell> cat fruits/grapes.csv
red,big,20
black,small,10
shell> cat fruits/pears.csv
green,big,30
yellow,small,20
Read the files
- read_csv:
fieldnames: color,size,price
path: "{{ item }}"
with_fileglob: "fruits/*.csv"
register: fruit
Instead of creating the variables fruit_*, it's simpler to create a dictionary of the fruits. For example, put the declarations below as appropriate
fruits: "{{ dict(f_keys|zip(f_vals)) }}"
f_vals: "{{ fruit.results|map(attribute='list')|list }}"
f_keys: "{{ fruit.results|map(attribute='item')|
map('basename')|
map('splitext')|
map('first')|list }}"
gives
fruits:
apples:
- {color: red, price: '20', size: big}
- {color: green, price: '10', size: small}
grapes:
- {color: red, price: '20', size: big}
- {color: black, price: '10', size: small}
pears:
- {color: green, price: '30', size: big}
- {color: yellow, price: '20', size: small}
Example of a complete playbook
- hosts: localhost
vars:
fruits: "{{ dict(f_keys|zip(f_vals)) }}"
f_vals: "{{ fruit.results|map(attribute='list')|list }}"
f_keys: "{{ fruit.results|map(attribute='item')|
map('basename')|
map('splitext')|
map('first')|list }}"
tasks:
- read_csv:
fieldnames: color,size,price
path: "{{ item }}"
with_fileglob: "fruits/*.csv"
register: fruit
- debug:
var: fruits
- debug:
var: fruits.apples
This will not be possible. According Registering variables
" When you register a variable in a task with a loop, the registered variable contains a value for each item in the loop. The data structure placed in the variable during the loop will contain a results attribute, that is a list of all responses from the module."
Therefore you need to extract the parts which you are interested in within the next tasks.
To get a better understanding in the behavior I've created a small test
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Create multiple results and register them
debug:
msg: "{{ item }}"
loop: [A, B, C]
register: result
- name: Show result
debug:
msg: "{{ result }}"
which is showing an equivalent behavior since with_fileglob is a with_X style loop.
TASK [Create multiple results and register them] ***
ok: [localhost] => (item=A) =>
msg: A
ok: [localhost] => (item=B) =>
msg: B
ok: [localhost] => (item=C) =>
msg: C
TASK [Show result] *********************************
ok: [localhost] =>
msg:
changed: false
msg: All items completed
results:
- ansible_loop_var: item
changed: false
failed: false
item: A
msg: A
- ansible_loop_var: item
changed: false
failed: false
item: B
msg: B
- ansible_loop_var: item
changed: false
failed: false
item: C
msg: C
Further Q&A
Ansible: How to register multiple variables within a single task?
Registering multiple variables in a loop
I am using this in an Ansible playbook:
- name: Gather info from Vcenter
vmware_vm_info:
hostname: "{{ result_item.vcenter }}"
username: "{{ ansible_username }}"
password: "{{ ansible_password }}"
validate_certs: no
register: vminfo
loop: "{{ result.list }}"
loop_control:
loop_var: result_item
I loop through a csv which has a list of VMs and their Vcenters. The json output from the Ansible task is this:
{
"results": [
{
"changed": false,
"virtual_machines": [
{
"guest_name": "Server1",
"guest_fullname": "SUSE Linux Enterprise 11 (64-bit)",
"power_state": "poweredOn",
},
{
"guest_name": "Server2",
"guest_fullname": "FreeBSD Pre-11 versions (64-bit)",
"power_state": "poweredOn",
},
Now I need to query this output for the VMs in my csv (guest_name matches vmname) and use set_fact to indicate whether the VMs in the csv are poweredOff or poweredOn. Next I can use it as a conditional on whether to power off the VM or not based on its current status.
I can't seem to get the json_query to work when matching to the VM name in the csv to the json output and then getting the corresponding power status. Any ideas?
CSV file:
vmname vcenter
Server1 Vcenter1
Server2 Vcenter1
Q: "set_fact to indicate whether the VMs in the CSV are powered off or powered on."
A: For example
- read_csv:
path: servers.csv
dialect: excel-tab
register: result
- set_fact:
servers: "{{ result.list|map(attribute='vmname')|list }}"
- set_fact:
virtual_machines: "{{ virtual_machines|default([]) +
[dict(_servers|zip(_values))] }}"
loop: "{{ vminfo.results }}"
vars:
_servers: "{{ servers|intersect(_dict.keys()|list) }}"
_values: "{{ _servers|map('extract',_dict)|list }}"
_dict: "{{ item.virtual_machines|
items2dict(key_name='guest_name', value_name='power_state') }}"
- debug:
var: virtual_machines
gives
virtual_machines:
- Server1: poweredOn
Server2: poweredOn
Servers missing in the vminfo.results will be silently ignored.
Q: "Use it as a conditional on whether to power off the VM or not."
A: For example Server1 in the first host
- debug:
msg: "Host={{ _host }} VM={{ _vm }} is poweredOn"
when: virtual_machines[_host][_vm] == 'poweredOn'
vars:
_host: 0
_vm: Server1
gives
msg: Host=0 VM=Server1 is poweredOn
I suppose, from your your example that you do have a TSV, so a tab separated values and not a CSV, which stands for comma separated values.
Based on this, the read_csv module, along with the dialect: excel-tab will help you read your TSV.
Then, you will need to use a filter projection to query the JSON based on the data in your TSV file.
You could also need to flatten the projection to get rid of the doubles list created by both the list in results and in virtual_machines.
An example of the resulting JMESPath query, for the Server1 ends up being:
results[].virtual_machines[?
guest_name == `Server1`
]|[]|[0].power_state
Then with all this in a playbook we do end up with:
- hosts: localhost
gather_facts: no
tasks:
- read_csv:
path: servers.csv
dialect: excel-tab
register: servers
- debug:
msg: >-
For {{ item.vmname }}, the state is {{
vminfo |
json_query(
'results[].virtual_machines[?
guest_name == `' ~ item.vmname ~ '`
]|[]|[0].power_state'
)
}}
loop: "{{ servers.list }}"
loop_control:
label: "{{ item.vmname }}"
vars:
vminfo:
results:
- changed: false
virtual_machines:
- guest_name: Server1
guest_fullname: SUSE Linux Enterprise 11 (64-bit)
power_state: poweredOn
- guest_name: Server2
guest_fullname: FreeBSD Pre-11 versions (64-bit)
power_state: poweredOn
Which yields the recap:
PLAY [localhost] **************************************************************************************************
TASK [read_csv] ***************************************************************************************************
ok: [localhost]
TASK [debug] ******************************************************************************************************
ok: [localhost] => (item=Server1) =>
msg: For Server1, the state is poweredOn
ok: [localhost] => (item=Server2) =>
msg: For Server2, the state is poweredOn
PLAY RECAP ********************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
How do you remove a json Key that contains a colon (:) with jinja2 rejectattr.
Environment:
ansible 2.9.1
config file = None
configured module search path = [u'/home/<user>/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.15+ (default, Oct 7 2019, 17:39:04) [GCC 7.4.0]
json data:
{
"tag:environment": "qa",
"tag:instance_id": "i-123456789"
}
Ansible playbook:
- name: Remove InstanceID
debug:
msg: "{{ instance_filter | rejectattr('['tag:environment'], 'defined' ') | list }}
Actual Results:
fatal: [localhost]: FAILED! => {
"msg": "template error while templating string: expected token ',', got 'tag'. String: {{ instance_filter | rejectattr('['tag:environment'], 'defined' ') | list }}"
}
Expected results:
{
"tag:environment": "qa"
}
The rejectattr is indeed one of the key filters to use to achieve your goal, but a few more things are needed. Here is the correct sequence of filters to remove that particular key from the dictionary variable you have:
Playbook:
---
- hosts: localhost
gather_facts: false
vars:
instance_filter:
tag:environment: qa
tag:instance_id: i-123456789
tasks:
- name: print var
debug:
var: instance_filter
- name: manipulate the var
debug:
msg: "{{ instance_filter | dict2items | rejectattr('key', 'equalto', 'tag:instance_id') | list | items2dict }}"
Output:
PLAY [localhost] *******************************************************************************************************************************************************************************************************
TASK [print var] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
"instance_filter": {
"tag:environment": "qa",
"tag:instance_id": "i-123456789"
}
}
TASK [manipulate the var] **********************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"tag:environment": "qa"
}
}
PLAY RECAP *************************************************************************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
hope it helps.
Q: "How do you remove a JSON key?"
A: It's possible to create custom filter plugins. For example
$ cat filter_plugins/dict_utils.py
def dict_del_key(d, key):
del d[key]
return d
class FilterModule(object):
''' Ansible filters. Interface to Python dictionary methods.'''
def filters(self):
return {
'dict_del_key' : dict_del_key
}
The play below
- hosts: localhost
vars:
dict:
'tag:environment': 'qa'
'tag:instance_id': 'i-123456789'
tasks:
- debug:
msg: "{{ dict|dict_del_key('tag:instance_id') }}"
gives
msg:
tag:environment: qa
Notes:
See If you quote those config keys, they will become strings.
See the difference between 7.3.1. Double-Quoted Style and 7.3.2. Single-Quoted Style.
FWIW. See other filters available at GitHub.