How to get entry of a dictionary from vmware_guest_disk_facts - json

I'm trying to get the datastore name for a specific hard disk but I haven't been successful in being able to figuring out choose an entry in the list.
This output is from the ansible module "vmware_guest_disk_facts"
I save this output to a variable called "vm_info".
"guest_disk_facts": {
"0": {
"backing_filename": "stuffstuff",
"capacity_in_kb": 106954752,
"backing_eagerlyscrub": false,
"backing_datastore": "WHAT I REALLY WANT",
"backing_writethrough": false,
"label": "Hard disk 1",
"backing_type": "FlatVer2",
"key": 2000,
"capacity_in_bytes": 109521666048,
"backing_thinprovisioned": false,
"controller_key": 1000,
"summary": "106,954,752 KB",
"unit_number": 0,
"backing_uuid": "info"
},
"1": {
"backing_filename": "stuffstuff",
"capacity_in_kb": 15728640,
"backing_eagerlyscrub": false,
"backing_datastore": "DON'T CARE OF ABOUT THIS ONE",
"backing_writethrough": false,
"label": "Hard disk 2",
"backing_type": "FlatVer2",
"key": 2001,
"capacity_in_bytes": 16106127360,
"backing_thinprovisioned": false,
"controller_key": 1000,
"summary": "15,728,640 KB",
"unit_number": 1,
"backing_uuid": "info"
}
- debug:
msg: "{{ item.guest_disk_facts | json_query(query) }}"
with_items: "{{ vm_info.results }}"
vars:
query: "guest_disk_facts.0.backing_datastore" #done w/ & w/o quotes around 0
I've also tried the following queries and I feel like I've exhausted all options at this point.
query: "guest_disk_facts.[0].backing_datastore"#done w/ & w/o quotes around 0
query: "guest_disk_facts[0].backing_datastore" #done w/ & w/o quotes around 0
query: "guest_disk_facts.*.backing_datastore" #will give me backing_datastore entries for both dictionaries in this case
I would like to just get backing_datastore for one entry in this list of dictionaries
msg: "WHAT I REALLY WANT"
but so far I'm returned with either this error:
Expecting: ['quoted_identifier', 'unquoted_identifier', 'lbracket', 'lbrace'], got: number: Parse error at column 17, token \"0\" (NUMBER), for expression
OR
msg: ""
OR
msg:[
"0",
]

The task below gives "WHAT YOU REALLY WANT"
- debug:
msg: "{{ guest_disk_facts['0'|quote].backing_datastore }}"
The point is quoting the quoted key. The keys '0' and '1' are not valid variables and must be quoted.
The loop below
- debug:
msg: "{{ guest_disk_facts[item|quote].backing_datastore }}"
loop: "{{ guest_disk_facts.keys() }}"
gives
ok: [localhost] => (item=1) =>
msg: DON'T CARE OF ABOUT THIS ONE
ok: [localhost] => (item=0) =>
msg: WHAT I REALLY WANT

Another way to do this -
- name: Get all disks from existing VM
vmware_guest_disk_info:
validate_certs: False
hostname: '{{ vcenter_hostname }}'
username: '{{ vcenter_username }}'
password: '{{ vcenter_password }}'
datacenter: Asia-Datacenter1
name: VM_8046
register: existing_disk
- name: Get Backing datastore for desired disk id
set_fact:
disk_zero_datastore: "{{ item.value.backing_datastore }}"
with_dict: "{{ existing_disk.guest_disk_info }}"
when: item.key == '0'
- debug:
msg: "{{ disk_zero_datastore }}"
when: disk_zero_datastore is defined

Related

Pass JSON from output for another query Ansible

Please help me figure out what I'm wrong with. I'm getting a JSON from Ansible and filtering it, after which I want to save the output and reuse it. But, unfortunately, I get an error that this attribute does not exist. Where did I go wrong?
playbook code:
var:
query_general: "body.results[].{display_name: display_name, subnets: subnets[]}"
- name: parsing query
set_fact:
myvar: "{{ results | json_query(query_general) }}"
register: output
- name: qwe
set_fact:
scndjson: "{{ output.myvar[].display_name }}"
- name: print
debug:
msg: "{{ scndjson }}"
I tried the json_query second case as well, but that didn't work either.
in register:output i have:
[
{
"display_name": "1test",
"subnets": [
{
"gateway_address": "0.0.0.0/25",
"network": "0.0.0.0/25"
}
]
},
{
"display_name": "test",
"subnets": [
{
"gateway_address": "0.0.0.1/25",
"network": "0.0.0.1/25"
}
]
}
]
error:
The task includes an option with an undefined variable.
it can be: output, display_name, etc
UPD:
I corrected the yaml, there are no errors, but the data is not displayed.
tasks:
- name:
nsxt_rest:
hostname: anyhost
username: anyuser
password: anypass
validate_certs: false
method: get
path: /policy/api/v1/infra/segments
register: nsx_results
- debug:
var: nsx_query_general
vars:
nsx_query_general: "{{ nsx_results | json_query('body.results[].{display_name: display_name, subnets: subnets[]}') }}"
register: output
- debug:
var: secondjson
vars:
secondjson: "{{ output|json_query('[].display_name') }}"
Output from nsx_query_general:
{
"nsx_query_general": [
{
"display_name": "test",
"subnets": [
{
"gateway_address": "0.0.0.0/25",
"network": "0.0.0.0/25"
}
]
},
{
"display_name": "1test",
"subnets": [
{
"gateway_address": "0.0.0.1/25",
"network": "0.0.0.1/25"
}
]
}]}
Output from secondjson:
{
"secondjson": "",
"_ansible_verbose_always": true,
"_ansible_no_log": false,
"changed": false
}
Given the registered variable output
output:
- display_name: 1test
subnets:
- gateway_address: 0.0.0.0/25
network: 0.0.0.0/25
- display_name: test
subnets:
- gateway_address: 0.0.0.1/25
network: 0.0.0.1/25
Either use json_query
scndjson: "{{ output|json_query('[].display_name') }}"
, or map attribute
scndjson: "{{ output|map(attribute='display_name')|list }}"
Both declarations create the list
scndjson: [1test, test]
Example of a complete playbook
- hosts: localhost
vars:
output:
- display_name: 1test
subnets:
- gateway_address: 0.0.0.0/25
network: 0.0.0.0/25
- display_name: test
subnets:
- gateway_address: 0.0.0.1/25
network: 0.0.0.1/25
tasks:
- debug:
var: scndjson
vars:
scndjson: "{{ output|json_query('[].display_name') }}"
- debug:
var: scndjson
vars:
scndjson: "{{ output|map(attribute='display_name')|list }}"

Ansible - issue with loop and json_query's

Question
I need to get the ID from the GET because it's needed in the URL in the PUT task to edit a specific "input" entry. I'm using the Ansible URI to talk to a REST API to manage this.
playbook
*host_vars/host.yml
*
---
inputs:
- title: "test_input_api"
type: "org.graylog2.inputs.syslog.udp.SyslogUDPInput"
global: false
configuration:
allow_override_date: false
bind_address: "0.0.0.0"
expand_structured_data: false
force_rdns: false
number_worker_threads: 8
override_source: null
port: 5999
recv_buffer_size: null
store_full_message: true
- title: "test_input_api_2"
type: "org.graylog2.inputs.syslog.udp.SyslogUDPInput"
global: false
configuration:
allow_override_date: false
bind_address: "0.0.0.0"
expand_structured_data: false
force_rdns: false
number_worker_threads: 8
override_source: null
port: 5998
recv_buffer_size: null
store_full_message: true
playbook.yml
---
- name: Configure system
hosts: graylog
connection: local
gather_facts: no
roles:
- graylog/inputs
roles/graylog/inputs/tasks/main.yml
---
- include_tasks: get_inputs.yml
- include_tasks: put_inputs.yml
roles/graylog/inputs/tasks/get_inputs.yml
---
- name: "API GET System Inputs"
uri:
url: http://{{ ansible_host }}:9000/api/system/inputs
url_username : "{{ system.users.triple_admin.api_token }}"
url_password: token
method: GET
return_content: yes
register: get_graylog_inputs
- name: Set Fact
set_fact:
get_input_id: "{{ get_graylog_inputs.content | from_json | json_query('inputs[?title == `{}`] | [0].id '.format(input.title)) }}"
loop: "{{ inputs }}"
loop_control:
loop_var: input
The registered var from the get show's the following
{
"json": {
"inputs": [
{
"attributes": {
"allow_override_date": "False",
"bind_address": "0.0.0.0",
"expand_structured_data": "False",
"force_rdns": "False",
"number_worker_threads": 8,
"override_source": "",
"port": 5999,
"recv_buffer_size": "",
"store_full_message": "True"
},
"content_pack": null,
"created_at": "2021-07-30T15:21:47.590Z",
"creator_user_id": "triple_admin",
"global": false,
"id": "6104170beca15547502665d6",
"name": "Syslog UDP",
"node": "ba52ad48-0b13-419d-b957-d47d8911b413",
"static_fields": {},
"title": "test_input_api",
"type": "org.graylog2.inputs.syslog.udp.SyslogUDPInput"
},
roles/graylog/inputs/tasks/put_inputs.yml
---
- name: "API PUT System Inputs"
uri:
url: http://{{ ansible_host }}:9000/api/system/inputs/{{ get_input_id }}
url_username : "{{ system.users.triple_admin.api_token }}"
url_password: token
headers:
X-Requested-By: X-Ansible
method: PUT
body_format: json
body: "{{ lookup('template', 'templates/post_template.j2') }}"
status_code: 201
return_content: yes
loop: "{{ inputs }}"
loop_control:
loop_var: input
"ansible_facts": {
"get_input_id": "61015085eca1554750236084",
"get_input_titles": "test_input_api"
},
"ansible_facts": {
"get_input_id": "610282d0eca155475024ac91",
"get_input_titles": "test_input_api_2"
Results of running the play
loop 1 - this needs to be matched to the title and therefor get id "61015085eca1554750236084"
"title": "test_input_api",
"url": "http://192.168.21.82:9000/api/system/inputs/610282d0eca155475024ac91",
loop 2
"title": "test_input_api_2",
"url": "http://192.168.21.82:9000/api/system/inputs/610282d0eca155475024ac91",
All help is welcome !
(not related) You don't need to json_decode the result get_graylog_inputs.content. If the server on the over side sends the correct Content-type: application/json header, you should have a get_graylog_inputs.json entry containing the already decoded json result.
You don't need to loop twice. Remove the set_fact loop (which is not correct anyway) in your first file and use the value from your register directly in the second loop.
You did not show any example of your input data so I have to guess a bit here from your jmespath expression... but you basically don't need json_query at all and can stick to generic core ansible filters.
Here is how I see the solution in the second file once you cleaned-up the first:
---
- name: "API PUT System Inputs"
vars:
get_input_id: "{{ get_graylog_inputs.json.inputs | selectattr('title', '==', input.title) | map(attribute='id') | first }}"
uri:
url: http://{{ ansible_host }}:9000/api/system/inputs/{{ get_input_id }}
url_username : "{{ system.users.triple_admin.api_token }}"
url_password: token
headers:
X-Requested-By: X-Ansible
method: PUT
body_format: json
body: "{{ lookup('template', 'templates/post_template.j2') }}"
status_code: 201
return_content: yes
loop: "{{ inputs }}"
loop_control:
loop_var: input
You will probably have to debug and tune the expression to get the input id as I could not do it myself against an example data structure.

Ansible parsing json

I am using an Ansible module (vmware_guest_info) with this code:
- name: Get status of VM post-change
vmware_guest_info:
datacenter: "xxxx"
hostname: "{{ result_item.vcenter }}"
username: "{{ xxxx }}"
password: "{{ ansible_password }}"
name: "{{ result_item.vmname }}"
schema: "vsphere"
validate_certs: no
properties: ["config.hardware.memoryMB", "config.hardware.numCPU", "config.hardware.numCoresPerSocket"]
register: postvmname_vminfo
loop: "{{ result.list }}"
loop_control:
loop_var: result_item
It produces the following JSON:
{
"results": [
{
"ansible_loop_var": "result_item",
"changed": false,
"failed": false,
"instance": {
"config": {
"hardware": {
"memoryMB": 4096,
"numCPU": 6,
"numCoresPerSocket": 1
}
}
},
The command that produces the json is looping though an excel file with these contents:
vmname vcenter
Server1 Vcenter1
Server2 Vcenter1
I need to create a dictionary between vmname and numCPU. How can I do this?
I have tried this:
- set_fact:
resultCPU: "{{ instance.config.hardware.numCPU }}"
with_items:
- "{{ postvmname_vminfo.results | json_query(query) }}"
vars:
query: "[?invocation.module_args.name=='{{ result_item.vmname }}']"
loop: "{{ result.list }}"
loop_control:
loop_var: result_item
I get item is undefined from the above. I believe the issue is with my query. I think what I really need to do is use zip to create a dictionary between the servers(vmname) in my Excel file and the value of numCPU.
Does anyone have any suggestions?

best way to modify json in ansible

I have a variable (via set_fact) containing a json string:
{
"PolicyVersion": {
"CreateDate": "2017-08-07T02:48:05Z",
"Document": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Resource": [
"arn:aws:iam::123456789123:role/Root_Update_svcacct",
"arn:aws:iam::123456789123:role/Root_Delete_svcacct",
"arn:aws:iam::123456789123:role/Root_Create_svcacct",
"arn:aws:iam::123456789123:role/Root_Full_svcacct",
"arn:aws:iam::987654321987:role/Member1_Create_svcacct",
"arn:aws:iam::987654321987:role/Member1_Update_svcacct",
"arn:aws:iam::987654321987:role/Member1_Delete_svcacct",
"arn:aws:iam::987654321987:role/Member1_Full_svcacct"
]
}
],
"Version": "2012-10-17"
},
"IsDefaultVersion": true,
"VersionId": "v2"
}
}
What is the best way to insert more elements in the "Resource" array?
"arn:aws:iam::001122334455:role/Member1_Create_svcacct",
"arn:aws:iam::001122334455:role/Member1_Update_svcacct",
"arn:aws:iam::001122334455:role/Member1_Delete_svcacct",
"arn:aws:iam::001122334455:role/Member1_Full_svcacct"
I am exploring dumping the variable to a file and inserting the block I want with external shell tools, which does not seem to be elegant.
I don't know about the best way, but one option is to write a simple library module to handle the mechanics of the update for you. You could use the jsonpointer module as a way of locating the data you wish to modify, and then return the modified object to ansible. A starting point might look like:
#!/usr/bin/python
from ansible.module_utils.basic import AnsibleModule
import json
try:
import jsonpointer
except ImportError:
jsonpointer = None
def main():
module = AnsibleModule(
argument_spec=dict(
data=dict(required=True, type='dict'),
pointer=dict(required=True),
action=dict(required=True,
choices=['append', 'extend', 'update']),
update=dict(type='dict'),
extend=dict(type='list'),
append=dict(),
),
supports_check_mode=True,
)
if jsonpointer is None:
module.fail_json(msg='jsonpointer module is not available')
action = module.params['action']
data = module.params['data']
pointer = module.params['pointer']
if isinstance(data, str):
data = json.loads(str)
try:
res = jsonpointer.resolve_pointer(data, pointer)
except jsonpointer.JsonPointerException as err:
module.fail_json(msg=str(err))
if action == 'append':
res.append(module.params['append'])
if action == 'extend':
res.extend(module.params['extend'])
elif action == 'update':
res.update(module.params['update'])
module.exit_json(changed=True,
result=data)
if __name__ == '__main__':
main()
If you drop this into, e.g., library/json_modify.py, you can use it in a playbook like this:
- hosts: localhost
gather_facts: false
vars:
myvar: {
"PolicyVersion": {
"CreateDate": "2017-08-07T02:48:05Z",
"Document": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Resource": [
"arn:aws:iam::123456789123:role/Root_Update_svcacct",
"arn:aws:iam::123456789123:role/Root_Delete_svcacct",
"arn:aws:iam::123456789123:role/Root_Create_svcacct",
"arn:aws:iam::123456789123:role/Root_Full_svcacct",
"arn:aws:iam::987654321987:role/Member1_Create_svcacct",
"arn:aws:iam::987654321987:role/Member1_Update_svcacct",
"arn:aws:iam::987654321987:role/Member1_Delete_svcacct",
"arn:aws:iam::987654321987:role/Member1_Full_svcacct"
]
}
],
"Version": "2012-10-17"
},
"IsDefaultVersion": true,
"VersionId": "v2"
}
}
tasks:
- json_modify:
data: "{{ myvar }}"
pointer: "/PolicyVersion/Document/Statement/0/Resource"
action: extend
extend:
- "arn:aws:iam::001122334455:role/Member1_Create_svcacct"
- "arn:aws:iam::001122334455:role/Member1_Update_svcacct"
- "arn:aws:iam::001122334455:role/Member1_Delete_svcacct"
- "arn:aws:iam::001122334455:role/Member1_Full_svcacct"
register: result
- debug:
var: result.result
The result of running this playbook and the proposed module is:
TASK [debug] *******************************************************************
ok: [localhost] => {
"result.result": {
"PolicyVersion": {
"CreateDate": "2017-08-07T02:48:05Z",
"Document": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Resource": [
"arn:aws:iam::123456789123:role/Root_Update_svcacct",
"arn:aws:iam::123456789123:role/Root_Delete_svcacct",
"arn:aws:iam::123456789123:role/Root_Create_svcacct",
"arn:aws:iam::123456789123:role/Root_Full_svcacct",
"arn:aws:iam::987654321987:role/Member1_Create_svcacct",
"arn:aws:iam::987654321987:role/Member1_Update_svcacct",
"arn:aws:iam::987654321987:role/Member1_Delete_svcacct",
"arn:aws:iam::987654321987:role/Member1_Full_svcacct",
"arn:aws:iam::001122334455:role/Member1_Create_svcacct",
"arn:aws:iam::001122334455:role/Member1_Update_svcacct",
"arn:aws:iam::001122334455:role/Member1_Delete_svcacct",
"arn:aws:iam::001122334455:role/Member1_Full_svcacct"
]
}
],
"Version": "2012-10-17"
},
"IsDefaultVersion": true,
"VersionId": "v2"
}
}
}
Actually Ansible is natively able to read JSON files.
see this question:
add-a-new-key-value-to-a-json-file-using-ansible
This is a little bit old. I know but I felt it could be useful to many people so I'll post here my solution as "the best way to modify a json" that I could personally find.
First of all the JSON. In my use-case I had a bunch of aws snapshots that were encrypted with the wrong kms key and I had to recreate the AMI with the correct key.
So I had to :
get data of the old snapshots ( like size device_name etc.)
create the new snaps with the different key
re-register a new ami with the correct block_device_mapping
Here's the code
- name: get ami
amazon.aws.ec2_ami_info:
image_ids: ami-<id>
region: "{{ region }}"
register: ami
- name: save snapshot ids and device_name and volume_size
set_fact:
snapshot_ids: "{{ ami | json_query('images[].block_device_mappings[].ebs.snapshot_id') }}"
device_name: "{{ ami | json_query('images[].block_device_mappings[].device_name') }}"
volume_size: "{{ ami | json_query('images[].block_device_mappings[].ebs.volume_size') }}"
basically each of the above is a list of each of the 3 things (device_name, snap_id, volume_size) that I need (but it could be extended)
then:
- name: get kms arn
aws_kms_info:
filters:
alias: "{{ kms_keys.alias }}"
region: "{{ region }}"
register: aws_kms_facts_out
- debug:
var: aws_kms_facts_out
- set_fact:
kms_arn: "{{ aws_kms_facts_out['keys'][0].key_arn }}"
- name: copy snap with new encryption key
community.aws.ec2_snapshot_copy:
source_region: "{{ region }}"
region: "{{ region }}"
source_snapshot_id: "{{ item }}"
encrypted: yes
kms_key_id: "{{ kms_arn }}"
loop: "{{ snapshot_ids }}"
register: new_snapshots
and then here's the catch
- set_fact:
new_snapshot_ids: "{{ new_snapshots| json_query('snapshots[].snapshot_id') }}"
- name: creating the block_device_mappings structure (still a list of objects)
ansible.builtin.debug:
msg: '{
"device_name": "{{ item.2 }}",
"delete_on_termination": "true",
"snapshot_id": "{{ item.0 }}",
"volume_size": {{ item.1 }},
"volume_type": "gp2"
}'
loop: "{{ new_snapshot_ids|zip_longest(volume_size,device_name)|list }}"
register: block_device_mappings
- set_fact:
bdm: "{{ block_device_mappings | json_query('results[].msg') }}"
finally
- name: create new ami from newly created snapshots
amazon.aws.ec2_ami:
region: "{{ region }}"
name: "{{ instance.name }}-{{ ansible_date_time.date }}"
state: present
architecture: x86_64
virtualization_type: hvm
root_device_name: /dev/sda1
device_mapping: "{{ bdm }}"
This is how you can do it without requiring any additional trick.
Of course this is declined to my particular use case but you can adapt it to any circumstance, that do not require a complete disassemble and reassemble of the Json itself

Check if variable is in a list within a dictionary in Ansible

I have the following tasks:
- name: Retrieve records from Route53
route53:
command: get
zone: "{{ stag_zone }}"
record: "{{ stag_record }}"
type: A
aws_access_key: "{{ aws_access_key }}"
aws_secret_key: "{{ aws_secret_key }}"
register: rec
- name: Print records
debug: var=rec
- name: Record contains IP
debug: msg="{{rec}} contains {{stag_ip}}"
when: "'{{stag_ip}}' in {{rec.set.values}}"
The Print records task outputs something like this:
ok: [XXX.XXX.XXX.XXX] => {
"var": {
"rec": {
"changed": false,
"invocation": {
"module_args": "",
"module_name": "route53"
},
"set": {
"alias": false,
"record": "YYY.XXX.ZZZ.",
"ttl": "300",
"type": "A",
"value": "AAA.AAA.AAA.AAA,BBB.BBB.BBB.BBB",
"values": [
"AAA.AAA.AAA.AAA",
"BBB.BBB.BBB.BBB"
],
"zone": "XXX.ZZZ."
}
}
}
}
And I want to execute "Record contains IP" task only when {{stag_ip}} is in {{rec.set.values}}. But my when clause is definitely broken. It outputs this:
fatal: [XXX.XXX.XXX.XXX] => Failed to template {% if 'QQQ.QQQ.QQQ.QQQ' in <built-in method values of dict object at 0x7fb66f54e6e0> %} True {% else %} False {% endif %}: template error while templating string: unexpected '<'
How can I "cast" rec.set.values to a list?
The problem here is because values is a dict method. So it has "precedence" over accessing keys. To fix this, one has to explicitly call get:
when: stag_ip in rec.set.get('values')