Ansible combine list of items in single list - json

There is probably simple solution to this i just cant figure it out.
I got the following list in Ansible:
result: [
{
"name": "DC1",
"region": "EU",
"net": "10.10.1.0"
},
{
"name": "DC1",
"region": "EU",
"net": "10.10.2.0"
},
{
"name": "DC2",
"region": "NA",
"net": "10.10.3.0"
},
]
and would like to get the new list in this format:
[
{
"name": "DC1",
"region": "EU",
"net": ["10.10.1.0","10.10.2.0"]
},
{
"name": "DC2",
"region": "NA",
"net": "10.10.3.0"
}
]
In the meantime i have also found the solution for the data format expected by Ansible module
- name: Convert items from the input list
set_fact:
list_conv: "{{ list_conv|default([])+ (item | dict2items )}}"
loop: "{{ result }}"
when: item.name == 'DC1'
- name: Create new list
set_fact:
list_final: "{{ list_final|default([]) + [{ item.key: item.value }] }}"
loop: "{{ list_conv | unique }}"
Output then will:
[
{"name": "DC1"},
{"region": "EU"},
{"net": "10.10.1.0"},
{"net": "10.10.2.0"}
]

There are more options.
The simplest option is converting the addresses to the lists
net_lists: "{{ result|json_query('[].{name: name, net: [net]}') }}"
gives
net_lists:
- name: DC1
net: [10.10.1.0]
- name: DC1
net: [10.10.2.0]
- name: DC2
net: [10.10.3.0]
Then, merge the items by name and append the lists
net_list: "{{ [net_lists, []]|
community.general.lists_mergeby('name', list_merge='append') }}"
gives what you want
net_list:
- name: DC1
net: [10.10.1.0, 10.10.2.0]
- name: DC2
net: [10.10.3.0]
Example of a complete playbook for testing
- hosts: localhost
vars:
result:
- name: DC1
net: 10.10.1.0
- name: DC1
net: 10.10.2.0
- name: DC2
net: 10.10.3.0
net_lists: "{{ result|json_query('[].{name: name, net: [net]}') }}"
net_list: "{{ [net_lists, []]|
community.general.lists_mergeby('name', list_merge='append') }}"
tasks:
- debug:
var: net_lists|to_yaml
- debug:
var: net_list|to_yaml
The next option is grouping the items by name, selecting the keys and values, and creating a dictionary. Declare the variables
net_groups: "{{ result|groupby('name') }}"
net_names: "{{ net_groups|map('first')|list }}"
net_lists: "{{ net_groups|map('last')|
map('map', attribute='net')|list }}"
net_dict: "{{ dict(net_names|zip(net_lists)) }}"
net_list: "{{ net_dict|dict2items(key_name='name', value_name='net') }}"
give
net_groups:
- - DC1
- - {name: DC1, net: 10.10.1.0}
- {name: DC1, net: 10.10.2.0}
- - DC2
- - {name: DC2, net: 10.10.3.0}
net_names: [DC1, DC2]
net_lists:
- [10.10.1.0, 10.10.2.0]
- [10.10.3.0]
net_dict:
DC1: [10.10.1.0, 10.10.2.0]
DC2: [10.10.3.0]
net_list:
- name: DC1
net: [10.10.1.0, 10.10.2.0]
- name: DC2
net: [10.10.3.0]
Example of a complete playbook for testing
- hosts: localhost
vars:
result:
- name: DC1
net: 10.10.1.0
- name: DC1
net: 10.10.2.0
- name: DC2
net: 10.10.3.0
net_groups: "{{ result|groupby('name') }}"
net_names: "{{ net_groups|map('first')|list }}"
net_lists: "{{ net_groups|map('last')|
map('map', attribute='net')|list }}"
net_dict: "{{ dict(net_names|zip(net_lists)) }}"
net_list: "{{ net_dict|dict2items(key_name='name', value_name='net') }}"
tasks:
- debug:
var: net_groups|to_yaml
- debug:
var: net_names|to_yaml
- debug:
var: net_lists|to_yaml
- debug:
var: net_dict|to_yaml
- debug:
var: net_list|to_yaml
Q: "How possibly can I generate such a list from my input data?"
- name: DC1
- net: 10.10.1.0
- net: 10.10.2.0
A: There are no filters to produce this structure. You can use Jinja instead. For example,
net_dict2_str: |
{% for k,v in net_dict.items() %}
{{ k }}:
- name: {{ k }}
{% for n in v %}
- net: {{ n }}
{% endfor %}
{% endfor %}
net_dict2: "{{ net_dict2_str|from_yaml }}"
give
net_dict2:
DC1:
- name: DC1
- net: 10.10.1.0
- net: 10.10.2.0
DC2:
- name: DC2
- net: 10.10.3.0
Fit the template to your needs.

Related

Pass JSON from output for another query Ansible

Please help me figure out what I'm wrong with. I'm getting a JSON from Ansible and filtering it, after which I want to save the output and reuse it. But, unfortunately, I get an error that this attribute does not exist. Where did I go wrong?
playbook code:
var:
query_general: "body.results[].{display_name: display_name, subnets: subnets[]}"
- name: parsing query
set_fact:
myvar: "{{ results | json_query(query_general) }}"
register: output
- name: qwe
set_fact:
scndjson: "{{ output.myvar[].display_name }}"
- name: print
debug:
msg: "{{ scndjson }}"
I tried the json_query second case as well, but that didn't work either.
in register:output i have:
[
{
"display_name": "1test",
"subnets": [
{
"gateway_address": "0.0.0.0/25",
"network": "0.0.0.0/25"
}
]
},
{
"display_name": "test",
"subnets": [
{
"gateway_address": "0.0.0.1/25",
"network": "0.0.0.1/25"
}
]
}
]
error:
The task includes an option with an undefined variable.
it can be: output, display_name, etc
UPD:
I corrected the yaml, there are no errors, but the data is not displayed.
tasks:
- name:
nsxt_rest:
hostname: anyhost
username: anyuser
password: anypass
validate_certs: false
method: get
path: /policy/api/v1/infra/segments
register: nsx_results
- debug:
var: nsx_query_general
vars:
nsx_query_general: "{{ nsx_results | json_query('body.results[].{display_name: display_name, subnets: subnets[]}') }}"
register: output
- debug:
var: secondjson
vars:
secondjson: "{{ output|json_query('[].display_name') }}"
Output from nsx_query_general:
{
"nsx_query_general": [
{
"display_name": "test",
"subnets": [
{
"gateway_address": "0.0.0.0/25",
"network": "0.0.0.0/25"
}
]
},
{
"display_name": "1test",
"subnets": [
{
"gateway_address": "0.0.0.1/25",
"network": "0.0.0.1/25"
}
]
}]}
Output from secondjson:
{
"secondjson": "",
"_ansible_verbose_always": true,
"_ansible_no_log": false,
"changed": false
}
Given the registered variable output
output:
- display_name: 1test
subnets:
- gateway_address: 0.0.0.0/25
network: 0.0.0.0/25
- display_name: test
subnets:
- gateway_address: 0.0.0.1/25
network: 0.0.0.1/25
Either use json_query
scndjson: "{{ output|json_query('[].display_name') }}"
, or map attribute
scndjson: "{{ output|map(attribute='display_name')|list }}"
Both declarations create the list
scndjson: [1test, test]
Example of a complete playbook
- hosts: localhost
vars:
output:
- display_name: 1test
subnets:
- gateway_address: 0.0.0.0/25
network: 0.0.0.0/25
- display_name: test
subnets:
- gateway_address: 0.0.0.1/25
network: 0.0.0.1/25
tasks:
- debug:
var: scndjson
vars:
scndjson: "{{ output|json_query('[].display_name') }}"
- debug:
var: scndjson
vars:
scndjson: "{{ output|map(attribute='display_name')|list }}"

Cannot create user because the group does not exits on ansible

I am running this playbook to install mysql.
File var.yml:
PORT : 3306
USER : mysql
USERID : "{{ PORT }}"
GROUP : mysql
GROUPID : "{{ PORT }}"
SECGROUP : wls
DBUSER : root
DBPASSWD : "{{ DBPASS }}"
GROUPS:
- name: "{{ GROUP }}"
gid: "{{ GROUPID }}"
USERS:
- name : "{{ USER }}"
uid: "{{ USERID }}"
group: "{{ GROUP }}"
groups: "{{ SECGROUP }}"
comment: "{{ USER }}"
playbook.yml:
tasks:
- name: 1. create group
group:
name: "{{ item.name }}"
gid: "{{ item.gid }}"
state: present
with_items:
- "{{ GROUPS }}"
- name: 2. create user account
user:
name: "{{ item.name }}"
group: "{{ item.group }}"
groups: "{{ item.groups }}"
uid: "{{ item.uid }}"
comment: "{{ item.comment }}"
with_items:
- "{{ USERS }}"
When I run it, I have an issue with the second task:
failed : (item={u'comment': u'mysql', u'group': u'mysql', u'name': u'mysql', u'groups': u'wls', u'uid': u'3306'}) => {"ansible_loop_var": "item", "changed": false, "item": {"comment}: "mysql", "group": "mysql", "groups" : "wls", "name": "mysql", "uid": "3306"}, "msg": "Group wls does not exist"}
How can I fix it?
It's because group wls does not exist as error says: "msg": "Group wls does not exist"
You cannot create an user with a non-existent group in its groups. You either need to remove group wls from USERS[0].groups or create group wls. Your task 1. only creates group mysql.
To create both groups in task 1. You can just add group wls to the list:
GROUPS:
- name: "{{ GROUP }}"
gid: "{{ GROUPID }}"
- name: "{{ SECGROUP }}"
gid: "<provide gid here>"
(Note: normally in Ansible you don't have to provide gid to group action and it would chose one, but the way you have written your task 1 requires it).
This will make your task 1 create both mysql and wls groups and also pass task 2.

Parse json value when value contains a string in ansible

This ansible playbook works
---
- hosts: localhost
gather_facts: False
vars:
jq: "[?contains(name, 'Pizza')]"
json: |
[{
"name": "Ted's Sub Shop - 720895714701",
"templateid": "24632"
},
{
"name": "Ted's Pizza - 720895714702",
"templateid": "24663"
}]
tasks:
- name: DEBUG
debug:
msg: "{{ json | from_json | json_query(jq) }}"
It returns the following
ok: [localhost] => {
"msg": [
{
"name": "Ted's Pizza - 720895714702",
"templateid": "24663"
}
]
}
I need to take it a few steps further and when the value of name contains Pizza I need it to return just the 12 digit number on the end. So the return output would look like this
ok: [localhost] => {
"msg": "720895714702"
}
Thoughts?
you could try
- name: testplaybook jinja2
hosts: localhost
gather_facts: no
vars:
json: |
[{
"name": "Ted's Sub Shop - 720895714701",
"templateid": "24632"
},
{
"name": "Ted's Pizza - 720895714702",
"templateid": "24663"
}]
tasks:
- name: DEBUG
debug:
msg: "{{ json | from_json | selectattr('name', 'contains' , 'Pizza')
| map(attribute='name')
| map('regex_replace', '^.*?(\\d*)$', '\\1')}}"
result:
ok: [localhost] => {
"msg": [
"720895714702"
]
}
It might be more efficient to add an attribute. For example the declaration below
json_id: "{{ json|zip(id_hash)|map('combine')|list }}"
id_hash: "{{ id_list|
map('community.general.dict_kv', 'id')|list }}"
id_list: "{{ json|
map(attribute='name')|
map('split', '-')|
map('last')|
map('int')|list }}"
expands to
json_id:
- id: 720895714701
name: Ted's Sub Shop - 720895714701
templateid: '24632'
- id: 720895714702
name: Ted's Pizza - 720895714702
templateid: '24663'
Then, the usage is trivial. For example
- debug:
msg: "{{ json_id|
selectattr('name', 'contains' , 'Pizza')|
map(attribute='id')|list }}"
gives
msg:
- 720895714702
The same result gives also json_query below
- debug:
msg: "{{ json_id|
json_query('[?contains(name, `Pizza`)].id') }}"

Loop over multiple JSON arrays to filter the relevant JSON object based on a key value in Ansible

I am trying to loop over 2 different JSON array and filter only if the 2 key values are same.
I am comparing cidr from po-orig.json/po file and subnet from pocooling json. If the key values are same in both file, then print the pertaining details of pocooling json file as a list.
- name: Combine GP MGMT
vars:
pocidr: >-
{{
po
| json_query('ansible_facts.policyobject[].json[]')
}}
poname: >-
{{
pocidr
| selectattr('cidr', '==', item.subnet)
| map(attribute='name')
| list
}}
set_fact:
result: >-
{{
result | default([])
+
[poname]
}}
with_items: "{{ pocooling }}"
Updated code
- name: Compare the Current & Actual GP policy
set_fact:
pocidr: "{{ po | json_query(\"ansible_facts.policyobject[].json[]\") }}"
- name: Combine GP MGMT
vars:
poname:
"{{
pocidr
| selectattr('cidr', '==', item.subnet)
| map(attribute='name')
| first
}}"
set_fact:
result:
"{{
result | default([])
+
[poname]
}}"
with_items: "{{ pocooling }}"
pocooling.json
[
{
"applianceIp": "10.10.10.10",
"dhcpBootOptionsEnabled": false,
"dhcpHandling": "Run a DHCP server",
"dhcpLeaseTime": "1 day",
"dhcpOptions": [],
"dnsNameservers": "upstream_dns",
"fixedIpAssignments": {},
"id": 2222,
"name": "Cooling",
"networkId": "78984654989",
"reservedIpRanges": [],
"subnet": "10.10.10.0/28"
},
{
"applianceIp": "10.10.10.10",
"dhcpBootOptionsEnabled": false,
"dhcpHandling": "Run a DHCP server",
"dhcpLeaseTime": "1 day",
"dhcpOptions": [],
"dnsNameservers": "upstream_dns",
"fixedIpAssignments": {},
"id": 2222,
"name": "Cooling",
"networkId": "123456789",
"reservedIpRanges": [],
"subnet": "10.11.11.0/28"
}
]
po-orig.json(po)
{
"ansible_facts":{
"policyobject":[
{
"invocation":{
"module_args":{
"status_code":[
200,
201,
202
],
"validate_certs":false
}
},
"item":{
"id":"123456",
"name":"DC"
},
"json":[
{
"category":"network",
"cidr":"10.1.1.1/28",
"createdAt":"2021-11-23T19:48:21Z",
"groupIds":[
],
"id":"545649843651365",
"name":"JM-Privat",
"networkIds":[
"545649843651365"
],
"type":"cidr",
"updatedAt":"2021-11-23T19:48:21Z"
},
{
"category":"network",
"cidr":"10.10.10.0/28",
"createdAt":"2021-12-07T13:54:05Z",
"groupIds":[
"897987654689854"
],
"id":"564678984565465",
"name":"PO-L-DE-MCC-COOLING_DMZ_LAB",
"networkIds":[
],
"type":"cidr",
"updatedAt":"2021-12-07T13:54:05Z"
}
]
}
]
}
}
Error:
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: {{ pocidr\n | selectattr('cidr', '==', item.subnet)\n | map(attribute='name')\n | list\n}}: 'dict object' has no attribute 'cidr
Expected result
[
{
"applianceIp": "10.10.10.10",
"dhcpBootOptionsEnabled": false,
"dhcpHandling": "Run a DHCP server",
"dhcpLeaseTime": "1 day",
"dhcpOptions": [],
"dnsNameservers": "upstream_dns",
"fixedIpAssignments": {},
"id": 2222,
"name": "Cooling",
"networkId": "78984654989",
"reservedIpRanges": [],
"subnet": "10.10.10.0/28"
}
]
Complete Playbook
- name: Test
hosts: localhost
connection: local
gather_facts: false
vars:
pocooling: "{{ lookup('file', 'pocooling.json') | from_json }}"
tasks:
- include_vars:
file: po-orig.json
name: po
- name: Combine GP MGMT
vars:
poname: "{{ po|json_query('ansible_facts.policyobject[].json[]')|
selectattr('cidr', '==', item.subnet)|
map(attribute='name')|
list }}"
set_fact:
result: "{{ result|default([]) + [poname] }}"
loop: "{{ pocooling }}"
Given the list
pocooling:
- networkId: '78984654989'
subnet: 10.10.10.0/28
- networkId: '123456789'
subnet: 10.11.11.0/28
Select list of cidr
_cidr: "{{ ansible_facts.policyobject|json_query('[].json[].cidr') }}"
gives
_cidr:
- 10.1.1.1/28
- 10.10.10.0/28
Then select the items that fit the condition
- debug:
msg: "{{ pocooling|selectattr('subnet', 'in' , _cidr) }}"
vars:
_cidr: "{{ ansible_facts.policyobject|json_query('[].json[].cidr') }}"
gives
msg:
- networkId: '78984654989'
subnet: 10.10.10.0/28
Your code, fixed and simplified
- include_vars:
file: po-orig.json
name: po
- name: Combine GP MGMT
vars:
poname: "{{ po|json_query('ansible_facts.policyobject[].json[]')|
selectattr('cidr', '==', item.subnet)|
map(attribute='name')|
list }}"
set_fact:
result: "{{ result|default([]) + [poname] }}"
loop: "{{ pocooling }}"
gives
result:
- - PO-L-DE-MCC-COOLING_DMZ_LAB
- []

Ansible parsing json

I am using an Ansible module (vmware_guest_info) with this code:
- name: Get status of VM post-change
vmware_guest_info:
datacenter: "xxxx"
hostname: "{{ result_item.vcenter }}"
username: "{{ xxxx }}"
password: "{{ ansible_password }}"
name: "{{ result_item.vmname }}"
schema: "vsphere"
validate_certs: no
properties: ["config.hardware.memoryMB", "config.hardware.numCPU", "config.hardware.numCoresPerSocket"]
register: postvmname_vminfo
loop: "{{ result.list }}"
loop_control:
loop_var: result_item
It produces the following JSON:
{
"results": [
{
"ansible_loop_var": "result_item",
"changed": false,
"failed": false,
"instance": {
"config": {
"hardware": {
"memoryMB": 4096,
"numCPU": 6,
"numCoresPerSocket": 1
}
}
},
The command that produces the json is looping though an excel file with these contents:
vmname vcenter
Server1 Vcenter1
Server2 Vcenter1
I need to create a dictionary between vmname and numCPU. How can I do this?
I have tried this:
- set_fact:
resultCPU: "{{ instance.config.hardware.numCPU }}"
with_items:
- "{{ postvmname_vminfo.results | json_query(query) }}"
vars:
query: "[?invocation.module_args.name=='{{ result_item.vmname }}']"
loop: "{{ result.list }}"
loop_control:
loop_var: result_item
I get item is undefined from the above. I believe the issue is with my query. I think what I really need to do is use zip to create a dictionary between the servers(vmname) in my Excel file and the value of numCPU.
Does anyone have any suggestions?