How to append JSON array with specific data - json

I am trying to run :
- name: Describe config aggregator
shell: >
aws configservice describe-configuration-aggregators --configuration-aggregator-name test-config
register: config_ouput
below is the data generated.
{
"ConfigurationAggregators": [
{
"ConfigurationAggregatorName": "test-config",
"ConfigurationAggregatorArn": "arn:aws:config:us-east-1:4567:config-aggregator/config-aggregator-uw2o9pzf",
"AccountAggregationSources": [
{
"AccountIds": [
"895677"
],
"AllAwsRegions": true
}
],
"CreationTime": 1624454176.124,
"LastUpdatedTime": 1626426755.504
}
]
}
Now I want to append the accountIds above with any new account say 1234567 which should give me result such as
{
"ConfigurationAggregators": [
{
"ConfigurationAggregatorName": "test-config",
"ConfigurationAggregatorArn": "arn:aws:config:us-east-1:8778:config-aggregator/test-config-pzf",
"AccountAggregationSources": [
{
"AccountIds": [
"895677,1234567"
],
"AllAwsRegions": true
}
],
"CreationTime": 1624454176.124,
"LastUpdatedTime": 1626426755.504
}
]
}
I am trying to do is:
- name: Export results to JSON
set_fact:
config_ouput_json: "{{ config_ouput + [{"AccountIds": "1234567","AllAwsRegions": true}]}}"
but this doesn't work, please let me know the right syntax.

Basically you require bit of JSON manipulation to achieve your task.
Steps :
Store output of first command in some json file. In your case you can keep that as registered variable of ansible.
Get existing account_ids in some variable.
Create a list of new accounts as variables in ansible.
Iterate over new account_ids and add to existing account_ids.
Update the aws config command.
Sample Code :
- name: initial validation
hosts: localhost
connection: local
vars:
newAccountIds:
- "123456"
- "566544"
- "555445"
tasks:
- name: register json file
include_vars:
file: 'abc.json'
name: bundle
- name: set value
set_fact:
values: "{{ bundle['ConfigurationAggregators'][0]['AccountAggregationSources'][0]['AccountIds'] }}"
- set_fact:
values: "{{ (values | default([])) + [item] }}"
with_items: "{{ newAccountIds }}"
- debug:
msg: "{{ values }}"
- debug:
msg: '"aws configservice put-configuration-aggregator --configuration-aggregator-name test-config --account-aggregation-sources "[{"AccountIds": {{ values | to_json }},"AwsRegions": ["us-east-1"]}]\""'
Sample Output :
PLAY [initial validation] ********************************************************************************************
TASK [Gathering Facts] ***********************************************************************************************
ok: [localhost]
TASK [register json file] ********************************************************************************************
ok: [localhost]
TASK [set value] *****************************************************************************************************
ok: [localhost]
TASK [set_fact] ******************************************************************************************************
ok: [localhost] => (item=123456)
ok: [localhost] => (item=566544)
ok: [localhost] => (item=555445)
TASK [debug] *********************************************************************************************************
ok: [localhost] => {
"msg": [
"895677",
"123456",
"566544",
"555445"
]
}
TASK [debug] *********************************************************************************************************
ok: [localhost] => {
"msg": "\"aws configservice put-configuration-aggregator --configuration-aggregator-name test-config --account-aggregation-sources \"[{\"AccountIds\": [\"895677\", \"123456\", \"566544\", \"555445\"],\"AwsRegions\": [\"us-east-1\"]}]\\\"\""}
PLAY RECAP ***********************************************************************************************************
localhost : ok=6 changed=0 unreachable=0 failed=0

Related

JSON query in Ansible Playbook failing to select desired data

I am new to writing Ansible playbooks and have hit a roadblock. I am trying to use the Site24x7 API to schedule maintenance and need to get a specific ID from a list of Monitor Groups. I created the following:
- name: Download Monitor Groups from Site24x7
uri:
url: https://www.site24x7.com/api/monitor_groups
method: GET
return_content: yes
headers:
Accept: "application/json; version=2.1"
Authorization: "Zoho-oauthtoken {{authtoken.json.access_token}}"
body:
return_content: yes
register: monitor_groups
- name: Get Monitor Group ID
set_fact:
monitorGroupID_query: "[?display_name=='{{hostname.stdout}}'].group_id"
- name: Assign Monitor Group ID
set_fact:
monitorGroupID: "{{ monitor_groups.json | json_query(monitor_group_id_query) }}"
- debug:
var: monitorGroupID
My API call returns data that looks like the following
"monitor_groups.json": {
"code": 0,
"data": [
{
"description": "System Generated",
"display_name": "server1",
"group_id": "319283000000505864",
"group_type": 1,
"health_threshold_count": 1,
"monitors": [
"319283000000483017"
]
},
{
"display_name": "server2",
"group_id": "319283000004701003",
"group_type": 3,
"health_threshold_count": 1,
"monitors": [
"319283000003989345",
"319283000004061005"
]
}
],
"message": "success"
}
My query constantly returns an empty string.
TASK [Assign Monitor Group ID] *****************************************************************************************
ok: [server1.fdu.edu] => {"ansible_facts": {"monitorGroupID": ""}, "changed": false}
TASK [debug] *****************************************************************************************
ok: [server1.fdu.edu] => {
"monitorGroupID": ""
}
Thank you in advance for your help
Single quotes don't work in JMESPath the way it does in almost any other "query language" -- you'll want ` characters wrapped around the JSON literal values. Also, the data:[] in your response is not implied, so if you meant to use it you'll need to either .json.data | json_query or alter your JMESPath to add the data[?display... part
- name: Assign Monitor Group ID
set_fact:
monitorGroupID: "{{ monitor_groups.json | json_query( monitorGroupID_query ) }}"
vars:
monitorGroupID_query: 'data[?display_name==`"{{hostname.stdout}}"`].group_id'
given hostname.stdout=server2 yields
ok: [localhost] => {"ansible_facts": {"monitorGroupID": ["319283000004701003"]}, "changed": false}
For example, given the data below
monitor_groups:
json:
code: 0
data:
- description: System Generated
display_name: server1
group_id: '319283000000505864'
group_type: 1
health_threshold_count: 1
monitors:
- '319283000000483017'
- display_name: server2
group_id: '319283000004701003'
group_type: 3
health_threshold_count: 1
monitors:
- '319283000003989345'
- '319283000004061005'
message: success
Create a dictionary of the hostnames and their group_id. Use it to evaluate the variable monitorGroupID
name_id: "{{ monitor_groups.json.data|
items2dict(key_name='display_name', value_name='group_id') }}"
monitorGroupID: "{{ name_id[hostname.stdout] }}"
gives
name_id:
server1: '319283000000505864'
server2: '319283000004701003'
Then, the task below
- debug:
var: monitorGroupID
gives
TASK [debug] **********************************************************
ok: [server1] =>
monitorGroupID: '319283000000505864'
ok: [server2] =>
monitorGroupID: '319283000004701003'

json parsing using ansible

Hi, Could someone please help how to sort the below using ansible
"msg": [
{
"Desc": "jkl - txt to search-\n",
"SCTASK": "SCTASK000001"
},
{
"Desc": "xyz - txt to search-\n",
"SCTASK": "SCTASK000002"
},
{
"Desc": "def - txt to search-\n",
"SCTASK": "SCTASK000003"
},
{
"Desc": "def - txt to search-\n",
"SCTASK": "SCTASK000004"
},
{
"Desc": "abc- txt to search-\n",
"SCTASK": "SCTASK000005"
}
]
I need to get both sctask and description where 'abc' word present in 'Desc'. I tried to use where condition in my code block but it's not filtering.
- set_fact:
SCT: "{{ jsoncontent.json | json_query(query)}}"
vars:
query: "result[*].{Desc: description, SCTASK: number}"
- name: debug
debug: msg="{{ item }}"
loop: "{{SCT}}"
when: SCT is search("abc")
Thanks!
Q: "Get both sctask and description where 'abc' word present in 'Desc'."
A: json_query is not needed. Simply use the filter selectattr
- set_fact:
SCT: "{{ jsoncontent.json.result|selectattr('Desc', 'search', 'abc') }}"
gives
SCT:
- Desc: |-
abc- txt to search-
SCTASK: SCTASK000005
Notes
If you want to select the items in the loop simply iterate the list
- debug:
msg: "{{ item }}"
loop: "{{ jsoncontent.json.result }}"
when: item.Desc is search("abc")
gives
TASK [debug] *******************************************************************************
skipping: [localhost] => (item={'Desc': 'jkl - txt to search-\n', 'SCTASK': 'SCTASK000001'})
skipping: [localhost] => (item={'Desc': 'xyz - txt to search-\n', 'SCTASK': 'SCTASK000002'})
skipping: [localhost] => (item={'Desc': 'def - txt to search-\n', 'SCTASK': 'SCTASK000003'})
skipping: [localhost] => (item={'Desc': 'def - txt to search-\n', 'SCTASK': 'SCTASK000004'})
ok: [localhost] => (item={'Desc': 'abc- txt to search-\n', 'SCTASK': 'SCTASK000005'}) =>
msg:
Desc: |-
abc- txt to search-
SCTASK: SCTASK000005
You can also create the list SCT in a loop. The task below gives the same result.
- set_fact:
SCT: "{{ SCT|d([]) + [item] }}"
loop: "{{ jsoncontent.json.result }}"
when: item.Desc is search("abc")

Ansible Assert on json_query

I am trying to run an assert in ansible on a result returned from a json_query. However, even though the returned value looks like my assert, the assert fails. Here's the assert:
- name: Assert environment variables
assert:
that:
- Job | json_query(jmesquery) == "00"
vars:
jmesquery: "resources[0].spec.template.spec.containers[0].env[?name=='MY_VARIABLE'].value"
A debug run on the same query returns:
- name: Test1
debug: msg="{{ Job | json_query(jmesquery) }}"
vars:
jmesquery: "resources[0].spec.template.spec.containers[0].env[?name=='MY_VARIABLE'].value"
TASK [Test1] *******************************************************************
ok: [localhost] => {
"msg": [
"00"
]
}
So why on earth is that assert failing?
Your debug message show it: your result is a list with one element, "00", so your assert should be:
- name: Assert environment variables
assert:
that:
- Job | json_query(jmesquery) == ["00"]
vars:
jmesquery: "resources[0].spec.template.spec.containers[0].env[?name=='MY_VARIABLE'].value"
For the record, here is the debug of a list like yours:
- hosts: localhost
gather_facts: no
tasks:
- debug:
msg: ["00"]
Gives:
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": [
"00"
]
}
And this is the debug of a simple string, like the assert, like you wrote it, is expecting:
- hosts: localhost
gather_facts: no
tasks:
- debug:
msg: "00"
Gives
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "00"
}

Parsing JSON from Ansible

I'm looking on the web but I'm unable to find a solution, the scenario in quite complex.
I have one json array with values like these:
[
{
"IP1": "1.2.3.4"
"IP1_VLAN": "900 - CLOUD-DEV"
"IP1_Role": "Management"
"IP2": "4.5.6.7"
"IP2_VLAN": "901 - CLOUD-DEV"
"IP2_Role": "Production"
"IP2": "8.9.10.11"
"IP2_VLAN": "902 - CLOUD-DEV"
"IP2_Role": "Backup"
}
]
My goal is: "Select the IP with the role of Management and tell me the address", it should select IP*_Role == Management and tell me in this case 1.2.3.4
Do you have any idea how to deal with this?
Following my comment, my 2 cent answer: searching through your actual data structure is going to be hard and ugly. If you have the option to change it a bit, things can get much easier.
Here is an example using a list declared in yaml in the following demo playbook. Data is extracted using json_query with the relevent jmespath expression (i.e. the query variable in the debug task)
---
- hosts: localhost
gather_facts: False
vars:
my_ips:
- ip: "1.2.3.4"
vlan: "900 - CLOUD-DEV"
role: "Management"
- ip: "4.5.6.7"
vlan: "901 - CLOUD-DEV"
role: "Production"
- ip: "8.9.10.11"
vlan: "902 - CLOUD-DEV"
role: "Backup"
tasks:
- name: Get ips per role
vars:
query: >-
[?role=='{{ item }}']
debug:
msg: "List of ips for role {{ item }}: {{ my_ips | json_query(query) }}"
loop:
- Management
- Production
- Backup
Which gives
PLAY [localhost] ********************************************************************************************************************************************************************************************************************************************************
TASK [Get ips per role] *************************************************************************************************************************************************************************************************************************************************
Wednesday 23 October 2019 16:14:23 +0200 (0:00:00.043) 0:00:00.043 *****
ok: [localhost] => (item=Management) => {
"msg": "List of ips for role Management: [{'ip': '1.2.3.4', 'vlan': '900 - CLOUD-DEV', 'role': 'Management'}]"
}
ok: [localhost] => (item=Production) => {
"msg": "List of ips for role Production: [{'ip': '4.5.6.7', 'vlan': '901 - CLOUD-DEV', 'role': 'Production'}]"
}
ok: [localhost] => (item=Backup) => {
"msg": "List of ips for role Backup: [{'ip': '8.9.10.11', 'vlan': '902 - CLOUD-DEV', 'role': 'Backup'}]"
}
PLAY RECAP **************************************************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
The equivalent variable declaration in plain json would be:
[
{
"ip": "1.2.3.4",
"vlan": "900 - CLOUD-DEV",
"role": "Management"
},
{
"ip": "4.5.6.7",
"vlan": "901 - CLOUD-DEV",
"role": "Production"
},
{
"ip": "8.9.10.11",
"vlan": "902 - CLOUD-DEV",
"role": "Backup"
}
]
If you have to load this from an external string (e.g. loading a file...), you can use the from_json filter in your playbook

How to add a key/value to a dict in an ansible playbook, conditional

Let's say I want to add InterestingVar dict key and the associated value when the variable test_var exists (passed with -e in command line), how can I do that ?
# ansible-playbook ./add_to_dict_on_condition.yml -i 127.0.0.1, -e env=test -e test_var=123
- hosts: localhost
gather_facts: no
vars:
- tags:
InterestingVar: "{{test_var}}" # How to omit this line if test_var == '' ?
Name: xxx
Env: "{{ env }}"
tasks:
- debug: var=tags
I tested
InterestingVar: "{{test_var|default(omit)}}
but I get :
"InterestingVar": "__omit_place_holder__caca01e207397883640613b08e8ce3a8fbdd6"
instead of nothing.
Any help will be greatly appreciated.
I use ansible 1.8
The only thing I can think of is to combine dictionaries with a set_fact task when your condition is met. This relies on the combine filter introduced in Ansible 2.0.
- hosts: localhost
connection: local
gather_facts: no
vars:
- tags:
Name: xxx
Env: "{{ env }}"
- optional_tags:
InterestingVar: "{{ test_var }}"
tasks:
- name: combine tags
set_fact:
tags: "{{ tags | combine(optional_tags) }}"
when: test_var is defined
- name: debug tags
debug: var=tags
Which outputs the following then test_var is not defined:
vagrant#Test-02:~$ ansible-playbook -i "localhost," conditional_key.yml -e "env=test"
PLAY ***************************************************************************
TASK [combine tags] ************************************************************
skipping: [localhost]
TASK [debug tags] **************************************************************
ok: [localhost] => {
"changed": false,
"tags": {
"Env": "test",
"Name": "xxx"
}
}
PLAY RECAP *********************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0
And this output when it is defined:
vagrant#Test-02:~$ ansible-playbook -i "localhost," conditional_key.yml -e "env=test" -e "test_var=123"
PLAY ***************************************************************************
TASK [combine tags] ************************************************************
ok: [localhost]
TASK [debug tags] **************************************************************
ok: [localhost] => {
"changed": false,
"tags": {
"Env": "test",
"InterestingVar": "123",
"Name": "xxx"
}
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0
If you are unable to use 2.0+ then another option may be to change Ansible's hash behaviour to merge dictionaries rather than overriding them by setting:
hash_behaviour=merge
in your ansible.cfg.
With this you could then use something like this:
- hosts: localhost
connection: local
gather_facts: no
vars:
- tags:
Name: xxx
Env: "{{ env }}"
- tags:
InterestingVar: "{{ test_var }}"
tasks:
- name: debug tags
debug: var=tags
With your vars defined in the following files:
vagrant#Test-01:~$ cat tags.yml
tags:
Name: xxx
Env: "{{ env }}"
vagrant#Test-01:~$ cat optional_tags.yml
tags:
InterestingVar: "{{ test_var }}"
This then gives you the output you want but you have to make sure not to include optional_vars.yml when you don't have test_var defined:
vagrant#Test-01:~$ ansible-playbook -i "localhost," conditional_key.yml -e "env=test" -e "test_var=123" -e#tags.yml -e#optional_tags.yml
PLAY [localhost] **************************************************************
TASK: [debug tags] ************************************************************
ok: [localhost] => {
"var": {
"tags": {
"Env": "test",
"InterestingVar": "123",
"Name": "xxx"
}
}
}
PLAY RECAP ********************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0
Be aware that when using this approach any expected overriding of dictionaries through inheritance will now merge the dictionaries instead so this may not be all that useful for anyone overriding things in their inventories.