Extracting data from json within ansible - json

I have an ansible playbook that creates some IAM users in AWS.
I'd like the playbook to return the username and access keys details for each account created.
Registering the output from the task is fairly straightforward:
- name: Create IAM users
iam:
iam_type: user
name: "{{ item }}"
state: present
access_key_state: create
with_items:
- "user1"
- "someotheruser"
- "user3"
register: users
I'm finding doing something with that output tricky.
The Json it produces is as follows: (slightly truncated to reduce the length here)
ok: [localhost] => {
"users": {
"changed": true,
"msg": "All items completed",
"results": [
"user_meta": {
"access_keys": [
{
"access_key_id": "key1",
"access_key_selector": "HMAC",
"create_date": "2016-05-19T08:37:11.007Z",
"secret_access_key": "secretkey1",
"status": "Active",
"user_name": "user1"
}
],
}
},
{
"user_meta": {
"access_keys": [
{
"access_key_id": "key2",
"access_key_selector": "HMAC",
"create_date": "2016-05-19T08:37:13.391Z",
"secret_access_key": "secretkey2",
"status": "Active",
"user_name": "someotheruser"
}
],
}
},
{
"user_meta": {
"access_keys": [
{
"access_key_id": "key3",
"access_key_selector": "HMAC",
"create_date": "2016-05-19T08:37:16.243Z",
"secret_access_key": "secretkey3",
"status": "Active",
"user_name": "user3"
}
],
}
}
]
}
}
I've want to display this neatly on completion of the playbook, the closest I've come to getting what I'm after is use debug as follows:
- debug: var="users.results[0].user_meta.access_keys"
- debug: var="users.results[1].user_meta.access_keys"
- debug: var="users.results[2].user_meta.access_keys"
However that feels lame. What if I add user 4 and 5?
I'd like to be able to refer to the entire results array so that it will work no matter how many users I add to the playbook.
Is there a way to achieve this?
As a side note, I've also tried to use from_json, e.g.
- debug: msg="{{ (users.stdout|from_json).results }}"
but I don't know if I'm hitting a bug with ansible 2.0.1.0 or if I'm using it wrong but I could only ever get "type error. expected string or buffer"
Any thoughts gladly received.

You can iterate through the registered variable as well.
Something like this should work:
- name : debug access keys
debug : var="{{ item.user_meta.access_keys }}"
with_items: users.results

In case someone else wants to do the same thing...
I found the best solution for me was to count the number of objects in the array using 'length' and the then set the range that debug should iterate over.
I subtract "1" from the value returned since the first object in the array is referenced by "0".
- name: Record Users Access Keys
debug: var=users.results[{{ item }}].user_meta.access_keys
with_sequence: start=0 end={{users.results|length -1}}
Hope that helps someone looking to do something similar.

Ansiballer's answer worked for me, but if anyone else is looking for something simpler you can use json_query:
- name: show access keys
debug:
var: item
loop: "{{ users | json_query('results[*].user_meta.access_keys') }}"
Ansible docs on json_query
You may have to install jmespath for this to work (apt-get install python-jmespath or similar)

Related

jmespath Select arrays only if they contain specific value

I have this JSON as input:
{
"users": {
"alpha": [
"read",
"annotate",
"write",
"delete",
"manage"
],
"beta": [
"read",
"annotate",
"write",
"delete",
"manage"
],
"gamma": [
"read",
"annotate",
"write",
"delete"
],
"delta": [
"read",
"annotate",
"write",
"delete",
"manage"
]
}
}
And I've been asked to return the lists (users) only if they contain the element manage. Since this has to be processed with Ansible filter json_query, it should use only the JMESPath query language.
Expected result is something like:
["alpha", "beta", "delta"]
Try this
managers: "{{ users|dict2items|json_query(_query) }}"
_query: "[?contains(value, 'manage')].key"
It's an analogy to (credit #Zeitounator)
managers: "{{ users|dict2items|selectattr('value', 'contains', 'manage')|
map(attribute='key') }}"
Q: "Do you always need a list?"
A: In most cases yes, but not always. It depends on what you want to achieve. For example, the dictionary is fine if you want to find out whether all users contain the element manage or not
all_users_manage: "{{ users|json_query(_query2) }}"
_query2: "*.contains(#, 'manage')"
gives
all_users_manage:
- true
- true
- false
- true
Test the list
- debug:
msg: Not all users are managers.
when: not all_users_manage is all
Or, you can find out whether any user contains the element manage
- debug:
msg: There are managers among the users.
when: all_users_manage is any

ansible json-query trying to select "id" by content file wildcard

Very new to JSON.
I'm trying to extract 2 variables from this json file.
It has many files and id's but I only want the file & id if it contains es7.x86_64
When done my desired variables would be:
id=13140
file=NessusAgent-8.3.0-es7.x86_64.rpm
{
"banners": [],
"containsRequiredAuth": true,
"created_at": "2017-10-13T00:53:32.137Z",
"description": "Download Nessus Agents for use with Tenable.io and Nessus Manager",
"documentation_link": null,
"downloads": [
{
"created_at": "2021-06-29T19:06:41.776Z",
"description": "Red Hat ES 7 (64-bit) / CentOS 7 / Oracle Linux 7 (including Unbreakable Enterprise Kernel)",
"file": "NessusAgent-8.3.0-es7.x86_64.rpm",
"id": 13140,
"meta_data": {
"md5": "f67a2bdd2a7180f66b75f319439d56d5",
"product": "Nessus Agents - 8.3.0",
"product_notes": null,
"product_release_date": "06/29/2021",
"product_type": "default",
"release_date": "06/03/2021",
"sha256": "8a6452086ce0a7193e0f24b1f2adbff3aa6bd0f4ac519384e8453bb68bae0460",
"version": "8.3.0"
},
"name": "NessusAgent-8.3.0-es7.x86_64.rpm",
"page_id": 61,
"publish": true,
"required_auth": false,
"size": 16375828,
"sort_order": null,
"type": "download",
"updated_at": "2021-06-29T19:08:47.628Z"
},
My utterly failed attempt to assign file & id variables that have es7.x86_64.
- name: Convert agent_tempfile to json and register result
shell: python -m json.tool "{{ agent_tempfile }}"
register: result
- name: Extract file & id for es7.x86_64 rpm's
set_fact:
agent_id: "{{ result | json_query('downloads[*es7.x86_64*].id') | first }}"
agent_file: "{{ result | json_query('downloads[*es7.x86_64*].file') | first }}"
I have a feeling I'm going to be doing a lot more of these types of queries soon. Can some one also direct me to a good guide that details parsing specific values from JSON output? The stuff I've found so far just lists arrays but I really want to know how to pull specific data out.
First, there are some great tools out there for playing with JMESPath syntax (the syntax used by the json_query filter). The examples in the JMESPath tutorial are all "live": you can paste your own data into the text fields, and then experiment with filters and check the result.
The jpterm command is a terminal tool for experimenting with JMESPath queries. This is my personal favorite.
To look for items that contain a specific substring (like es7.x86_64), you can use the contains operator, like this:
json_query("downloads[?contains(name, 'es7.x86_64')]")
To make this work for your code, we first need to deal with the fact
that the result of your first task is going to be a string, rather
than a dictionary. We'll need to pass the standard output through the
from_json filter.
We can also avoid having two almost identical json_query expression
by moving the bulk of the expression into a task-local variable.
This gives us something like:
- hosts: localhost
gather_facts: false
tasks:
- command: cat data.json
register: result
- set_fact:
agent_id: "{{ selected[0].id }}"
agent_file: "{{ selected[0].file }}"
vars:
selected: >-
{{
result.stdout |
from_json |
json_query("downloads[?contains(name, 'es7.x86_64')]")
}}
- debug:
msg:
- "ID: {{ agent_id }}"
- "FILE: {{ agent_file }}"
When that task runs, the value of selected will be something like:
[
{
"file": "NessusAgent-8.3.0-es7.x86_64.rpm",
"id": 13140,
"name": "NessusAgent-8.3.0-es7.x86_64.rpm",
"page_id": 61,
"publish": true,
"required_auth": false,
"size": 16375828,
"sort_order": null,
"type": "download",
"updated_at": "2021-06-29T19:08:47.628Z"
}
]
This assumes you're only expecting a single result, so we can just ask
for selected[0] to get at that dictionary, and then it's a simple
matter of getting at the .id and .file attributes.
Running the above playbook produces:
TASK [debug] *********************************************************************************
ok: [localhost] => {
"msg": [
"ID: 13140",
"FILE: NessusAgent-8.3.0-es7.x86_64.rpm"
]
}

ansible dictionary key value pairs

I am new to Ansible/json and I am trying to parse the following json:
{
"resultCPU": {
"changed": false,
"msg": "All items completed",
"results": [
{
"ansible_facts": {
"CPU": "6",
"VM": "tigger"
},
{
"ansible_facts": {
"CPU": "4",
"VM": "pooh"
},
I need to set the value of the items in this json so that pooh=4 and tigger=6. I will need to refer to both these values later (advice on how to do that would help as well).
I have tried using cpuvm "{{ resultCPU.results |selectattr('VM') |map(attribute='CPU')|list }}" but it complains "'dict object' has no attribute VM".
What am I doing wrong?
Your selectattr ignored the ansible_facts between results: [] and the VM you tried to map. Thankfully, the attribute kwarg understands dot notation
- debug:
msg: >-
VMs are {{ resultCPU.results | map(attribute="ansible_facts.VM") | list }}
CPUs are {{ resultCPU.results | map(attribute="ansible_facts.CPU") | list }}
The together filter may interest you, too
and as for "I will need to refer to both these values later (advice on how to do that would help as well)." that is usually what set_fact: is used for, but without more specifics about what you have tried and what shape you are expecting, it's hard to give a more specific answer

Ansible fetching Stdout key value

I need some help regarding, how to fetch specific value using Ansible
My task:
- name: 'check Describe Information'
debug:
var: describeresult.stdout
I need IP address value from below stdout, what should I put in debug var to fetch IP address
TASK [check Describe Information] **********************************************
task path: /home/tom/Getipaddress.yml:28
ok: [127.0.0.1] => {
"describeresult.stdout": {
"failures": [],
"tasks": [
{
"attachments": [
{
"details": [
{
"name": "subnetId",
"value": "subnet-xxxxxxxxxxxxxx"
},
{
"name": "networkInterfaceId",
"value": "eni-xxxxxxxxxxxxxxxx"
},
{
"name": "macAddress",
"value": "xxxxxxxxxxxxxxxxxx"
},
{
"name": "privateIPv4Address",
"value": "xxxxxxxxxxxxxxxxxx"
}
Plus I am using AWS ECS command to generate the above output with --output as a JSON not sure how to use --query to filter or fetch above IP address only
Use json_query. For example
- set_fact:
my_privateIPv4Address: "{{ describeresult.stdout.tasks|
json_query(query) }}"
vars:
query: "[].attachments[].details[?name=='privateIPv4Address'].value"
json_query returns by default a list.
You can use something like this for getting private IP address only
"{{describeresult.stdout['tasks'][0]['attachments'][0]['details'][3]['value']}}"
The above might have slight syntax issues but should work.
I have done something like this earlier for ec2_group_info module in ansible
You can start off by generating output in pieces describeresult.stdout['tasks'] should return JSON for the task attribute.
[0] indicates the first element when there is a box bracket for a JSON implying more than one element/list.
Let me know how it goes.

Json parse through ansible

I have below json
"edge_router_uuid.stdout": {
"buildInfo": {
"buildNumber": "20004",
"buildTimestamp": "1539995399724",
},
"isUp": true,
"pod": "gateway",
"reachable": true,
"region": "dc-1",
"tags": {
........
Actually it is a big json and I am showing above is just a part of it. I need to use the "debug" in ansible to get the variable values for region and reachable values. When I am trying the below
- debug:
var: edge_router_uuid.stdout.region
getting below error.
ok: [10.10.10.10] => {
"edge_router_uuid.stdout.region": "VARIABLE IS NOT DEFINED!"
The JSON syntax is wrong. The problem is "," behind the last value
"buildInfo": {
"buildNumber": "20004",
"buildTimestamp": "1539995399724",
}
Correct syntax is
"buildInfo": {
"buildNumber": "20004",
"buildTimestamp": "1539995399724"
}
Finally I am able to get it as below
- set_fact:
response_dict: "{{ edge_router_uuid.stdout }}"
I ended up with a dictionary named response_dict. Then I am able to get the region value like this:
- debug:
var: response_dict.region