How to parse specific variable of a dict in Ansible - json

Question

I have an Ansible job that run on 2 or more urls. Each url returns the same variables with different values (but have similar pattern). Here is the return data of the job:
"message": [
{
"url": "http://0.0.0.1:xxx1",
"content": [
{
"message": "This is message number 1",
"message2": "This is message2 number 1"
},
{
"message": "This is message number 2",
"message2": "This is message2 number 2"
},
{
"message": "This is message number 3",
"message2": "This is message2 number 3"
}
]
},
{
"url": "http://0.0.0.2:xxx2",
"content": [
{
"message": "This is message number 1",
"message2": "This is message2 number 1"
},
{
"message": "This is message number 2",
"message2": "This is message2 number 2"
},
{
"message": "This is message number 3",
"message2": "This is message2 number 3"
}
]
}
I want to parse the variable 'message' without changing the structure of the returned data. My expected result is like this:
"message": [
{
"url": "http://0.0.0.1:xxx1",
"content": [
{
"message": "message number 1",
"message2": "This is message2 number 1"
},
{
"message": "message number 2",
"message2": "This is message2 number 2"
},
{
"message": "message number 3",
"message2": "This is message2 number 3"
}
]
},
{
"url": "http://0.0.0.2:xxx2",
"content": [
{
"message": "message number 1",
"message2": "This is message2 number 1"
},
{
"message": "message number 2",
"message2": "This is message2 number 2"
},
{
"message": "message number 3",
"message2": "This is message2 number 3"
}
]
}
I know how to parse the data. My struggle is to keep the structure as is. How can I achieve what I want?

Answer 1

Convert the structure in Jinja
updates: |
{% for i in message %}
- content:
{% for m in i.content %}
{% set arr=m.message.split() %}
{% set message={'message': arr[2:]|join(' ')} %}
- {{ m|combine(message) }}
{% endfor %}
{% for k in i %}
{% if k != 'content' %}
{{ k }}: {{ i[k] }}
{% endif %}
{% endfor %}
{% endfor %}
gives
updates: |-
- content:
- {'message': 'message number 1', 'message2': 'This is message2 number 1'}
- {'message': 'message number 2', 'message2': 'This is message2 number 2'}
- {'message': 'message number 3', 'message2': 'This is message2 number 3'}
url: http://0.0.0.1:xxx1
- content:
- {'message': 'message number 1', 'message2': 'This is message2 number 1'}
- {'message': 'message number 2', 'message2': 'This is message2 number 2'}
- {'message': 'message number 3', 'message2': 'This is message2 number 3'}
url: http://0.0.0.2:xxx2
Convert the block to YAML
messag2: "{{ updates|from_yaml }}"
gives
messag2:
- content:
- message: message number 1
message2: This is message2 number 1
- message: message number 2
message2: This is message2 number 2
- message: message number 3
message2: This is message2 number 3
url: http://0.0.0.1:xxx1
- content:
- message: message number 1
message2: This is message2 number 1
- message: message number 2
message2: This is message2 number 2
- message: message number 3
message2: This is message2 number 3
url: http://0.0.0.2:xxx2
Example of a complete playbook for testing
- hosts: localhost
vars:
message:
- content:
- message: This is message number 1
message2: This is message2 number 1
- message: This is message number 2
message2: This is message2 number 2
- message: This is message number 3
message2: This is message2 number 3
url: http://0.0.0.1:xxx1
- content:
- message: This is message number 1
message2: This is message2 number 1
- message: This is message number 2
message2: This is message2 number 2
- message: This is message number 3
message2: This is message2 number 3
url: http://0.0.0.2:xxx2
updates: |
{% for i in message %}
- content:
{% for m in i.content %}
{% set arr=m.message.split() %}
{% set message={'message': arr[2:]|join(' ')} %}
- {{ m|combine(message) }}
{% endfor %}
{% for k in i %}
{% if k != 'content' %}
{{ k }}: {{ i[k] }}
{% endif %}
{% endfor %}
{% endfor %}
messag2: "{{ updates|from_yaml }}"
tasks:
- debug:
var: updates
- debug:
var: messag2
Q: "This solution works in vars. Using it in tasks, since my data is from json output, set_fact gives me an offending line in the result."
A: There is no reason for moving the updates and messag2 variables into the set_fact task.
a) The play below works fine also when message comes from a task
- hosts: localhost
vars:
updates: |
{% for i in message %}
- content:
{% for m in i.content %}
{% set arr=m.message.split() %}
{% set message={'message': arr[2:]|join(' ')} %}
- {{ m|combine(message) }}
{% endfor %}
{% for k in i %}
{% if k != 'content' %}
{{ k }}: {{ i[k] }}
{% endif %}
{% endfor %}
{% endfor %}
messag2: "{{ updates|from_yaml }}"
tasks:
- include_vars:
file: message.yml
- debug:
var: updates
- debug:
var: messag2
b) You can put the declarations of updates and messag2 into the set_fact tasks. The play below works fine
- hosts: localhost
tasks:
- include_vars:
file: message.yml
- set_fact:
updates: |
{% for i in message %}
- content:
{% for m in i.content %}
{% set arr=m.message.split() %}
{% set message={'message': arr[2:]|join(' ')} %}
- {{ m|combine(message) }}
{% endfor %}
{% for k in i %}
{% if k != 'content' %}
{{ k }}: {{ i[k] }}
{% endif %}
{% endfor %}
{% endfor %}
- set_fact:
messag2: "{{ updates|from_yaml }}"
- debug:
var: updates
- debug:
var: messag2
c) You can't put both declarations of updates and messag2 into single set_fact task because messag2 knows nothing about updates this way. The play below
- hosts: localhost
tasks:
- include_vars:
file: message.yml
- set_fact:
updates: |
{% for i in message %}
- content:
{% for m in i.content %}
{% set arr=m.message.split() %}
{% set message={'message': arr[2:]|join(' ')} %}
- {{ m|combine(message) }}
{% endfor %}
{% for k in i %}
{% if k != 'content' %}
{{ k }}: {{ i[k] }}
{% endif %}
{% endfor %}
{% endfor %}
messag2: "{{ updates|from_yaml }}"
- debug:
var: updates
- debug:
var: messag2
failes
... The error was: 'updates' is undefined ...