ansible hash to json to jinja template - json

i got the following and stuck by getting the right answer. i got a dict that i want to template with item.key in file name and all the values in the template.
my_dict:
name1:
{ path=/x/y/z, action=all, filter=no },
{ path=/a/b/c, action=some, filter=yes }
name2:
{ path=/z/y/x, action=nothing, filter=no },
{ path=/c/b/a, action=all, filter=yes }
tasks:
- name: generate check config
template:
src: check.j2
dest: "{{ config_dir }}/{{ item.key }}-directories.json"
owner: Own
group: Wheel
mode: 0644
with_dict:
- "{{ my_dict }}"
when:
- my_dict is defined
become: true
My template looks like
{
"configs": [
{% for value in my_dict %}
{
"path": "{{ value.path }}",
"action": "{{ value.action }}",
{% if value.filter is defined %}
"filter": "{{ value.filter }}"
{% endif %}
}{% if !loop.last %},{% endif %}
{% endfor %}
]
}
So i tested so much that now i dont see any forest cause of too many trees.
Above should result in 2 files.
File name = name1-directories.json
Content:
{
"configs": [
{
"path": /x/y/z,
"action": all,
"filter": no
},
{
"path": /a/b/c,
"action": some,
"filter": yes
}
]
}
Thx in Advance

Let me start with the following. I see some problems with your current solution.
You're template references the value of the array items with value.<key> when it should instead read item.value.<key>.
with_dict expects a dict, but you're passing an array containing a dict as the only element. In yaml, - denotes array elemtents. To use that correctly you just write: with_dict: "{{ my_dict }}"
Using the shorthand yaml syntax is discouraged in ansible as it makes the playbooks harder to read.
I would suggest you do the following:
There is a jinja2 Filter that just converts your dict to json:
{{ dict_variable | to_json }} # or
{{ dict_variable | to_nice_json }}
The second one makes it human readable. What you're currently trying to do may work ( haven't looked into it so thoroughly) but it's not pretty and error prone.
To make it work with the jinja2 filter restructure your variables at the top the following way:
my_dict:
- name1:
configs:
- path: /x/y/z
action: all
filter: no
- path: /a/b/c
action: some
filter: yes
- name2:
configs:...
When the vars are formatted like this, you can just use the copy module to print the configs to the files like this:
- name: Print the configs to the files
copy:
content: "{{ item.value | to_nice_json }}"
dest: "{{ config_dir }}/{{ item.key }}-directories.json"
with_dict: "{{ my_dict }}"

Related

Can't extract data from nested array in Ansible

I am trying to extract the values of vs_name for every item in the array list but it looks like there is something I am doing wrong but can't figure it out.
Here is the output I want to parse
ok: [localhost] => {
"msg": {
"AV-FAS": {
"vs_name": "AV-FAS",
"vs_type": "admin"
},
"AV-FAS-01": {
"vs_name": "AV-FAS-01",
"vs_type": "node"
},
"AV-FAS-02": {
"vs_name": "AV-FAS-02",
"vs_type": "node"
}
Here is my code:
- name: populate vs list
set_fact:
vs_list: "{{ vs_list|default([]) }} + [ '{{ item.vs_name }}' ]"
with_items: "{{ output }}"
Q: "Extract the values of vs_name."
A: Use filter json_query. For example
- set_fact:
vs_list: "{{ output|json_query('*.vs_name') }}"
gives
"vs_list": [
"AV-FAS",
"AV-FAS-01",
"AV-FAS-02"
]
The next option is mapping of an attribute. The filter dict2items is needed to convert the dictionary to a list. For example, the task below gives the same result
- set_fact:
vs_list: "{{ output|dict2items|
map(attribute='value.vs_name')|
list }}"
Q: "List vs_name when vs_type = 'admin'"
A: Add filter selectattr to the pipe. For example
- set_fact:
vs_list: "{{ output|dict2items|
selectattr('value.vs_type', 'eq', 'admin')|
map(attribute='value.vs_name')|
list }}"
gives
"vs_list": [
"AV-FAS"
]

how to iterate over list from json in ansible

I tried to register an output to a variable, but i couldnt filter the way i want it.
output:
oc get hpa -o json |jq -r '.items[].spec'
{
"maxReplicas": 3,
"minReplicas": 1,
"scaleTargetRef": {
"apiVersion": "apps.openshift.io/v1",
"kind": "DeploymentConfig",
"name": "hello-openshift"
},
"targetCPUUtilizationPercentage": 70
}
{
"maxReplicas": 4,
"minReplicas": 2,
"scaleTargetRef": {
"apiVersion": "apps/v1",
"kind": "Deployment",
"name": "testrhel"
},
"targetCPUUtilizationPercentage": 79
}
Register the output to variable
- name: check for existing
shell: oc get hpa -o json |jq -r '.items[].spec'
register: existing
I would like to loop the output.name and compare it to another variable.
- name: set_fact
exist: {% if item.name == newvar and item.kind == newvar2 %}yes{%else%}no{%endif%}
loop:
- "{{ existing }}"
- name: task
shell: do something
when: exist == yes
Thanks in advance.
edit:
currently i am using below to get my comparison for the variables.
- name: Get existing hpa output
shell: oc get hpa -o json -n {{ namespace }} |jq -r '.'
register: tempvar
- name: set hpa variable to fact
set_fact:
existing_deploy: "{{ tempvar.stdout}}"
- name: Comparing existing hpa to new config
set_fact:
hpa_exist: "{% if deploy_type == item.spec.scaleTargetRef.kind|lower and deploy_name == item.spec.scaleTargetRef.name|lower %}yes{% else %}no{% endif %}"
with_items:
- "{{ existing_deploy['items'] }}"
but the variable got overwrite when i trying to use when condition
- name: task a
include_tasks: a.yml
when: hpa_exist
- name: task b
include_tasks: b.yml
when: not hpa_exist
deploymentconfig/hello-openshift condition always fail even when it is true. leading to execute task b, which is not supposed to
Check out the documentation of the shell module.
The output of the shell on stdout will be in <var>.stdout (so in existing.stdout in your case.)
Once you got that, you obviously have json as text, but you want to parse it. To do that, use the from_json filter as shown in this answer.
Summa summarum your task should look like this:
- name: set_fact
set_fact:
exist: {% if item['scaleTargetRef']['name'] == newvar and item['scaleTargetRef']['kind'] == newvar2 %}yes{% else %}no{% endif %}
loop: "{{ existing.stdout | from_json}}"
But your output needs to be a valid list, so basically, it needs to look like this:
[{
"maxReplicas": 3,
"minReplicas": 1,
"scaleTargetRef": {
"apiVersion": "apps.openshift.io/v1",
"kind": "DeploymentConfig",
"name": "hello-openshift"
},
"targetCPUUtilizationPercentage": 70
},
{
"maxReplicas": 4,
"minReplicas": 2,
"scaleTargetRef": {
"apiVersion": "apps/v1",
"kind": "Deployment",
"name": "testrhel"
},
"targetCPUUtilizationPercentage": 79
}]
But you might actually have a logic error, because you are looping over the list and overwriting the variable exist on every turn. So you will end up with one variable exist in the end and that will hold the value of the last iteration.
Check out how to register variables with a loop if you need the output of every iteration.
If you want to do something for each item that meets the condition, you can do this:
- name: check for existing
shell: oc get hpa -o json | jq -r '.items[].spec'
register: existing
- name: include tasks a
include_tasks: a.yml
when:
- deploy_type == item['scaleTargetRef']['kind'] | lower
- deploy_name == item['scaleTargetRef']['name'] | lower
loop: "{{ existing.stdout | from_json }}"
- name: include tasks b
include_tasks: b.yml
when: (deploy_type != item['scaleTargetRef']['kind'] | lower) or
(deploy_name != item['scaleTargetRef']['name'] | lower)
loop: "{{ existing.stdout | from_json }}"
You do not need any of the set_fact stuff in that case.

Loop over Ansible variable array in Jinja2 template to create nested dictionary

Here is the output.
"result.containers":[
{
"Image":"ca.docker/webproxy:1.0.0",
"Names":[
"/customer1"
]
},
{
"Image":"docker.local/egacustomer:1.0.1",
"Names":[
"/webproxy"
]
}
]
I'm trying to create a nested dictionary using jinja2.
i'm trying to achieve the below using results.container and setfact.
"containerlist": "[webproxy:
name: customer1,
egacustomer:
name: webproxy]"
Here is my jinja2 code.
- set_fact:
containerlist: |
[
{% for item in result.containers %}
{{ item.Image | regex_replace('.*?/(.*?):.*', '\\1') }}:
'name': {{ item.Names | regex_replace("^/", "") }},
{% endfor %}
Which throws the error. Could someone help me with the right Jinja2 Code.Any help would be greatly appreciated
"containerlist": "[\n\\1:\n 'name': ['/customer'],\n\\1:\n 'name': ['/webproxy'],\n,\n]\n"
For what you're trying to do I would normalize use an Ansible looping task rather than trying to use a Jinja {% for ... %} loop. For example, these tasks...
- set_fact:
container_list: >-
{{
container_list + [{
item.Image.split('/')[-1].split(':')[0]:
item.Names[0][1:]
}]
}}
loop: "{{ result.containers }}"
vars:
container_list: []
- debug:
var: container_list
In the set_fact task, we're looping over result.containers, and for each iteration of the loop we redefine container_list as "the current contents of container_list + a new dictionary".
The above produces the following output:
TASK [set_fact] ****************************************************************
ok: [localhost] => (item={'Image': 'ca.docker/webproxy:1.0.0', 'Names': ['/customer1']})
ok: [localhost] => (item={'Image': 'docker.local/egacustomer:1.0.1', 'Names': ['/webproxy']})
TASK [debug] *******************************************************************
ok: [localhost] => {
"container_list": [
{
"webproxy": "customer1"
},
{
"egacustomer": "webproxy"
}
]
}

transform values of yaml hash into keys of json hash in Ansible

I'm trying to get Ansible to convert an array of hashes, into to a list of key value pairs with the keys being one of the values from the first hash and the values being a different value from the first hash.
An example will help.
I want to convert :-
TASK [k8s_cluster : Cluster create | debug result of private ec2_vpc_subnet_facts] ***
ok: [localhost] => {
"result": {
"subnets": [
{
"availability_zone": "eu-west-1c",
"subnet_id": "subnet-cccccccc",
},
{
"availability_zone": "eu-west-1a",
"subnet_id": "subnet-aaaaaaaa",
},
{
"availability_zone": "eu-west-1b",
"subnet_id": "subnet-bbbbbbbb",
}
]
}
}
into
eu-west-1a: subnet-aaaaaaaa
eu-west-1b: subnet-bbbbbbbb
eu-west-1c: subnet-cccccccc
I've tried result.subnets | map('subnet.availability_zone': 'subnets.subnet_id') (which doesn't work at all) and json_query('subnets[*].subnet_id' which simply pickes out the subnet_id values and puts them into a list.
I think I could do this with Zip and Hash in Ruby but I don't know how to make this work in Ansible, or more specifically in Jmespath.
I have generated the below list I will add a new line to the generated list(thought to share this first)
---
- name: play
hosts: localhost
tasks:
- name: play
include_vars: vars.yml
- name: debug
debug:
msg: "{% for each in subnets %}{{ each.availability_zone }}:{{ each.subnet_id }}{% raw %},{% endraw %}{% endfor %}"
output --->
ok: [localhost] => {
"msg": "eu-west-1c:subnet-cccccccc,eu-west-1a:subnet-aaaaaaaa,eu-west-1b:subnet-bbbbbbbb,"
}
Jmespath does not allow to use dynamic names in multi select hashes. I have found an extension to jmespath allowing to do such thing by using key references, but it is not part of the plain jmespath implementation nor ansible.
To do this in plain ansible, you will have to create a new variable and populate it with a loop. There might be other ways using other filters but this is the solution I came up with:
- name: Create the expected hash
set_fact:
my_hash: >-
{{
my_hash
| default({})
| combine({ item.availability_zone: item.subnet_id })
}}
loop: "{{ subnets }}"
- name: Print result
debug:
var: my_hash

Ansible: filter a list by its attributes

I have variable named "network" registered in Ansible:
{
"addresses": {
"private_ext": [
{
"type": "fixed",
"addr": "172.16.2.100"
}
],
"private_man": [
{
"type": "fixed",
"addr": "172.16.1.100"
},
{
"type": "floating",
"addr": "10.90.80.10"
}
]
}
}
Is it possible to get the IP address ("addr") with type="floating" doing something like this?
- debug: var={{ network.addresses.private_man | filter type="fixed" | get "addr" }}
I know the syntax is wrong but you get the idea.
To filter a list of dicts you can use the selectattr filter together with the equalto test:
network.addresses.private_man | selectattr("type", "equalto", "fixed")
The above requires Jinja2 v2.8 or later (regardless of Ansible version).
Ansible also has the tests match and search, which take regular expressions:
match will require a complete match in the string, while search will require a match inside of the string.
network.addresses.private_man | selectattr("type", "match", "^fixed$")
To reduce the list of dicts to a list of strings, so you only get a list of the addr fields, you can use the map filter:
... | map(attribute='addr') | list
Or if you want a comma separated string:
... | map(attribute='addr') | join(',')
Combined, it would look like this.
- debug: msg={{ network.addresses.private_man | selectattr("type", "equalto", "fixed") | map(attribute='addr') | join(',') }}
I've submitted a pull request (available in Ansible 2.2+) that will make this kinds of situations easier by adding jmespath query support on Ansible. In your case it would work like:
- debug: msg="{{ addresses | json_query(\"private_man[?type=='fixed'].addr\") }}"
would return:
ok: [localhost] => {
"msg": [
"172.16.1.100"
]
}
Not necessarily better, but since it's nice to have options here's how to do it using Jinja statements:
- debug:
msg: "{% for address in network.addresses.private_man %}\
{% if address.type == 'fixed' %}\
{{ address.addr }}\
{% endif %}\
{% endfor %}"
Or if you prefer to put it all on one line:
- debug:
msg: "{% for address in network.addresses.private_man if address.type == 'fixed' %}{{ address.addr }}{% endfor %}"
Which returns:
ok: [localhost] => {
"msg": "172.16.1.100"
}