Jinja2 filter list using string contains test - jinja2

I'm trying to filter a list in ansible in Jinja2 when the elements contain a string, but the Jinja documentation doesn't seem clear enough for me to figure it out.
This is what I have so far:
- name: run script
command: /usr/tmp/run_script.py
register: script_results
- name: display run info
debug:
var: "{{script_results.stdout_lines | select(\"'running script' in script_results.stdout_lines\") }}"
But all I get is the error:
"<generator object _select_or_reject at 0x13851e0>": "VARIABLE IS NOT DEFINED!"
So for example, if stdout_lines contains ["apples","running script one","oranges","running script two"], I want to print
running script one
running script two
They have documentation for select and documentation for built-in-tests, but they don't display the "in" test, and I don't know how they work in the context of this ansible variable.
I tried solving it like this:
- name: display run info
debug:
var: item
with_items: "{{script_results.stdout_lines}}"
when: "'running script' in item"
But that displays "skipping" for every line that doesn't pass the test ... kinda defeating the purpose!

The select filter would take another filter. Like in the docs odd, which will return only the odd elements of the list. The filter you would like to combine select with is equalto.
Now here's the thing. Ansible bundles a very old version of Jinja2, which simply does not contain the equalto filter. Yes, that renders it useless unless you want to filter odd elements. (Which nobody ever in history wanted to...)
Furthermore I was yet unable to make custom filter plugins work in Ansible 2. So you're pretty much forced to hack something ugly together.
helloV already showed one option. Here is another idea:
- name: run script
shell: /usr/tmp/run_script.py | grep "running script"
register: script_results
Update:
I recently discovered you can use match (not a standard Jinja2 filter but added by Ansible) together with select. Thats a good replacement for the eualto filter plus you can use regular expressions. This should work:
{{ script_results.stdout_lines | select("match", ".*running script.*") }}

I understand there may be more than one way to do this. Will this work for you?
- debug: var={{item}}
when: item.find('running script') > -1
with_items: script_results.stdout_lines

I know this is an old thread but I too was looking for an answer to this and I was able to use the if in method. I think the other issue you were having was how to display the resulting list. You can do a jinja loop right in the ansible modules, debug for example. Of course since Ansible is horrible with presenting data using debug, you could also utilize the blockinfile module with the pipe (|). Hopefully this helps others and gives yet another option.
- debug:
msg: |
{% for item in (script_results.stdout_lines) %}
{% if 'running script' in item %}
{{ item }}
{% endif %}
{% endfor %}
tags: debug
or to add the filtered data to a file:
- name: Update the scriptStatus file.
delegate_to: localhost
run_once: TRUE
blockinfile:
path: '/tmp/scriptStatus.txt'
block: |
{% for item in (script_results.stdout_lines) %}
{% if 'running script' in item %}
{{ item }}
{% endif %}
{% endfor %}
tags: chkScripts

I ended up writing a python script to do it, because I couldn't get ansible or ancient-jinja2 to make the cut.
Ansible tasks:
- name: gather run info
command: "{{role_path}}/files/print_results.py {{script_results.stdout_lines}}"
register: script_print_results
delegate_to: 127.0.0.1
run_once: true
- name: display run info
debug:
var: script_print_results.stdout_lines
delegate_to: 127.0.0.1
run_once: true
Python script:
for result_line in sys.argv[1:]:
if "running script:" in result_line:
print result_line[1:-1]

You can build a new list with set_fact and print the elements of a new list.
- hosts: localhost
gather_facts: false
vars:
script_stdout_lines:
- apples
- running script one
- oranges
- running script two
tasks:
- set_fact:
new_list: "{{ new_list | default([]) + [item] }}"
with_items: "{{ script_stdout_lines }}"
when: '"running script" in item'
- debug: var=new_list
Result:
TASK [set_fact] *********************************************************************************************************************
skipping: [localhost] => (item=apples)
ok: [localhost] => (item=running script one)
skipping: [localhost] => (item=oranges)
ok: [localhost] => (item=running script two)
TASK [debug] ************************************************************************************************************************
ok: [localhost] => {
"new_list": [
"running script one",
"running script two"
]
}
It prints skipping during set_fact operation but at the end it provides a new list with the only matching items.

Related

JMESPath Query in Ansible

--- EDIT ---
Partial-Solution: Messed around with the JMESPath syntax and was able to successfully get a match for the first test case (without the optional variable) using:
jmesquery: "{{ datacenter }}{{ subcategory }}.{{ refine_hosts }}.[*][].[*][][]"
I am writing an Ansible Playbook that takes a list of hosts from a network server, parses the JSON list, and finds hostnames that matches the user's input when they deploy the playbook as a Jenkin's Job through it's API.
The issue I am encountering is that I am unable to successfully query the JSON host list. Currently, I am only trying to run the following test case:
datacenter: a
subcategory: bc
refine_hosts: QA
However, the final version of this playbook should be able to take in values for datacenter, subcategory, and refine_hosts with an optional input value of host_type. An example test case including the optional input value would be the following:
datacenter: a
subcategory: bc
refine_hosts: QA
host_type: WEBSITE
In my playbook, I am using JMESPath within the following task:
- name: Build HOSTS list
set_fact:
hosts_list: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: '%%datacenter%%-%%subcategory%%.%%refine_hosts%%.[*][*][][]'
The JSON host list is structured in the following manner (I am unable to edit the structure of the host list, but it will always follow the following structure nonetheless):
{
"a-bc":{
"all":{
"webServer":[
],
"archive":[
"someHostAlias-123.privateDomain.com"
],
"central":[
"someHostAlias-456.privateDomain.com"
]
},
"QA":{
"xyz":{
"INBOUND_HTTP":[
"someHostAlias-789.privateDomain.com"
],
"WEBSITE":[
"someHostAlias-1011.privateDomain.com"
]
}
}
}
}
I have been using the following websites for this issue:
JMESPath Tutorial
Ansible JMESPath Documentation
JSONPath Expression Tester
StackOverflow: How to Use Variable in JMESPath Expression
Gitter: JMESPath/chat
I apologize if the query seems obvious, this is my first attempt at an Ansible Playbook. All help/feedback is greatly appreciated.
One of the issue of your query is that you are confusing [*] — a list projection — that selects all the elements of a list with .* — an object projection — that selects all the properties of a dictionary.
So, one solution in JMESPath, would be to do:
jmesquery: >-
"{{ datacenter }}-{{ subcategory }}".{{ refine_hosts }}.*.
{{ host_type if host_type | default('') != '' else '*' }}[] | []
Given the playbook:
- hosts: localhost
gather_facts: no
tasks:
- debug:
msg: "{{ jsondata | json_query(jmesquery) }}"
loop: "{{ fake_user_input }}"
loop_control:
label: "{{ jmesquery }}"
vars:
jmesquery: >-
"{{ datacenter }}-{{ subcategory }}".{{ refine_hosts }}.*.
{{ host_type if host_type | default('') != '' else '*' }}[] | []
datacenter: "{{ item.datacenter }}"
subcategory: "{{ item.subcategory }}"
refine_hosts: "{{ item.refine_hosts }}"
host_type: "{{ item.host_type | default('') }}"
fake_user_input:
- datacenter: a
subcategory: bc
refine_hosts: QA
host_type: WEBSITE
- datacenter: a
subcategory: bc
refine_hosts: QA
jsondata:
a-bc:
all:
webServer: []
archive:
- someHostAlias-123.privateDomain.com
central:
- someHostAlias-456.privateDomain.com
QA:
xyz:
INBOUND_HTTP:
- someHostAlias-789.privateDomain.com
WEBSITE:
- someHostAlias-1011.privateDomain.com
This yields:
ok: [localhost] => (item="a-bc".QA.*. WEBSITE[] | []) =>
msg:
- someHostAlias-1011.privateDomain.com
ok: [localhost] => (item="a-bc".QA.*. *[] | []) =>
msg:
- someHostAlias-789.privateDomain.com
- someHostAlias-1011.privateDomain.com

Extracting values from YAML into Jinja template for Ansible playbook

I have a YAML file with content as like below:
cat ../../ansible/playbooks/vars/patching-config.yml
---
patching_tag_name: "Patching"
my_windows_patching:
- {
OS: "WINDOWS",
tag_value: "myProdA",
frequency: "Month", #patching frequency. OneTime|Day|Hour|Week|Month|Minute
interval: 1, #interval of the schedule.
rebootSetting: "never", #ifRequired|never|always
PatchGroup: testA,
startDate: "2020-01-16T23:59:59Z",
expiryDate: "2020-02-16T23:59:59Z",
duration: "PT2H0M",
timeZone: "Australia/Sydney",
updateClassifications: "Critical,Important,Moderate"
}
I want to extract the values of updateClassifications from above YML file in Jinja Template file MaintenanceWindow.yml.j2
Resources:
WindowsNonProdBaseline:
Type: AWS::SSM::PatchBaseline
Properties:
Name: Windows-Non-Prod-Baseline
Description: Baseline containing all updates approved for Windows instances
OperatingSystem: {{ item.OS }}
PatchGroups:
- {{ item.PatchGroup }}
ApprovalRules:
PatchRules:
- PatchFilterGroup:
PatchFilters:
- Values:
# - Critical
# - Important
# - Moderate
{% for item in item.updateClassifications %}
- {{ item }}
{% endfor %}
I'm trying with the code described above, below one more time snippet:
{% for item in item.updateClassifications %}
- {{ item }}
{% endfor %}
I'm calling patching-config.yml in my tasks/main.yml as below
- include_vars: "{{playbook_dir}}/vars/patching-config.yml"
ignore_errors: yes
- name: create a cloudformation stack
cloudformation:
stack_name: "New-Ansible-cloudformation"
state: "present"
disable_rollback: true
template_body: "{{ lookup('template', '../../cloudformation/patching/MaintenanceWindow.yml.j2') }}"
with_items: "{{ telstra_windows_patching }}"
Finally, invoking role as below
cat ansible/playbooks/patching.yml
---
- hosts: localhost
roles:
- patching-cf-ssm
Unfortunately, it is not working.
Any lead shall be greatly appreciated.
Couple of things:
Your task is using telstra_windows_patching in with_items where as your variable file has variable name as my_windows_patching.
Assuming you are using the same name say my_windows_patching in your task and var file, if you are trying to save json object in yaml variable my_windows_patching you don't need - before curly braces. You can define something like this
my_windows_patching:
{
OS: "WINDOWS",
tag_value: "myProdA",
frequency: "Month", #patching frequency. OneTime|Day|Hour|Week|Month|Minute
interval: 1, #interval of the schedule.
rebootSetting: "never", #ifRequired|never|always
PatchGroup: testA,
startDate: "2020-01-16T23:59:59Z",
expiryDate: "2020-02-16T23:59:59Z",
duration: "PT2H0M",
timeZone: "Australia/Sydney",
updateClassifications: "Critical,Important,Moderate"
}
If you want to use elements inside my_windows_patching object with dot notation directly you could change the variable from object to a list something like,
my_windows_patching:
- OS: "WINDOWS"
tag_value: "myProdA"

Why is Ansible unable to read unicode string as JSON?

Summary
When retrieving data using the uri module in Ansible, I am unable to parse a section of it as JSON to retrieve a nested value.
The desired value is the ci field inside the content.data or json.data field (see output below).
Steps to Reproduce
site.yml
---
- hosts: localhost
gather_facts: false
tasks:
- name: Get String
uri:
url: "http://localhost/get-data"
method: POST
body_format: json
body: "{ \"kong-jid\": \"run-sn-discovery\" }"
return_content: yes
register: output
- set_fact:
ci: "{{ output.json.data.ci }}"
- debug:
msg: "{{ ci }}"
The {{ output }} variable
{
u'status': 200,
u'cookies': {},
u'url': u'http://kong-demo:8000/get-data',
u'transfer_encoding': u'chunked',
u'changed': False,
u'connection': u'close',
u'server': u'kong/0.34-1-enterprise-edition',
u'content':
u'{"data":"\\"{u\'ci\': u\'3bb8d625dbac3700e4f07b6e0f96195b\'}\\""}',
'failed': False,
u'json': {u'data': u'"{u\'ci\': u\'3bb8d625dbac3700e4f07b6e0f96195b\'}"'},
u'content_type': u'application/json',
u'date': u'Thu, 18 Apr 2019 15:50:25 GMT',
u'redirected': False,
u'msg': u'OK (unknown bytes)'
}
Result
[user#localhost]$ ansible-playbook site.yml
[WARNING]: Could not match supplied host pattern, ignoring: all
[WARNING]: provided hosts list is empty, only localhost is available
PLAY [localhost] ***************************************************************************************************************
TASK [Pass Redis data to next task as output] **********************************************************************************
ok: [localhost]
TASK [set_fact] ****************************************************************************************************************
fatal: [localhost]: FAILED! => {}
MSG:
The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'ci'
The error appears to have been in 'site.yml': line 19, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- set_fact:
^ here
exception type: <class 'ansible.errors.AnsibleUndefinedVariable'>
exception: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'ci'
Important Troubleshooting Information
It appears the root issue is related to which Ansible type being interpreted. I desire to parse ci from the output in one task.
The two-task solution shown below works, but this leads me to believe this should be possible in one line...
Two-Task Solution
- set_fact:
ci: "{{ output.json.data | from_json }}"
- debug:
msg: "{{ ci['ci'] }}"
But the ci fact set from {{ output.json.data | from_json }} reports a different TYPE than the inline type...
Unicode or Dict?
- debug:
msg: "{{ output.json.data | from_json | type_debug }}" # returns unicode
- set_fact:
ci: "{{ output.json.data | from_json }}"
- debug:
msg: "{{ ci | type_debug }}" # returns dict
Why isn't {{ output.json.data | from_json | type_debug }}
the same as {{ ci | type_debug }}?
Although json and data are keys in their resp objects, ci is just part of a larger string (which happens to look like a JSON object
If the relevant line in your datastructure would be:
u'json': {u'data': {'ci': u'3bb8d625dbac3700e4f07b6e0f96195b'}},
then you could expect to use "{{ output.json.data.ci }}" but not when the .ci part is just a normal part of a string.

Processing JSON from REST call in Ansible [duplicate]

I'm currently using Ansible 1.7.2. I have the following test playbook:
---
- hosts: localhost
tasks:
- name: set fact 1
set_fact: foo="[ 'zero' ]"
- name: set fact 2
set_fact: foo="{{ foo }} + [ 'one' ]"
- name: set fact 3
set_fact: foo="{{ foo }} + [ 'two', 'three' ]"
- name: set fact 4
set_fact: foo="{{ foo }} + [ '{{ item }}' ]"
with_items:
- four
- five
- six
- debug: var=foo
The first task sets a fact that's a list with one item in it. The subsequent tasks append to that list with more values. The first three tasks work as expected, but the last one doesn't. Here's the output when I run this:
PLAY [localhost] **************************************************************
GATHERING FACTS ***************************************************************
ok: [localhost]
TASK: [set fact 1] ************************************************************
ok: [localhost]
TASK: [set fact 2] ************************************************************
ok: [localhost]
TASK: [set fact 3] ************************************************************
ok: [localhost]
TASK: [set fact 4] ************************************************************
ok: [localhost] => (item=four)
ok: [localhost] => (item=five)
ok: [localhost] => (item=six)
TASK: [debug var=foo] *********************************************************
ok: [localhost] => {
"foo": [
"zero",
"one",
"two",
"three",
"six"
]
}
PLAY RECAP ********************************************************************
localhost : ok=6 changed=0 unreachable=0 failed=0
Given the with_items in task 4 and the fact that the output shows the task properly iterated over the items in that list, I would have expected the result to contain all the numbers zero through six. But that last task seems to only be evaluating set_fact with the last item in the list. Is this possibly a bug in Ansible?
Edit: I also just tested this on ansible 1.8 and the output was identical.
There is a workaround which may help. You may "register" results for each set_fact iteration and then map that results to list:
---
- hosts: localhost
tasks:
- name: set fact
set_fact: foo_item="{{ item }}"
with_items:
- four
- five
- six
register: foo_result
- name: make a list
set_fact: foo="{{ foo_result.results | map(attribute='ansible_facts.foo_item') | list }}"
- debug: var=foo
Output:
< TASK: debug var=foo >
---------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
ok: [localhost] => {
"var": {
"foo": [
"four",
"five",
"six"
]
}
}
As mentioned in other people's comments, the top solution given here was not working for me in Ansible 2.2, particularly when also using with_items.
It appears that OP's intended approach does work now with a slight change to the quoting of item.
- set_fact: something="{{ something + [ item ] }}"
with_items:
- one
- two
- three
And a longer example where I've handled the initial case of the list being undefined and added an optional when because that was also causing me grief:
- set_fact: something="{{ something|default([]) + [ item ] }}"
with_items:
- one
- two
- three
when: item.name in allowed_things.item_list
I was hunting around for an answer to this question. I found this helpful. The pattern wasn't apparent in the documentation for with_items.
https://github.com/ansible/ansible/issues/39389
- hosts: localhost
connection: local
gather_facts: no
tasks:
- name: set_fact
set_fact:
foo: "{{ foo }} + [ '{{ item }}' ]"
with_items:
- "one"
- "two"
- "three"
vars:
foo: []
- name: Print the var
debug:
var: foo
Jinja 2.6 does not have the map function. So an alternate way of doing this would be:
set_fact: foo="{% for i in bar_result.results %}{{ i.ansible_facts.foo_item }}{%endfor%}"
Below works for me:
- name: set fact
set_fact:
foo_item: "{{foo_item | default([]) + [item]}}"
loop:
- four
- five
- six
Updated 2018-06-08: My previous answer was a bit of hack so I have come back and looked at this again. This is a cleaner Jinja2 approach.
- name: Set fact 4
set_fact:
foo: "{% for i in foo_result.results %}{% do foo.append(i) %}{% endfor %}{{ foo }}"
I am adding this answer as current best answer for Ansible 2.2+ does not completely cover the original question. Thanks to Russ Huguley for your answer this got me headed in the right direction but it left me with a concatenated string not a list. This solution gets a list but becomes even more hacky. I hope this gets resolved in a cleaner manner.
- name: build foo_string
set_fact:
foo_string: "{% for i in foo_result.results %}{{ i.ansible_facts.foo_item }}{% if not loop.last %},{% endif %}{%endfor%}"
- name: set fact foo
set_fact:
foo: "{{ foo_string.split(',') }}"
Looks like this behavior is how Ansible currently works, although there is a lot of interest in fixing it to work as desired. There's currently a pull request with the desired functionality so hopefully this will get incorporated into Ansible eventually.

variable from loop in when statement ansible

I am trying to use a variable in a when statement and ansible pops up a warning like this
[WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ item.name}}.changed
I use a loop first:
- include_tasks: anaconda_env.yml
with_items: "{{anaconda_templates}}"
and in the anaconda_env file.yml i have this:
- name: anaconda.templates
template:
owner: "{{item.owner|default(common_owner)}}"
group: "{{item.group|default(common_group)}}"
mode: "{{item.mode|default(common_mode)}}"
src: "{{item.template}}"
dest: "{{item.dest}}"
register: "{{item.name}}"
- name: anaconda.handler
command: echo '1'
notify: "{{item.name}}"
when: "{{ item.name}}.changed"
And in another situation I tried "{{ item.name}}.rc == 1" and I have the same issue. Any idea how can I avoid this Wanring message.
I found the issue here but no solution
https://github.com/ansible/ansible/issues/27225
My original answer didn't work, but I believe the one below will (or at least did with my limited mock data):
- set_fact:
current_template: "{{item}}"
- name: anaconda.handler
command: echo '1'
notify: "{{item.name}}"
when: current_template.changed is defined