Extracting values from YAML into Jinja template for Ansible playbook - jinja2

I have a YAML file with content as like below:
cat ../../ansible/playbooks/vars/patching-config.yml
---
patching_tag_name: "Patching"
my_windows_patching:
- {
OS: "WINDOWS",
tag_value: "myProdA",
frequency: "Month", #patching frequency. OneTime|Day|Hour|Week|Month|Minute
interval: 1, #interval of the schedule.
rebootSetting: "never", #ifRequired|never|always
PatchGroup: testA,
startDate: "2020-01-16T23:59:59Z",
expiryDate: "2020-02-16T23:59:59Z",
duration: "PT2H0M",
timeZone: "Australia/Sydney",
updateClassifications: "Critical,Important,Moderate"
}
I want to extract the values of updateClassifications from above YML file in Jinja Template file MaintenanceWindow.yml.j2
Resources:
WindowsNonProdBaseline:
Type: AWS::SSM::PatchBaseline
Properties:
Name: Windows-Non-Prod-Baseline
Description: Baseline containing all updates approved for Windows instances
OperatingSystem: {{ item.OS }}
PatchGroups:
- {{ item.PatchGroup }}
ApprovalRules:
PatchRules:
- PatchFilterGroup:
PatchFilters:
- Values:
# - Critical
# - Important
# - Moderate
{% for item in item.updateClassifications %}
- {{ item }}
{% endfor %}
I'm trying with the code described above, below one more time snippet:
{% for item in item.updateClassifications %}
- {{ item }}
{% endfor %}
I'm calling patching-config.yml in my tasks/main.yml as below
- include_vars: "{{playbook_dir}}/vars/patching-config.yml"
ignore_errors: yes
- name: create a cloudformation stack
cloudformation:
stack_name: "New-Ansible-cloudformation"
state: "present"
disable_rollback: true
template_body: "{{ lookup('template', '../../cloudformation/patching/MaintenanceWindow.yml.j2') }}"
with_items: "{{ telstra_windows_patching }}"
Finally, invoking role as below
cat ansible/playbooks/patching.yml
---
- hosts: localhost
roles:
- patching-cf-ssm
Unfortunately, it is not working.
Any lead shall be greatly appreciated.

Couple of things:
Your task is using telstra_windows_patching in with_items where as your variable file has variable name as my_windows_patching.
Assuming you are using the same name say my_windows_patching in your task and var file, if you are trying to save json object in yaml variable my_windows_patching you don't need - before curly braces. You can define something like this
my_windows_patching:
{
OS: "WINDOWS",
tag_value: "myProdA",
frequency: "Month", #patching frequency. OneTime|Day|Hour|Week|Month|Minute
interval: 1, #interval of the schedule.
rebootSetting: "never", #ifRequired|never|always
PatchGroup: testA,
startDate: "2020-01-16T23:59:59Z",
expiryDate: "2020-02-16T23:59:59Z",
duration: "PT2H0M",
timeZone: "Australia/Sydney",
updateClassifications: "Critical,Important,Moderate"
}
If you want to use elements inside my_windows_patching object with dot notation directly you could change the variable from object to a list something like,
my_windows_patching:
- OS: "WINDOWS"
tag_value: "myProdA"

Related

Ansible Error when using dict with csv: "The task includes an option with an undefined variable. The error was 'dict object' has no attribute 'schema'

I'm trying to get my playbook working with csv while using a dict due to the nature of the data [each task asks for different parts of the data in a row so i can't use a list?].
BD and EPG always will appear exactly once per file and can be used as a key if necessary.
I'm getting an error "the task includes an option with an undefined variable". Since the variable (schema) appears as a column header in the csv file I must have some sort of syntax issue.
What I am trying to do is loop, one row at a time, through the csv file and have schema: "{{ item.schema }}" evaluate to that particular rows value under the schema column, etc.
Playbook:
tasks:
- name: Read CSV
read_csv:
# Name of the csv
path: ./Create_EPGs_and_BDs.csv
dialect: excel
key: bd
# Creates register value to be used later
register: csv_data
#
##################################################
#Creates BD in MSO Template
##################################################
#
- name: Add a new BD
cisco.mso.mso_schema_template_bd:
<<: *aci_login
state: present
schema: "{{ item.schema }}"
template: "{{ item.template }}"
bd: "{{ item.bd }}"
vrf:
name: "{{ item.vrf }}"
loop: "{{ csv_data.dict|dict2items }}"
#
##################################################
#Creates EPG in MSO Template
##################################################
#
- name: Add a new EPG
cisco.mso.mso_schema_template_anp_epg:
<<: *aci_login
state: present
schema: "{{ item.schema }}"
template: "{{ item.template }}"
anp: "{{ item.app_profile }}"
epg: "{{ item.epg }}"
bd:
name: "{{ item.bd }}"
loop: "{{ csv_data.dict|dict2items }}"
CSV File:
https://i.stack.imgur.com/VlyLP.jpg
What is the correct syntax to pull the csv values for the entire row and then let me access each column's value in the playbook for that particular row?
The dict2items is going to transpose your dict into a list of dicts, shaped like [{"key": "the-value-of-bd-for-that-row", "value": {"schema": "schema1", ...}}, ...]
Thus, you just need to add .value in between your item and the dict key it references:
- name: Add a new BD
cisco.mso.mso_schema_template_bd:
<<: *aci_login
state: present
schema: "{{ item.value.schema }}"
template: "{{ item.value.template }}"
# or if prefer, item.value.bd
bd: "{{ item.key }}"
vrf:
name: "{{ item.value.vrf }}"
loop: "{{ csv_data.dict|dict2items }}"
it's your code style, but you'll want to be very careful using only one space for yaml indentation, as it's very easy to make a mistake doing that, and certainly makes asking questions on SO harder by doing so :-)
In the future, the use of debug: var=item will go a long way toward helping you understand the shape of your data when you encounter any such "has no attribute" error

Insert multiline json string into helm template for base64 encoding

I am trying to insert multiline json string into helm template for base64 encoding required for Kubernetes secret.
Goals:
helm value is injected into json string
multi-line json string must be base64 encoded using b64enc
myfile1.json does not work but myfile2.json works.
I prefer not to put entire json file in values.yaml.
apiVersion: v1
kind: Secret
metadata:
name: {{ template "mychart.fullname" . }}
labels:
app: {{ template "mychart.name" . }}
chart: {{ template "mychart.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: Opaque
data:
myfile.json: {{ |-
{
"item1": {
"name": "{{ .Values.item1.name }}"
},
"item2": {
}
} | b64enc }}
myfile2.json: {{ .Values.myfile2 | b64enc }}
You actually don't need to base64-encode the secret in the helm chart. If you use the stringData field instead of data field, Kubernetes knows that it needs to base64 encode the data upon the secret's deployment.
From the docs (Source):
The Secret contains two maps: data and stringData. The data field is used to store arbitrary data, encoded using base64. The stringData field is provided for convenience, and allows you to provide secret data as unencoded strings.
So we can rewrite your secret using stringData instead of data and keep multiline json strings in templates like so:
apiVersion: "v1"
kind: "Secret"
metadata:
name: {{ template "mychart.fullname" . }}
labels:
app: {{ template "mychart.name" . }}
chart: {{ template "mychart.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: "Opaque"
stringData:
myfile.json: |-
{
"item1": {
"name": "{{ .Values.item1.name }}"
},
"item2": {
}
}
myfile2.json: {{ .Values.myfile2 }}
Note that this does not mean you suddenly need to worry about having unencoded secrets. stringData will ultimately be base64-encoded and converted to data when it is installed, so it will behave exactly the same once it's loaded into Kubernetes.
Again, from the docs (emphasis mine) (Source):
stringData allows specifying non-binary secret data in string form. It is provided as a write-only convenience method. All keys and values are merged into the data field on write, overwriting any existing values. It is never output when reading from the API.
My impression (and others seem to have hit it too) is you have to compromise either on it being multi-line or on not putting it in a file. I think the problem is that you have to use a yaml instruction (the |-) to get multiple lines and that is part of the template itself so you can't get an 'output' from it in a way that you can then feed into b64enc.
If this were a ConfigMap you wouldn't need to feed into b64enc so it would be as simple as:
myfile.json: |
{
"item1": {
"name": "{{ .Values.item1.name }}"
},
"item2": {
}
}
Or if you were to compromise on single-line approach then that could be:
myfile.json: {{ tpl ("{ 'item1': { 'name': '{{ .Values.item1.name }}' }, 'item2': { } }") . | toJson | b64enc }}
If it were coming from a file then you could use {{ tpl (.Files.Get "files/myfile.json") . | b64enc | quote }}
Another option would be to put the whole json in the values file
Or you could have a myfile entry in your values file like:
myfile:
item1:
name: "bob"
item2:
name: "fred"
And then use it with myfile.json: {{ .Values.myfile | toJson | b64enc }}
I found a solution. You can use tpl function on json file to render template.
apiVersion: v1
kind: Secret
metadata:
name: {{ template "mychart.fullname" . }}
labels:
app: {{ template "mychart.name" . }}
chart: {{ template "mychart.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: Opaque
data:
myfile.json: {{ tpl(.Files.Get "myfile.json") . | b64enc }}
myfile.json
{
"item1": {
"name": "{{ .Values.item1.name }}"
},
"item2": {
}
}

variable from loop in when statement ansible

I am trying to use a variable in a when statement and ansible pops up a warning like this
[WARNING]: when statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found: {{ item.name}}.changed
I use a loop first:
- include_tasks: anaconda_env.yml
with_items: "{{anaconda_templates}}"
and in the anaconda_env file.yml i have this:
- name: anaconda.templates
template:
owner: "{{item.owner|default(common_owner)}}"
group: "{{item.group|default(common_group)}}"
mode: "{{item.mode|default(common_mode)}}"
src: "{{item.template}}"
dest: "{{item.dest}}"
register: "{{item.name}}"
- name: anaconda.handler
command: echo '1'
notify: "{{item.name}}"
when: "{{ item.name}}.changed"
And in another situation I tried "{{ item.name}}.rc == 1" and I have the same issue. Any idea how can I avoid this Wanring message.
I found the issue here but no solution
https://github.com/ansible/ansible/issues/27225
My original answer didn't work, but I believe the one below will (or at least did with my limited mock data):
- set_fact:
current_template: "{{item}}"
- name: anaconda.handler
command: echo '1'
notify: "{{item.name}}"
when: current_template.changed is defined

how it I parse each row of data from the google sheets api with ansible's with_items?

I am using the Google Sheets V4 Values collection and I am having trouble figuring out how to get each row to parse in to an {{ item }}
My Ansible ymal looks like.
tasks:
- name: debug url
debug:
msg: "{{ g_url }}"
- name: GET data from google spead sheet api
uri:
url: "{{ g_url }}"
return_content: yes
dest: /tmp/o_gd_form.json
register: google_data
- name: whats the dump?
debug:
var: "{{ item.0 |to_json }}"
with_items: "{{ google_data.json.values() }}" # this is the line that needs to be fixed
And the the responding json looks like:
{
"range": "Sheet1!A1:D5",
"majorDimension": "ROWS",
"values": [
["Item", "Cost", "Stocked", "Ship Date"],
["Wheel", "$20.50", "4", "3/1/2016"],
["Door", "$15", "2", "3/15/2016"],
["Engine", "$100", "1", "30/20/2016"],
["Totals", "$135.5", "7", "3/20/2016"]
],
}
I can't seem figure out how to write the the with_items to return an {{ item }} of json like ["Engine", "$100", "1", "30/20/2016"].
any help or do I need to split this task out to some middleware parser?
The Google sheets api documentation is at:
https://developers.google.com/sheets/samples/reading
To get what you want, use:
- debug: msg="Name={{ item.0 }} Cost={{ item.1 }}"
with_list: "{{ google_data.json['values'] }}"
There are two problems in your code:
values is a special keyword, so you can't access json.values, but should use json['values'] instead.
with_items flattens the list of lists, so you end up with long list of strings. You should use with_list to iterate over outer list (rows) and get inner lists (column values) as items.

Jinja2 filter list using string contains test

I'm trying to filter a list in ansible in Jinja2 when the elements contain a string, but the Jinja documentation doesn't seem clear enough for me to figure it out.
This is what I have so far:
- name: run script
command: /usr/tmp/run_script.py
register: script_results
- name: display run info
debug:
var: "{{script_results.stdout_lines | select(\"'running script' in script_results.stdout_lines\") }}"
But all I get is the error:
"<generator object _select_or_reject at 0x13851e0>": "VARIABLE IS NOT DEFINED!"
So for example, if stdout_lines contains ["apples","running script one","oranges","running script two"], I want to print
running script one
running script two
They have documentation for select and documentation for built-in-tests, but they don't display the "in" test, and I don't know how they work in the context of this ansible variable.
I tried solving it like this:
- name: display run info
debug:
var: item
with_items: "{{script_results.stdout_lines}}"
when: "'running script' in item"
But that displays "skipping" for every line that doesn't pass the test ... kinda defeating the purpose!
The select filter would take another filter. Like in the docs odd, which will return only the odd elements of the list. The filter you would like to combine select with is equalto.
Now here's the thing. Ansible bundles a very old version of Jinja2, which simply does not contain the equalto filter. Yes, that renders it useless unless you want to filter odd elements. (Which nobody ever in history wanted to...)
Furthermore I was yet unable to make custom filter plugins work in Ansible 2. So you're pretty much forced to hack something ugly together.
helloV already showed one option. Here is another idea:
- name: run script
shell: /usr/tmp/run_script.py | grep "running script"
register: script_results
Update:
I recently discovered you can use match (not a standard Jinja2 filter but added by Ansible) together with select. Thats a good replacement for the eualto filter plus you can use regular expressions. This should work:
{{ script_results.stdout_lines | select("match", ".*running script.*") }}
I understand there may be more than one way to do this. Will this work for you?
- debug: var={{item}}
when: item.find('running script') > -1
with_items: script_results.stdout_lines
I know this is an old thread but I too was looking for an answer to this and I was able to use the if in method. I think the other issue you were having was how to display the resulting list. You can do a jinja loop right in the ansible modules, debug for example. Of course since Ansible is horrible with presenting data using debug, you could also utilize the blockinfile module with the pipe (|). Hopefully this helps others and gives yet another option.
- debug:
msg: |
{% for item in (script_results.stdout_lines) %}
{% if 'running script' in item %}
{{ item }}
{% endif %}
{% endfor %}
tags: debug
or to add the filtered data to a file:
- name: Update the scriptStatus file.
delegate_to: localhost
run_once: TRUE
blockinfile:
path: '/tmp/scriptStatus.txt'
block: |
{% for item in (script_results.stdout_lines) %}
{% if 'running script' in item %}
{{ item }}
{% endif %}
{% endfor %}
tags: chkScripts
I ended up writing a python script to do it, because I couldn't get ansible or ancient-jinja2 to make the cut.
Ansible tasks:
- name: gather run info
command: "{{role_path}}/files/print_results.py {{script_results.stdout_lines}}"
register: script_print_results
delegate_to: 127.0.0.1
run_once: true
- name: display run info
debug:
var: script_print_results.stdout_lines
delegate_to: 127.0.0.1
run_once: true
Python script:
for result_line in sys.argv[1:]:
if "running script:" in result_line:
print result_line[1:-1]
You can build a new list with set_fact and print the elements of a new list.
- hosts: localhost
gather_facts: false
vars:
script_stdout_lines:
- apples
- running script one
- oranges
- running script two
tasks:
- set_fact:
new_list: "{{ new_list | default([]) + [item] }}"
with_items: "{{ script_stdout_lines }}"
when: '"running script" in item'
- debug: var=new_list
Result:
TASK [set_fact] *********************************************************************************************************************
skipping: [localhost] => (item=apples)
ok: [localhost] => (item=running script one)
skipping: [localhost] => (item=oranges)
ok: [localhost] => (item=running script two)
TASK [debug] ************************************************************************************************************************
ok: [localhost] => {
"new_list": [
"running script one",
"running script two"
]
}
It prints skipping during set_fact operation but at the end it provides a new list with the only matching items.