I have this complex variable:
vars:
- data_sources:
- source1:
- attrA: foo1
- attrB: bar1
- source2:
- attrA: foo2
- attrB: bar2
and I would like to loop over the structured variable with jinja2 to
generate some xml:
{% for d in {{data_sources}} %}
...
{% endfor %}
but I am getting error: expected token ':', got '}'", 'failed': True} for the very first line of the for loop. Any idea why and a solution? Thanks
UPADTE I made some corrections to the original example.
You must not use curly braces in jinja directives but only in the body:
{% for d in data_sources %}
{{ d['attrA'] }}
{% endfor %}
Also your data_sources is not what it meant to be:
vars:
data_sources:
- source1:
attrA: foo1
attrB: bar1
- source2:
attrA: foo2
attrB: bar2
Related
Data:
primaries:
ca:
- 10.51.60.45
- 10.51.60.46
ny:
- 10.52.60.45
- 10.52.60.46
az:
- 10.53.60.45
- 10.53.60.46
I want a flattened list of all IP's(or a for loop which can iterate through just the IP's), but the cities ca and ny and az could be anything.
Ansible's extract filter, which extracts the value of a key from a container, makes this very simple.
{{ primaries | map('extract', primaries) | flatten }}
You can also directly use the dictionary's values() method, which is slightly less flexible (the extract approach allows you to filter the keys beforehand, which you can't do here.)
{{ primaries.values() | flatten }}
You just need to iterate through the keys of the dictionary.
{% for region, ips in primaries.items() %}
{% for ip in ips %}
{{ ip }}
{% endfor %}
{% endfor %}
Read the Jinja docs on for.
Using Ansible, you can get a flattened list of ips using the json_query filter:
List of ip addresses:
{% for addr in primaries|json_query('*[][]') %}
- {{ addr }}
{% endfor %}
This results in:
List of ip addresses:
- 10.51.60.45
- 10.51.60.46
- 10.52.60.45
- 10.52.60.46
- 10.53.60.45
- 10.53.60.46
Here's a runnable example:
- hosts: localhost
gather_facts: false
vars:
primaries:
ca:
- 10.51.60.45
- 10.51.60.46
ny:
- 10.52.60.45
- 10.52.60.46
az:
- 10.53.60.45
- 10.53.60.46
tasks:
- copy:
dest: addresses.txt
content: |
List of ip addresses:
{% for addr in primaries|json_query('*[][]') %}
- {{ addr }}
{% endfor %}
The json_query filter uses the JMESPath query language.
hope to find you well and healthy :)
I have one jinja loop which is working, unfortunately not the way I would like :|
Short story:
Using salt-run for orchestration, minion targeting is accomplished by passing the pre-defined nodegroup as a pillar
{% set minions = salt.saltutil.runner('cache.mine', tgt=nodegroup,tgt_type='nodegroup').keys() %}
{% for minion_id in minions %}
patch-n1-{{ minion_id }}:
salt.state:
- tgt: {{ minion_id }}
- sls:
- patching.patch-n1
- pillar:
minion_id: {{ minion_id }}
reboot_minion-{{ minion_id }}:
salt.function:
- name: cmd.run_bg
- arg:
- 'salt-call system.reboot 1'
- tgt: {{ minion_id }}
{% endfor %}
The problem is that with this loop both tasks are executed minion by minion. In my case, this is not efficient ...
If remove the loop both states are applied however again doesn't help so much.
Main goal is to apply patch-n1-{{ minion_id }} and reboot_minion-{{ minion_id }} for each minion in the nodegroup indipended from each other.
Or said in a different way I need a for loop which to work simultaneously for all minions in it.
Do you have any ideas about that?
Thanks!
When we target minions by globbing, or nodegroups, the defined states will be applied to them in parallel. So one way to achieve this is by moving the "reboot minion" functionality into the state file patch-n1.sls itself.
Example /srv/salt/patch-n1.sls file:
# Some tasks to perform patching, just using 'include' for example
# from /srv/salt/patching/os_pkg.sls
include:
- patching.os_pkg
reboot-after-package-update:
module.run:
- name: system.reboot
And in orchestrate /srv/salt/orch/patch_all.sls file:
patch-group1:
salt.state:
- tgt: group1
- tgt_type: nodegroup
- sls:
- patch-n1
patch-group2:
salt.state:
- tgt: group2
- tgt_type: nodegroup
- sls:
- patch-n2
When we run the orchestration, the each minion will run patching and reboot in parallel.
system.reboot didn't help at my case bacause my minions are mixed (Windows, Linux)
reboot-after-package-update:
module.run:
- name: system.reboot
I was able to achieve my goal by:
# Some tasks to perform patching, just using 'include' for example
# from /srv/salt/patching/uptodate.sls
include:
- patching.uptodate
reboot-after-package-update:
cmd.run:
- name: shutdown -r -t 60
{% set stage = pillar['stage'] %}
{% set nodegroup = salt['pillar.get']('nodegroup', 'PILLAR nodegroup NOT FOUND!') %}
{% set minions = salt.saltutil.runner('cache.mine', tgt=nodegroup,tgt_type='nodegroup').keys() %}
wait_for_reboot-{{ stage }}:
salt.wait_for_event:
- name: salt/minion/*/start
- id_list: {% for minion_id in minions %}
- {{minion_id}}{% endfor %}
- timeout: 6000
- require:
- cmd: reboot-after-package-update
In my Jekyll site, I have a page that stores an array of data in front matter, like this:
---
layout: page
title: MyTitle
array:
- key1: value1
- key2: value2
---
What I want to do in my template: given a keyX, obtain valueX from the array.
I figured out a way to access the array:
{% assign subpage = site.pages | where: 'title', 'MyTitle' %}
{% assign array = subpage[0].array %}
Now the query that I need to write is: "from the array, extract the value that matches keyX".
Is there a way to search through the array, without the need for looping? All the examples I can find are based on one-dimensional arrays...
Your array is an array of non standardized objects (they don't have the same keys).
{{ page.array | inspect }}
returns
[{"key1"=>"value1"}, {"key2"=>"value2"}]
Here the only way to search is to loop over all array's items.
If you refactor your array to become an object, you can then get value from a key.
---
[...]
object:
key1: value1
key2: value2
...
Example :
{% assign searched = "key1" %}
{{ page.object[searched] }}
I found this workaround in the meantime:
{% for valueList in array %}
{% for valuePair in valueList %}
{% if valuePair[0] == "key1" %}
{% assign value = valuePair[1] %}
{% endif %}
{% endfor %}
{% endfor %}
Jinja2 supports very useful filters to modify a string, eg.
{{ my_string|capitalize }}
What about you want to build the input string? When the string is simple you can always use
{% set my_string = string_1 + string_2 %}
{{ my_string|capitalize }}
But it would be wonderful to actually build this string using templates, just like
{% set my_string = "{{ 'a' }}b{{ 'c' }}" %}
{{ my_string|capitalize }}
that would output Abc..
Did I miss something?
Answer exists in Jinja 2.8, as documented here
The answer is
{% set my_string %}
{{ 'a'}}b{{ 'c' }}
{% endset %}
Trying to use Google Deployment Manager with YAML and Jinja with a multi-line variables, such as:
startup_script_passed_as_variable: |
line 1
line 2
line 3
And later:
{% if 'startup_script_passed_as_variable' in properties %}
- key: startup-script
value: {{properties['startup_script_passed_as_variable'] }}
{% endif %}
Gives MANIFEST_EXPANSION_USER_ERROR:
ERROR: (gcloud.deployment-manager.deployments.create) Error in
Operation operation-1432566282260-52e8eed22aa20-e6892512-baf7134:
MANIFEST_EXPANSION_USER_ERROR
Manifest expansion encountered the following errors: while scanning a simple key in "" could not found expected ':' in ""
Tried (and failed):
{% if 'startup_script' in properties %}
- key: startup-script
value: {{ startup_script_passed_as_variable }}
{% endif %}
also
{% if 'startup_script' in properties %}
- key: startup-script
value: |
{{ startup_script_passed_as_variable }}
{% endif %}
and
{% if 'startup_script' in properties %}
- key: startup-script
value: |
{{ startup_script_passed_as_variable|indent(12) }}
{% endif %}
The problem is the combination of YAML and Jinja. Jinja escapes the variable but fails to indent it as YAML would require when being passed as a variable.
Related: https://github.com/saltstack/salt/issues/5480
Solution: Pass the multi-line variable as an array
startup_script_passed_as_variable:
- "line 1"
- "line 2"
- "line 3"
The quoting is important if your value starts with # (which startup script on GCE does, ie #!/bin/bash) since it will be treated as a comment otherwise.
{% if 'startup_script' in properties %}
- key: startup-script
value:
{% for line in properties['startup_script'] %}
{{line}}
{% endfor %}
{% endif %}
Putting it here since there aren't much Q&A material for Google Deployment manager.
There is no clean way to do this in Jinja. As you yourself have pointed out, because YAML is a whitespace-sensitive language, it is difficult to effectively template around.
One possible hack is to split the string property into a list and then iterate over the list.
For example, providing the property:
startup-script: |
#!/bin/bash
python -m SimpleHTTPServer 8080
you can use it in your Jinja template:
{% if 'startup_script' in properties %}
- key: startup-script
value: |
{% for line in properties['startup-script'].split('\n') %}
{{ line }}
{% endfor %}
Here is also a full working example of this.
This method will work, but generally cases like this are when people start considering using a python template. Because you are working with an object model in python, you do not have to deal with indentation problems. For example:
'metadata': {
'items': [{
'key': 'startup-script',
'value': context.properties['startup_script']
}]
}
An example of the python template can be found in the Metadata From File example.