How do I use json variables in a yaml file (Helm) - json

I have a HELM values file which looks like so:
service:
environment: dev
spring_application_json: >-
{
"spring" : {
"boot" : {
"admin" : {
"client" : {
"enabled" : "false",
"url" : "http://website1",
"instance" : {
"service-base-url" : "http://website2",
"management-base-url" : "http://website3"
}
}
}
}
}
}
And a corresponding template file which grabs this value and inserts it as an environment variable to a container.
spec:
replicas: {{ .Values.replicaCount }}
template:
spec:
imagePullSecrets:
- name: {{ .Values.image.pullSecret }}
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
env:
- name: ENVIRONMENT
value: "{{ .Values.service.environment }}"
- name: SPRING_APPLICATION_JSON
value: "{{ .Values.service.spring_application_json }}"
However when I run the helm install I get the following error:
Error: YAML parse error on deployment.yaml: error converting YAML to JSON: yaml: line 40: did not find expected key
Which points to the line:
value: "{{ .Values.service.spring_application_json }}"
I believe its a problem with the way I'm trying to parse in a json string as a multiline environment variable? The ENVIRONMENT 'dev' variable works perfectly and this same YAML also works perfectly with docker-compose.

There's an example a bit like this in the docs for spring cloud dataflow but the format in their documentation has the quotes escaped.
I was able to recreate the error and get past it by changing the values file entry to:
service:
spring_application_json:
{
"spring" : {
"boot" : {
"admin" : {
"client" : {
"enabled" : "false",
"url" : "http://website1",
"instance" : {
"service-base-url" : "http://website2",
"management-base-url" : "http://website3"
}
}
}
}
}
}
And the deployment entry to:
- name: SPRING_APPLICATION_JSON
value: {{ .Values.service.spring_application_json | toJson | quote }}
Notice no quotes around this part as that is handled anyway.

Related

Setting fact from json with hyphens

I have a json coming from an API like this -
{
"Clusters": {
"cluster_name": "cluster1",
"desired_configs": {
"ams-env": {
"tag": "15646576543547354",
"version": 2
},
"ams-grafana-env": {
"tag": "156765743275788",
"version": 2
},
"ams-grafana-ini": {
"tag": "987657435754385457",
"version": 2
}
}
}
}
And I need to parse it with ansible. The trouble is, the variable that will be passed is the part with the hyphens.
I'm able to print the tag name with debug: var but I cant turn it into a fact and I also cant make it print when I use debug: msg
This is the play - I would like to take the "tag" for whichever config_name is passed at runtime and create a new var to be passed into later tasks
- name: Parsing Json
hosts: localhost
connection: local
tags: setup_infra
vars:
- config_name: ams-env
tasks:
- name: access fact
set_fact:
access_auths: "{{ lookup('file', 'ambari.json') | from_json }}"
- name: This works
debug:
var: access_auths.Clusters.desired_configs['{{ config_name }}'].tag
- name: This does not work
set_fact:
new_config: "{{ access_auths.Clusters.desired_configs['{{ config_name }}'].tag }}"
- name: Debug 0.3
debug:
var: new_config
Thanks in advance for any help
This issue is happening because of the nesting of {{ jinja delimiters. The config_name var you set is already a text "ams-env", so we don't need to quote it again as well.
The following tasks should work:
- debug:
msg: "tag is {{ access_auths['Clusters']['desired_configs'][config_name]['tag'] }}"
- set_fact:
new_config: "{{ access_auths['Clusters']['desired_configs'][config_name]['tag'] }}"

Ansible pass json in varibale in ansible-playbook --extra-vars=

i try to pass JSON structure into ansible-playbook without success:
this is the command I try to pass
ansible-playbook --extra-vars='[{\"${foo1}\": \"somevalue1\", \"${foo2}\": \"somevalue2\"}, {\"${zoo1}\": \"somevalue111\", \"${zoo2}\": \"somevalue222\"}]' test.yml
getting error:
ERROR! Syntax Error while loading YAML.
expected ',' or '}', but got '{'
or this :
ansible-playbook --extra-vars='[{"${foo1}":"somevalue1","${foo2}":"somevalue2"},{"${zoo1}":"somevalue111","${zoo2}":"somevalue222"}]' test.yml
Getting no output
The ideal way is passing the JSON into variable like this so i could iterate the json array in ansible :
ansible-playbook --extra-vars="AAA='[{\"${foo1}\": \"somevalue1\", \"${foo2}\": \"somevalue2\"}, {\"${zoo1}\": \"somevalue111\", \"${zoo2}\": \"somevalue222\"}]'" test.yml
with this playbook :
---
-
gather_facts: false
hosts: localhost
name: test
tasks:
- name: debug
debug:
msg: "{{ AAA }}"
The output is :
ok: [localhost] =>
msg:
- ? ''
: somevalue2
- ? ''
: somevalue222
In short, what is the best way to pass JSON structure into ansible without using the file?
I don't follow why you're escaping the " inside of a ' string, but you will want to switch away from the KEY=VALUE syntax because in that form, ansible splits on whitespace -- by leading with the { it informs ansible that the --extrav-vars is in fact JSON and stops using the key-value parser
ansible -e '{"AAA": [{"hello":{"world": true}}, {"array":{"yup":"forsure"}}]}' -m debug -a var=AAA localhost
produces
localhost | SUCCESS => {
"AAA": [
{
"hello": {
"world": true
}
},
{
"array": {
"yup": "forsure"
}
}
]
}

Extracting values from YAML into Jinja template for Ansible playbook

I have a YAML file with content as like below:
cat ../../ansible/playbooks/vars/patching-config.yml
---
patching_tag_name: "Patching"
my_windows_patching:
- {
OS: "WINDOWS",
tag_value: "myProdA",
frequency: "Month", #patching frequency. OneTime|Day|Hour|Week|Month|Minute
interval: 1, #interval of the schedule.
rebootSetting: "never", #ifRequired|never|always
PatchGroup: testA,
startDate: "2020-01-16T23:59:59Z",
expiryDate: "2020-02-16T23:59:59Z",
duration: "PT2H0M",
timeZone: "Australia/Sydney",
updateClassifications: "Critical,Important,Moderate"
}
I want to extract the values of updateClassifications from above YML file in Jinja Template file MaintenanceWindow.yml.j2
Resources:
WindowsNonProdBaseline:
Type: AWS::SSM::PatchBaseline
Properties:
Name: Windows-Non-Prod-Baseline
Description: Baseline containing all updates approved for Windows instances
OperatingSystem: {{ item.OS }}
PatchGroups:
- {{ item.PatchGroup }}
ApprovalRules:
PatchRules:
- PatchFilterGroup:
PatchFilters:
- Values:
# - Critical
# - Important
# - Moderate
{% for item in item.updateClassifications %}
- {{ item }}
{% endfor %}
I'm trying with the code described above, below one more time snippet:
{% for item in item.updateClassifications %}
- {{ item }}
{% endfor %}
I'm calling patching-config.yml in my tasks/main.yml as below
- include_vars: "{{playbook_dir}}/vars/patching-config.yml"
ignore_errors: yes
- name: create a cloudformation stack
cloudformation:
stack_name: "New-Ansible-cloudformation"
state: "present"
disable_rollback: true
template_body: "{{ lookup('template', '../../cloudformation/patching/MaintenanceWindow.yml.j2') }}"
with_items: "{{ telstra_windows_patching }}"
Finally, invoking role as below
cat ansible/playbooks/patching.yml
---
- hosts: localhost
roles:
- patching-cf-ssm
Unfortunately, it is not working.
Any lead shall be greatly appreciated.
Couple of things:
Your task is using telstra_windows_patching in with_items where as your variable file has variable name as my_windows_patching.
Assuming you are using the same name say my_windows_patching in your task and var file, if you are trying to save json object in yaml variable my_windows_patching you don't need - before curly braces. You can define something like this
my_windows_patching:
{
OS: "WINDOWS",
tag_value: "myProdA",
frequency: "Month", #patching frequency. OneTime|Day|Hour|Week|Month|Minute
interval: 1, #interval of the schedule.
rebootSetting: "never", #ifRequired|never|always
PatchGroup: testA,
startDate: "2020-01-16T23:59:59Z",
expiryDate: "2020-02-16T23:59:59Z",
duration: "PT2H0M",
timeZone: "Australia/Sydney",
updateClassifications: "Critical,Important,Moderate"
}
If you want to use elements inside my_windows_patching object with dot notation directly you could change the variable from object to a list something like,
my_windows_patching:
- OS: "WINDOWS"
tag_value: "myProdA"

Insert multiline json string into helm template for base64 encoding

I am trying to insert multiline json string into helm template for base64 encoding required for Kubernetes secret.
Goals:
helm value is injected into json string
multi-line json string must be base64 encoded using b64enc
myfile1.json does not work but myfile2.json works.
I prefer not to put entire json file in values.yaml.
apiVersion: v1
kind: Secret
metadata:
name: {{ template "mychart.fullname" . }}
labels:
app: {{ template "mychart.name" . }}
chart: {{ template "mychart.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: Opaque
data:
myfile.json: {{ |-
{
"item1": {
"name": "{{ .Values.item1.name }}"
},
"item2": {
}
} | b64enc }}
myfile2.json: {{ .Values.myfile2 | b64enc }}
You actually don't need to base64-encode the secret in the helm chart. If you use the stringData field instead of data field, Kubernetes knows that it needs to base64 encode the data upon the secret's deployment.
From the docs (Source):
The Secret contains two maps: data and stringData. The data field is used to store arbitrary data, encoded using base64. The stringData field is provided for convenience, and allows you to provide secret data as unencoded strings.
So we can rewrite your secret using stringData instead of data and keep multiline json strings in templates like so:
apiVersion: "v1"
kind: "Secret"
metadata:
name: {{ template "mychart.fullname" . }}
labels:
app: {{ template "mychart.name" . }}
chart: {{ template "mychart.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: "Opaque"
stringData:
myfile.json: |-
{
"item1": {
"name": "{{ .Values.item1.name }}"
},
"item2": {
}
}
myfile2.json: {{ .Values.myfile2 }}
Note that this does not mean you suddenly need to worry about having unencoded secrets. stringData will ultimately be base64-encoded and converted to data when it is installed, so it will behave exactly the same once it's loaded into Kubernetes.
Again, from the docs (emphasis mine) (Source):
stringData allows specifying non-binary secret data in string form. It is provided as a write-only convenience method. All keys and values are merged into the data field on write, overwriting any existing values. It is never output when reading from the API.
My impression (and others seem to have hit it too) is you have to compromise either on it being multi-line or on not putting it in a file. I think the problem is that you have to use a yaml instruction (the |-) to get multiple lines and that is part of the template itself so you can't get an 'output' from it in a way that you can then feed into b64enc.
If this were a ConfigMap you wouldn't need to feed into b64enc so it would be as simple as:
myfile.json: |
{
"item1": {
"name": "{{ .Values.item1.name }}"
},
"item2": {
}
}
Or if you were to compromise on single-line approach then that could be:
myfile.json: {{ tpl ("{ 'item1': { 'name': '{{ .Values.item1.name }}' }, 'item2': { } }") . | toJson | b64enc }}
If it were coming from a file then you could use {{ tpl (.Files.Get "files/myfile.json") . | b64enc | quote }}
Another option would be to put the whole json in the values file
Or you could have a myfile entry in your values file like:
myfile:
item1:
name: "bob"
item2:
name: "fred"
And then use it with myfile.json: {{ .Values.myfile | toJson | b64enc }}
I found a solution. You can use tpl function on json file to render template.
apiVersion: v1
kind: Secret
metadata:
name: {{ template "mychart.fullname" . }}
labels:
app: {{ template "mychart.name" . }}
chart: {{ template "mychart.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
type: Opaque
data:
myfile.json: {{ tpl(.Files.Get "myfile.json") . | b64enc }}
myfile.json
{
"item1": {
"name": "{{ .Values.item1.name }}"
},
"item2": {
}
}

Ansible Extract JSON Tag

I'm trying to work with Infoblox API, and it's responses. I would need to extract values of tags from the response, that seems to be in JSON format, but I cannot find the way to do it.
Here is my playbook:
- name: "Checking _node_exporter Service Record for {{ inventory_hostname }}"
local_action:
module: uri
url: "{{ infobloxapiurl }}record:srv?name=_node_exporter.domain.com&target={{ inventory_hostname }}"
force_basic_auth: yes
user: "{{ infobloxuser }}"
password: "{{ infobloxpassword }}"
validate_certs: no
return_content: yes
register: _infoblox_results
- debug:
var: _infoblox_results.json
The _infoblox_results.json variable looks like this:
TASK [prometheus : debug] *******************************************************************************************************************************************************************************************
task path: /ansible/roles/tasks/task.yml:38
ok: [server.domain.com] => {
"_infoblox_results.json": [
{
"_ref": "record:srv/ZG5zLmJpbmRfc3J2JC5fZGVmYXVsdC5jb20udmNpbnQuZXcxL19ub2RlX2V4cG9ydGVyLzAvMC85MTAwL3Zhcm5pc2g3MDJ0c3QuZXcxLnZjaW50LmNvbQ:_node_exporter.domain.com/default",
"name": "_node_exporter.domain.com",
"port": 9100,
"priority": 0,
"target": "server.domain.com",
"view": "default",
"weight": 0
}
]
}
I want to use the data of _ref from _infoblox_results.json, but I wasn't able to extract it with regex_replace (it drops back the full _infoblox_results.json):
- name: Get Record ID
set_fact:
_rcdid: "{{ _infoblox_results.json | regex_replace('record:srv.*\\/default,', '\\1') }}"
- debug:
var: _rcdid
when: _infoblox_results.json != []
Neither with json_query (it drops back nothing):
- name: Get Record ID
set_fact:
_rcdid: "{{ _infoblox_results.json | json_query('_ref') }}"
- debug:
var: _rcdid
when: _infoblox_results.json != []
Can someone please point me into the right direction?
You have already an object in the memory, so simply refer to its value: _infoblox_results.json[0]._ref contains the string record:srv/ZG5zLmJpbmRfc3J2JC5fZGVmYXVsdC5jb20udmNpbnQuZXcxL19ub2RlX2V4cG9ydGVyLzAvMC85MTAwL3Zhcm5pc2g3MDJ0c3QuZXcxLnZjaW50LmNvbQ:_node_exporter.domain.com/default.
With that you can split the string and select the second element:
- name: Get Record ID
set_fact:
_rcdid: "{{ _infoblox_results.json[0]._ref.split('/')[1] }}"