Ansible pass json in varibale in ansible-playbook --extra-vars= - json

i try to pass JSON structure into ansible-playbook without success:
this is the command I try to pass
ansible-playbook --extra-vars='[{\"${foo1}\": \"somevalue1\", \"${foo2}\": \"somevalue2\"}, {\"${zoo1}\": \"somevalue111\", \"${zoo2}\": \"somevalue222\"}]' test.yml
getting error:
ERROR! Syntax Error while loading YAML.
expected ',' or '}', but got '{'
or this :
ansible-playbook --extra-vars='[{"${foo1}":"somevalue1","${foo2}":"somevalue2"},{"${zoo1}":"somevalue111","${zoo2}":"somevalue222"}]' test.yml
Getting no output
The ideal way is passing the JSON into variable like this so i could iterate the json array in ansible :
ansible-playbook --extra-vars="AAA='[{\"${foo1}\": \"somevalue1\", \"${foo2}\": \"somevalue2\"}, {\"${zoo1}\": \"somevalue111\", \"${zoo2}\": \"somevalue222\"}]'" test.yml
with this playbook :
---
-
gather_facts: false
hosts: localhost
name: test
tasks:
- name: debug
debug:
msg: "{{ AAA }}"
The output is :
ok: [localhost] =>
msg:
- ? ''
: somevalue2
- ? ''
: somevalue222
In short, what is the best way to pass JSON structure into ansible without using the file?

I don't follow why you're escaping the " inside of a ' string, but you will want to switch away from the KEY=VALUE syntax because in that form, ansible splits on whitespace -- by leading with the { it informs ansible that the --extrav-vars is in fact JSON and stops using the key-value parser
ansible -e '{"AAA": [{"hello":{"world": true}}, {"array":{"yup":"forsure"}}]}' -m debug -a var=AAA localhost
produces
localhost | SUCCESS => {
"AAA": [
{
"hello": {
"world": true
}
},
{
"array": {
"yup": "forsure"
}
}
]
}

Related

Ansible access JSON object with dynamic key and special character

I have an Ansible script that performs API call and stores the JSON result in a variable. The JSON result from the API call is in the following format:
{
"result": {
"Object::18": {
"message": "object message"
},
message: "hello"
}
}
In this case the key Object::18 is dynamically generated from the ID given as argument in the Ansible script:
$ ansible-playbook test.yaml --extra-var='id=18'
How do I get the JSON value for key result.Object::{{id}}.message in this case? Please find below a sample Ansible script:
- name: Get API object
hosts: localhost
vars:
object_id: '{{ id }}'
# Sample JSON result returned by the API call
# Object::<id> (the key "Object::<id>" depends on the ID given as argument for the ansible script)
# For example, if run using --extra-vars "id=18", it will be Object::18. If id=19 then Object::19
sample_json_result:
{
"result": {
"Object::18": {
"message": "object message"
},
message: "hello"
}
}
tasks:
- debug:
# How to get result.Object::<id>.message???
# For example 1: --extra-vars "id=18", result.Object::18.message
# Example 2: --extra-vars "id=19", result.Object::19.message
msg: '{{ sample_json_result.result }}'
# The following does NOT work
#msg: '{{ sample_json_result.result.Object\:\:{{id}}.message }}'
Thank you and please let me know if you have any further questions.
EDIT:
For those who are interested, solution as suggested by Zeitounator
- name: Get API object
hosts: localhost
vars:
object_id: '{{ id }}'
# Sample JSON result returned by the API call
# Object::<id> (the key "Object::<id>" depends on the ID given as argument for the ansible script)
# For example, if run using --extra-vars "id=18", it will be Object::18. If id=19 then Object::19
sample_json_result:
{
"result": {
"Object::18": {
"message": "object message"
},
message: "hello"
}
}
tasks:
- debug:
msg: '{{ sample_json_result.result["Object::" ~ id].message }}'

Ansible: Invalid JSON when using --extra-vars

Hi community,
I have been struggling with an issue in ansible issue for days now.
Everything is executed wihtin a Jenkins pipeline.
The ansible command looks like:
sh """
ansible-playbook ${env.WORKSPACE}/cost-optimization/ansible/manage_dynamo_db.yml \
--extra-vars '{"projectNameDeployConfig":${projectNameDeployConfig},"numberOfReplicas":${numberOfReplicas},"dynamodbtask":${dynamodbtask}}'
"""
And the playbooks is:
playbook.yml
---
- hosts: localhost
vars:
numberOfReplicas: "{{numberOfReplicas}}"
dynamodbtask: "{{dynamodbtask}}"
namespace: "{{projectNameDeployConfig}}"
status: "{{status}}"
- tasks:
- name: "Get replica number for the pods"
command: aws dynamodb put-item --table-name pods_replicas
register: getResult
when: dynamodbtask == "get"
- name: "Update replica number for specified pods"
command: |
aws dynamodb put-item
--table-name pods_replicas
--item '{"ProjectNameDeployConfig":{"S":{{namespace}}},"NumberReplicas":{"N":{{numberOfReplicas}}}}'
register: updatePayload
when: dynamodbtask == "put" and getResult is skipped
However, there is always the following error:
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["aws", "dynamodb", "put-item", "--table-name",
"pods_replicas", "--item", "{\"ProjectNameDeployConfig\":{\"S\":LERN-PolicyCenterV10},\"NumberReplicas\":
{\"N\":0}}"], "delta": "0:00:01.702107", "end": "2020-02-09 16:58:26.055579",
"msg": "non-zero return code", "rc": 255, "start": "2020-02-09 16:58:24.353472", "stderr": "\nError parsing parameter '--item': Invalid JSON: No JSON object could be decoded\nJSON received: {\"ProjectNameDeployConfig\":{\"S\":LERN-PolicyCenterV10},\"NumberReplicas\":{\"N\":0}}", "stderr_lines": ["", "Error parsing parameter '--item': Invalid JSON: No JSON object could be decoded", "JSON received: {\"ProjectNameDeployConfig\":{\"S\":LERN-PolicyCenterV10},\"NumberReplicas\":{\"N\":0}}"], "stdout": "", "stdout_lines": []}
There are two answers to your question: the simple one and the correct one
The simple one is that had you actually fed the JSON into jq, or python -m json.tool, you would have observed that namespace is unquoted:
"{\"ProjectNameDeployConfig\":{\"S\": LERN-PolicyCenterV10 },\"NumberReplicas\": {\"N\":0}}"
where I added a huge amount of space, but didn't otherwise alter the quotes
The correct answer is that you should never use jinja2 to try and assemble structured text when there are filters that do so for you.
What you actually want is to use the to_json filter:
- name: "Update replica number for specified pods"
command: |
aws dynamodb put-item
--table-name pods_replicas
--item {{ dynamodb_item | to_json | quote }}
vars:
dynamodb_item:
"ProjectNameDeployConfig":
"S": '{{ projectNameDeployConfig }}'
"NumberReplicas":
"N": 0
register: updatePayload
when: dynamodbtask == "put" and getResult is skipped
although you'll notice that I changed your variable name because namespace is the name of a type in jinja2, so you can either call it ns or I just used the interpolation value from your vars: block at the top of the playbook, as it doesn't appear that it changed from then

Ansible: json element strip key with colon

How do you remove a json Key that contains a colon (:) with jinja2 rejectattr.
Environment:
ansible 2.9.1
config file = None
configured module search path = [u'/home/<user>/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.15+ (default, Oct 7 2019, 17:39:04) [GCC 7.4.0]
json data:
{
"tag:environment": "qa",
"tag:instance_id": "i-123456789"
}
Ansible playbook:
- name: Remove InstanceID
debug:
msg: "{{ instance_filter | rejectattr('['tag:environment'], 'defined' ') | list }}
Actual Results:
fatal: [localhost]: FAILED! => {
"msg": "template error while templating string: expected token ',', got 'tag'. String: {{ instance_filter | rejectattr('['tag:environment'], 'defined' ') | list }}"
}
Expected results:
{
"tag:environment": "qa"
}
The rejectattr is indeed one of the key filters to use to achieve your goal, but a few more things are needed. Here is the correct sequence of filters to remove that particular key from the dictionary variable you have:
Playbook:
---
- hosts: localhost
gather_facts: false
vars:
instance_filter:
tag:environment: qa
tag:instance_id: i-123456789
tasks:
- name: print var
debug:
var: instance_filter
- name: manipulate the var
debug:
msg: "{{ instance_filter | dict2items | rejectattr('key', 'equalto', 'tag:instance_id') | list | items2dict }}"
Output:
PLAY [localhost] *******************************************************************************************************************************************************************************************************
TASK [print var] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
"instance_filter": {
"tag:environment": "qa",
"tag:instance_id": "i-123456789"
}
}
TASK [manipulate the var] **********************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"tag:environment": "qa"
}
}
PLAY RECAP *************************************************************************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
hope it helps.
Q: "How do you remove a JSON key?"
A: It's possible to create custom filter plugins. For example
$ cat filter_plugins/dict_utils.py
def dict_del_key(d, key):
del d[key]
return d
class FilterModule(object):
''' Ansible filters. Interface to Python dictionary methods.'''
def filters(self):
return {
'dict_del_key' : dict_del_key
}
The play below
- hosts: localhost
vars:
dict:
'tag:environment': 'qa'
'tag:instance_id': 'i-123456789'
tasks:
- debug:
msg: "{{ dict|dict_del_key('tag:instance_id') }}"
gives
msg:
tag:environment: qa
Notes:
See If you quote those config keys, they will become strings.
See the difference between 7.3.1. Double-Quoted Style and 7.3.2. Single-Quoted Style.
FWIW. See other filters available at GitHub.

ansible parse json array reply from api

I am trying to parse a json response from an API. The response in a browser looks like:
[{url: "abc.com/xyz"}]
I request it from ansible:
- name: Get url
uri:
url: my-url...
method: GET
force: yes
return_content: yes
#HEADER_Content-Type: "application/json"
register: json_response
I get a reply from ansible that looks like this (with debug):
- name: print reply
debug:
var: json_response
verbosity: 1
which gives:
ok: [server] => {
"json_response": {
... //removed for readability
"content": "({:url \"https://the-file-I-want\"})"
}
So it seems like some parsing happened already (note the colons :).
Accessing the content seem to work (with debug json_response['content']):
ok: [server] => {
"json_response['content']": "({:url \"https://the-file-I-want\"})"
}
But I cannot seems to access the json response url. If I try to take the first element of the array, I get "(" so it seems it is still a string.
- name: print reply2
debug:
var: json_response['content'][0]
verbosity: 1
from_json does not seem to work: fatal: [server]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined....
How do I parse a json reply like this one?
I created a json file response.json with the following contents:
{
content: ({:url \"https://the-file-I-want\"})
}
Then, in my playbook I loaded the file and to get the url you need, I created a custom jinja filter since Jinja2 does not have any filter for finding sub-string or regexp.
My custom filter named filter.py(you can name it anything) is in a dir called filter_plugins in the same directory as my playbook. My filter.py file is as follows:
import re
class FilterModule(object):
''' Custom filters are loaded by FilterModule objects '''
def filters(self):
return {'urlsubstr': self.urlsubstr}
def urlsubstr(self,content):
url = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_#.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', content)
return url[0]
After creating the custom filter, I got the url like this:
- hosts: localhost
vars:
json_response: "{{ lookup('file', 'response.json') | from_json }}"
tasks:
- debug: msg="{{ json_response.content | urlsubstr }}"
with_dict: "{{ json_response }}"
This is the output of running my playbook:
TASK [setup] *******************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => (item={'value': u'({:url "https://the-file-I-want"})', 'key': u'content'}) => {
"item": {
"key": "content",
"value": "({:url \"https://the-file-I-want\"})"
},
"msg": "https://the-file-I-want"
}
Hope this helps.
To make the response json use the to_json filter, then navigate to the url key. That should give the value you're looking for: https://.....
Here's the documentation:
http://docs.ansible.com/ansible/playbooks_filters.html#filters-for-formatting-data
The original response works on any case but it seems overkill unless there's a problem with the conversion to JSON, like in yours.
Nevertheless I think it might help me do something else I intend to do, which is why I was looking in here.

Ansible: Remove whitespaces in json file

I have a json file content loaded in my ansible variables.
The json content (and the resulting file I write from it) has unnecessary file spaces and blank lines. I want to minify the json file by removing all that unnecessary stuff.
Is it possible to do something like {{ myjson_content| to_json_minify }} ?
Maybe this can be done through a regex ?
You can read it from json and convert back into json with separators option.
{{ my_json_content | from_json | to_json(separators=(',',':')) }}
Note: this is not documented, but if you look at the source code you will see that the filter accepts arbitrary keywords args which are later passed to the python json.dumps function. So you can basically pass to to_json any parameter accepted by json.dumps.
playbook.yml
---
- hosts: localhost
vars:
my_json_content:
'
{ "a" : 0,
"b": 1,
"c": 2}
'
tasks:
- debug:
msg: "json = {{ my_json_content }}"
- debug:
msg: "minified_json = {{ my_json_content | from_json | to_json(separators=(',',':')) }}"
$ ansible-playbook playbook.yml
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "json = { \"a\" : 0,\n\"b\": 1,\n\"c\": 2}\n"
}
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "minified_json = {\"a\":0,\"c\":2,\"b\":1}"
}