I have a json file content loaded in my ansible variables.
The json content (and the resulting file I write from it) has unnecessary file spaces and blank lines. I want to minify the json file by removing all that unnecessary stuff.
Is it possible to do something like {{ myjson_content| to_json_minify }} ?
Maybe this can be done through a regex ?
You can read it from json and convert back into json with separators option.
{{ my_json_content | from_json | to_json(separators=(',',':')) }}
Note: this is not documented, but if you look at the source code you will see that the filter accepts arbitrary keywords args which are later passed to the python json.dumps function. So you can basically pass to to_json any parameter accepted by json.dumps.
playbook.yml
---
- hosts: localhost
vars:
my_json_content:
'
{ "a" : 0,
"b": 1,
"c": 2}
'
tasks:
- debug:
msg: "json = {{ my_json_content }}"
- debug:
msg: "minified_json = {{ my_json_content | from_json | to_json(separators=(',',':')) }}"
$ ansible-playbook playbook.yml
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "json = { \"a\" : 0,\n\"b\": 1,\n\"c\": 2}\n"
}
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "minified_json = {\"a\":0,\"c\":2,\"b\":1}"
}
Related
i try to pass JSON structure into ansible-playbook without success:
this is the command I try to pass
ansible-playbook --extra-vars='[{\"${foo1}\": \"somevalue1\", \"${foo2}\": \"somevalue2\"}, {\"${zoo1}\": \"somevalue111\", \"${zoo2}\": \"somevalue222\"}]' test.yml
getting error:
ERROR! Syntax Error while loading YAML.
expected ',' or '}', but got '{'
or this :
ansible-playbook --extra-vars='[{"${foo1}":"somevalue1","${foo2}":"somevalue2"},{"${zoo1}":"somevalue111","${zoo2}":"somevalue222"}]' test.yml
Getting no output
The ideal way is passing the JSON into variable like this so i could iterate the json array in ansible :
ansible-playbook --extra-vars="AAA='[{\"${foo1}\": \"somevalue1\", \"${foo2}\": \"somevalue2\"}, {\"${zoo1}\": \"somevalue111\", \"${zoo2}\": \"somevalue222\"}]'" test.yml
with this playbook :
---
-
gather_facts: false
hosts: localhost
name: test
tasks:
- name: debug
debug:
msg: "{{ AAA }}"
The output is :
ok: [localhost] =>
msg:
- ? ''
: somevalue2
- ? ''
: somevalue222
In short, what is the best way to pass JSON structure into ansible without using the file?
I don't follow why you're escaping the " inside of a ' string, but you will want to switch away from the KEY=VALUE syntax because in that form, ansible splits on whitespace -- by leading with the { it informs ansible that the --extrav-vars is in fact JSON and stops using the key-value parser
ansible -e '{"AAA": [{"hello":{"world": true}}, {"array":{"yup":"forsure"}}]}' -m debug -a var=AAA localhost
produces
localhost | SUCCESS => {
"AAA": [
{
"hello": {
"world": true
}
},
{
"array": {
"yup": "forsure"
}
}
]
}
How do you remove a json Key that contains a colon (:) with jinja2 rejectattr.
Environment:
ansible 2.9.1
config file = None
configured module search path = [u'/home/<user>/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.15+ (default, Oct 7 2019, 17:39:04) [GCC 7.4.0]
json data:
{
"tag:environment": "qa",
"tag:instance_id": "i-123456789"
}
Ansible playbook:
- name: Remove InstanceID
debug:
msg: "{{ instance_filter | rejectattr('['tag:environment'], 'defined' ') | list }}
Actual Results:
fatal: [localhost]: FAILED! => {
"msg": "template error while templating string: expected token ',', got 'tag'. String: {{ instance_filter | rejectattr('['tag:environment'], 'defined' ') | list }}"
}
Expected results:
{
"tag:environment": "qa"
}
The rejectattr is indeed one of the key filters to use to achieve your goal, but a few more things are needed. Here is the correct sequence of filters to remove that particular key from the dictionary variable you have:
Playbook:
---
- hosts: localhost
gather_facts: false
vars:
instance_filter:
tag:environment: qa
tag:instance_id: i-123456789
tasks:
- name: print var
debug:
var: instance_filter
- name: manipulate the var
debug:
msg: "{{ instance_filter | dict2items | rejectattr('key', 'equalto', 'tag:instance_id') | list | items2dict }}"
Output:
PLAY [localhost] *******************************************************************************************************************************************************************************************************
TASK [print var] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
"instance_filter": {
"tag:environment": "qa",
"tag:instance_id": "i-123456789"
}
}
TASK [manipulate the var] **********************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"tag:environment": "qa"
}
}
PLAY RECAP *************************************************************************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
hope it helps.
Q: "How do you remove a JSON key?"
A: It's possible to create custom filter plugins. For example
$ cat filter_plugins/dict_utils.py
def dict_del_key(d, key):
del d[key]
return d
class FilterModule(object):
''' Ansible filters. Interface to Python dictionary methods.'''
def filters(self):
return {
'dict_del_key' : dict_del_key
}
The play below
- hosts: localhost
vars:
dict:
'tag:environment': 'qa'
'tag:instance_id': 'i-123456789'
tasks:
- debug:
msg: "{{ dict|dict_del_key('tag:instance_id') }}"
gives
msg:
tag:environment: qa
Notes:
See If you quote those config keys, they will become strings.
See the difference between 7.3.1. Double-Quoted Style and 7.3.2. Single-Quoted Style.
FWIW. See other filters available at GitHub.
Summary
When retrieving data using the uri module in Ansible, I am unable to parse a section of it as JSON to retrieve a nested value.
The desired value is the ci field inside the content.data or json.data field (see output below).
Steps to Reproduce
site.yml
---
- hosts: localhost
gather_facts: false
tasks:
- name: Get String
uri:
url: "http://localhost/get-data"
method: POST
body_format: json
body: "{ \"kong-jid\": \"run-sn-discovery\" }"
return_content: yes
register: output
- set_fact:
ci: "{{ output.json.data.ci }}"
- debug:
msg: "{{ ci }}"
The {{ output }} variable
{
u'status': 200,
u'cookies': {},
u'url': u'http://kong-demo:8000/get-data',
u'transfer_encoding': u'chunked',
u'changed': False,
u'connection': u'close',
u'server': u'kong/0.34-1-enterprise-edition',
u'content':
u'{"data":"\\"{u\'ci\': u\'3bb8d625dbac3700e4f07b6e0f96195b\'}\\""}',
'failed': False,
u'json': {u'data': u'"{u\'ci\': u\'3bb8d625dbac3700e4f07b6e0f96195b\'}"'},
u'content_type': u'application/json',
u'date': u'Thu, 18 Apr 2019 15:50:25 GMT',
u'redirected': False,
u'msg': u'OK (unknown bytes)'
}
Result
[user#localhost]$ ansible-playbook site.yml
[WARNING]: Could not match supplied host pattern, ignoring: all
[WARNING]: provided hosts list is empty, only localhost is available
PLAY [localhost] ***************************************************************************************************************
TASK [Pass Redis data to next task as output] **********************************************************************************
ok: [localhost]
TASK [set_fact] ****************************************************************************************************************
fatal: [localhost]: FAILED! => {}
MSG:
The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'ci'
The error appears to have been in 'site.yml': line 19, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- set_fact:
^ here
exception type: <class 'ansible.errors.AnsibleUndefinedVariable'>
exception: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'ci'
Important Troubleshooting Information
It appears the root issue is related to which Ansible type being interpreted. I desire to parse ci from the output in one task.
The two-task solution shown below works, but this leads me to believe this should be possible in one line...
Two-Task Solution
- set_fact:
ci: "{{ output.json.data | from_json }}"
- debug:
msg: "{{ ci['ci'] }}"
But the ci fact set from {{ output.json.data | from_json }} reports a different TYPE than the inline type...
Unicode or Dict?
- debug:
msg: "{{ output.json.data | from_json | type_debug }}" # returns unicode
- set_fact:
ci: "{{ output.json.data | from_json }}"
- debug:
msg: "{{ ci | type_debug }}" # returns dict
Why isn't {{ output.json.data | from_json | type_debug }}
the same as {{ ci | type_debug }}?
Although json and data are keys in their resp objects, ci is just part of a larger string (which happens to look like a JSON object
If the relevant line in your datastructure would be:
u'json': {u'data': {'ci': u'3bb8d625dbac3700e4f07b6e0f96195b'}},
then you could expect to use "{{ output.json.data.ci }}" but not when the .ci part is just a normal part of a string.
Hi I have a problem of getting one of the variables extracted from a json output after doing a curl to be parsed and registered back to ansible
Playbook:
- name: debug stdout
debug:
msg: "{{ result.stdout | from_json }}"
register: dataresult
- name: debug fact
debug:
msg: "{{ dataresult.data.start_time_string }}"
output :
TASK [backup_api : debug stdout]
***********************************************
task path: /home/ansible/cm-dha/roles/backup_api/tasks/main.yml:36
ok: [127.0.0.1] => {
"msg": {
"data": [
{
"backup_id": 40362,
"certified": null,
"instance_id": 148,
"start_time": 1506985211,
"start_time_string": "10/03/2017 03:00:11 am"
}
],
"timestamp": 1507022232
}
}
error:
fatal: [127.0.0.1]: FAILED! => {
"failed": true,
"msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'dict object' has no attribute 'data'\n\nThe error appears to have been in '/home/ansible/cm-dha/roles/backup_api/tasks/main.yml': line 48, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: debug fact\n ^ here\n"
The error is happening when trying to extract the value start_time_string
so how to do it probably as I tried too many things like using with_items, with_dict , simulating the data[] output to debug and even doing a json query but without success
so any help here?
Don't use debug to assign facts, use set_fact instead:
- name: debug stdout
set_fact:
dataresult: "{{ result.stdout | from_json }}"
- name: debug fact
debug:
msg: "{{ dataresult.data[0].start_time_string }}"
Thanks to accepted answer, I've made it as below for my case. Leaving it here, maybe it helps someone.
- name: Connectors
shell: curl -X GET http://myserver:8083/connectors
register: out6
- set_fact:
dataresult: "{{ out6.stdout | from_json }}"
- debug: var=dataresult
I am trying to parse a json response from an API. The response in a browser looks like:
[{url: "abc.com/xyz"}]
I request it from ansible:
- name: Get url
uri:
url: my-url...
method: GET
force: yes
return_content: yes
#HEADER_Content-Type: "application/json"
register: json_response
I get a reply from ansible that looks like this (with debug):
- name: print reply
debug:
var: json_response
verbosity: 1
which gives:
ok: [server] => {
"json_response": {
... //removed for readability
"content": "({:url \"https://the-file-I-want\"})"
}
So it seems like some parsing happened already (note the colons :).
Accessing the content seem to work (with debug json_response['content']):
ok: [server] => {
"json_response['content']": "({:url \"https://the-file-I-want\"})"
}
But I cannot seems to access the json response url. If I try to take the first element of the array, I get "(" so it seems it is still a string.
- name: print reply2
debug:
var: json_response['content'][0]
verbosity: 1
from_json does not seem to work: fatal: [server]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined....
How do I parse a json reply like this one?
I created a json file response.json with the following contents:
{
content: ({:url \"https://the-file-I-want\"})
}
Then, in my playbook I loaded the file and to get the url you need, I created a custom jinja filter since Jinja2 does not have any filter for finding sub-string or regexp.
My custom filter named filter.py(you can name it anything) is in a dir called filter_plugins in the same directory as my playbook. My filter.py file is as follows:
import re
class FilterModule(object):
''' Custom filters are loaded by FilterModule objects '''
def filters(self):
return {'urlsubstr': self.urlsubstr}
def urlsubstr(self,content):
url = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_#.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', content)
return url[0]
After creating the custom filter, I got the url like this:
- hosts: localhost
vars:
json_response: "{{ lookup('file', 'response.json') | from_json }}"
tasks:
- debug: msg="{{ json_response.content | urlsubstr }}"
with_dict: "{{ json_response }}"
This is the output of running my playbook:
TASK [setup] *******************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => (item={'value': u'({:url "https://the-file-I-want"})', 'key': u'content'}) => {
"item": {
"key": "content",
"value": "({:url \"https://the-file-I-want\"})"
},
"msg": "https://the-file-I-want"
}
Hope this helps.
To make the response json use the to_json filter, then navigate to the url key. That should give the value you're looking for: https://.....
Here's the documentation:
http://docs.ansible.com/ansible/playbooks_filters.html#filters-for-formatting-data
The original response works on any case but it seems overkill unless there's a problem with the conversion to JSON, like in yours.
Nevertheless I think it might help me do something else I intend to do, which is why I was looking in here.