ansible parse json array reply from api - json

I am trying to parse a json response from an API. The response in a browser looks like:
[{url: "abc.com/xyz"}]
I request it from ansible:
- name: Get url
uri:
url: my-url...
method: GET
force: yes
return_content: yes
#HEADER_Content-Type: "application/json"
register: json_response
I get a reply from ansible that looks like this (with debug):
- name: print reply
debug:
var: json_response
verbosity: 1
which gives:
ok: [server] => {
"json_response": {
... //removed for readability
"content": "({:url \"https://the-file-I-want\"})"
}
So it seems like some parsing happened already (note the colons :).
Accessing the content seem to work (with debug json_response['content']):
ok: [server] => {
"json_response['content']": "({:url \"https://the-file-I-want\"})"
}
But I cannot seems to access the json response url. If I try to take the first element of the array, I get "(" so it seems it is still a string.
- name: print reply2
debug:
var: json_response['content'][0]
verbosity: 1
from_json does not seem to work: fatal: [server]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined....
How do I parse a json reply like this one?

I created a json file response.json with the following contents:
{
content: ({:url \"https://the-file-I-want\"})
}
Then, in my playbook I loaded the file and to get the url you need, I created a custom jinja filter since Jinja2 does not have any filter for finding sub-string or regexp.
My custom filter named filter.py(you can name it anything) is in a dir called filter_plugins in the same directory as my playbook. My filter.py file is as follows:
import re
class FilterModule(object):
''' Custom filters are loaded by FilterModule objects '''
def filters(self):
return {'urlsubstr': self.urlsubstr}
def urlsubstr(self,content):
url = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_#.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', content)
return url[0]
After creating the custom filter, I got the url like this:
- hosts: localhost
vars:
json_response: "{{ lookup('file', 'response.json') | from_json }}"
tasks:
- debug: msg="{{ json_response.content | urlsubstr }}"
with_dict: "{{ json_response }}"
This is the output of running my playbook:
TASK [setup] *******************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => (item={'value': u'({:url "https://the-file-I-want"})', 'key': u'content'}) => {
"item": {
"key": "content",
"value": "({:url \"https://the-file-I-want\"})"
},
"msg": "https://the-file-I-want"
}
Hope this helps.

To make the response json use the to_json filter, then navigate to the url key. That should give the value you're looking for: https://.....
Here's the documentation:
http://docs.ansible.com/ansible/playbooks_filters.html#filters-for-formatting-data
The original response works on any case but it seems overkill unless there's a problem with the conversion to JSON, like in yours.
Nevertheless I think it might help me do something else I intend to do, which is why I was looking in here.

Related

How can I loop through json object REST API body in Ansible?

When using Ansible I am able to execute when passed one-by-one like this:
---
- name: Using a REST API
become: false
hosts: localhost
gather_facts: false
tasks:
- debug:
msg: “Let’s get list of Interfaces”
- name: Adding a Bridge-Interface
uri:
url: https://router/rest/interface/bridge
method: PUT
validate_certs: false
url_username: ansible
url_password: ansible
force_basic_auth: yes
body_format: json
status_code: 201
body: '{"name":"bridge_ansible"}'
register: results
- debug:
var: results
I want to iterate through a set of commands so I thought of looping, but that does not work for me, I am using this code:
---
- name: Using a REST API
become: false
hosts: localhost
gather_facts: false
tasks:
- debug:
msg: “Let’s get list of Interfaces”
- name: Adding a Bridge-Interface
uri:
url: "{{item.url}}"
method: PUT
validate_certs: false
url_username: ansible
url_password: ansible
force_basic_auth: yes
body_format: json
status_code: 201
body: "{{item.body}}"
register: results
loop:
- {body:'{"name":"bridge_ansible"}', url:'https://router/rest/interface/bridge'}
- {body:'{"address":"6.6.6.6", "interface":"bridge_ansible"}', url:'https://router/rest/ip/address'}
- debug:
var: results
I get an error for this {body:'{"name":"bridge_ansible"}', url:'https://router/rest/interface/bridge'} in the json object { I think my syntax is not correct but cannot understand the correct thing. Can someone please help
ERROR! We were unable to read either as JSON nor YAML, these are the errors we got from each:
JSON: Expecting value: line 1 column 1 (char 0)
Syntax Error while loading YAML.
did not find expected ',' or '}'
The error appears to be in '/ansible-playbook/1-demo.yaml': line 23, column 19, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
loop:
- {body:'\{"name":"bridge_ansible"\}', url:'https://router/rest/interface/bridge'}
^ here
This one looks easy to fix. It seems that there is a value started
with a quote, and the YAML parser is expecting to see the line ended
with the same kind of quote. For instance:
when: "ok" in result.stdout
Could be written as:
when: '"ok" in result.stdout'
Or equivalently:
when: "'ok' in result.stdout"
We could be wrong, but this one looks like it might be an issue with
unbalanced quotes. If starting a value with a quote, make sure the
line ends with the same set of quotes. For instance this arbitrary
example:
foo: "bad" "wolf"
Could be written as:
foo: '"bad" "wolf"'
Thanks

Filter a JSON document in Ansible

I have a JSON reply from a GitHub repository with a list of possible downloads for a certain release (the assets array in the document).
I want to get the browser download URL when the name of an asset ends with x64.AppImage.
In Ansible, the filters are built apon jmespath and using its terminal tool, I can query the url with the following expression:
assets[?ends_with(name, 'x64.AppImage')].browser_download_url
With the following playbook, the JSON document is queried and stored in the json_reply variable.
---
- hosts: local
tasks:
- name: Get list of Rambox releases
uri:
url: "https://api.github.com/repos/saenzramiro/rambox/releases/latest"
body_format: json
register: json_reply
- name: Filter reply
debug: URL -> "{{ item }}"
with_items:
- "{{ json_reply.json | json_query(json_filter) }}"
vars:
- json_filter: assets[?ends_with(name, 'x64.AppImage')].browser_download_url
However, executing this gives the following error:
fatal: [localhost]: FAILED! => {
"msg": "JMESPathError in json_query filter plugin:\nIn function ends_with(), invalid type for value: latest-mac.json, expected one of: ['string'], received: \"unknown\""
}
Where latest-mac.json is the first object in the assets array.
How can I make Ansible to iterate over all the assets array and apply my filter?
PS:
If instead of querying if the name ends with a word I specify it directly, the filter works:
assets[?name == 'Rambox-0.5.13-x64.AppImage')].browser_download_url
JSON example:
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/8001922",
"prerelease": false,
"created_at": "2017-10-04T21:14:15Z",
"published_at": "2017-10-05T01:10:55Z",
"assets": [
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/assets/4985942",
"id": 4985942,
"name": "latest-mac.json",
"uploader": {
"login": "saenzramiro",
"id": 2694669
},
"browser_download_url": "https://github.com/saenzramiro/rambox/releases/download/0.5.13/latest-mac.json"
},
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/assets/4985640",
"id": 4985640,
"name": "Rambox-0.5.13-x64.AppImage",
"uploader": {
"login": "saenzramiro",
"id": 2694669
},
"browser_download_url": "https://github.com/saenzramiro/rambox/releases/download/0.5.13/Rambox-0.5.13-x64.AppImage"
}
],
"tarball_url": "https://api.github.com/repos/saenzramiro/rambox/tarball/0.5.13"
}
The problem of type errors in JMESPath filters is discussed in issue 27299.
You can use this patched json_query.py filter plugin.
Or apply double conversion to your object as a workaround: | to_json | from_json |.
This will convert object to JSON (thus plain strings) and back, so json_query will treat strings as supported type.
Loop through each asset
Print the browser URL of the item if it ends with x64.AppImage
Solution not using JMESPath:
- name: Filter reply
debug: var=item.browser_download_url
with_items: "{{ json_reply.json.assets }}"
when: item.browser_download_url | regex_search('x64.AppImage$')
As #helloV said, you can accomplish this using Ansible loops, although there's no reason to involve a regular expression match. You can use the same test you're already using:
- name: Filter reply
debug:
var: item.browser_download_url
with_items: "{{ json_reply.json.assets }}"
when: item.name.endswith('x64.AppImage')
The root problem would appear to be an Ansible bug. The error comes from the following check in the jmespath library:
if actual_typename not in allowed_types:
raise exceptions.JMESPathTypeError(
function_name, current,
self._convert_to_jmespath_type(actual_typename), types)
At the point this code is called, the data type of values in your json response is AnsibleUnsafeText, where as allowed_types is [str, unicode]. I think the transformation of values from native types to the AnsibleUnsafeText type probably is some sort of standard Ansible module behavior being imposed by the uri module. We can work around it by using curl instead, like this:
- name: Get list of Rambox releases
command: >
curl -s "https://api.github.com/repos/saenzramiro/rambox/releases/latest"
register: json_reply
And then:
- name: Filter reply
debug:
var: item.browser_download_url
with_items: >
{{ json_reply.stdout|from_json|json_query('assets[?ends_with(name, `x64.AppImage`)]') }}

extracting a variable from json output then debug and register the outout with ansible

Hi I have a problem of getting one of the variables extracted from a json output after doing a curl to be parsed and registered back to ansible
Playbook:
- name: debug stdout
debug:
msg: "{{ result.stdout | from_json }}"
register: dataresult
- name: debug fact
debug:
msg: "{{ dataresult.data.start_time_string }}"
output :
TASK [backup_api : debug stdout]
***********************************************
task path: /home/ansible/cm-dha/roles/backup_api/tasks/main.yml:36
ok: [127.0.0.1] => {
"msg": {
"data": [
{
"backup_id": 40362,
"certified": null,
"instance_id": 148,
"start_time": 1506985211,
"start_time_string": "10/03/2017 03:00:11 am"
}
],
"timestamp": 1507022232
}
}
error:
fatal: [127.0.0.1]: FAILED! => {
"failed": true,
"msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'dict object' has no attribute 'data'\n\nThe error appears to have been in '/home/ansible/cm-dha/roles/backup_api/tasks/main.yml': line 48, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: debug fact\n ^ here\n"
The error is happening when trying to extract the value start_time_string
so how to do it probably as I tried too many things like using with_items, with_dict , simulating the data[] output to debug and even doing a json query but without success
so any help here?
Don't use debug to assign facts, use set_fact instead:
- name: debug stdout
set_fact:
dataresult: "{{ result.stdout | from_json }}"
- name: debug fact
debug:
msg: "{{ dataresult.data[0].start_time_string }}"
Thanks to accepted answer, I've made it as below for my case. Leaving it here, maybe it helps someone.
- name: Connectors
shell: curl -X GET http://myserver:8083/connectors
register: out6
- set_fact:
dataresult: "{{ out6.stdout | from_json }}"
- debug: var=dataresult

How to inspect a json response from Ansible URI call

I have a service call that returns system status in json format. I want to use the ansible URI module to make the call and then inspect the response to decide whether the system is up or down
{"id":"20161024140306","version":"5.6.1","status":"UP"}
This would be the json that is returned
This is the ansible task that makes a call:
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: data
Question is how can I access data and inspect it as per ansible documentation this is how we store results of a call. I am not sure of the final step which is to check the status.
This works for me.
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: result
until: result.json.status == "UP"
retries: 10
delay: 30
Notice that result is a ansible dictionary and when you set return_content=yes the response is added to this dictionary and is accessible using json key
Also ensure you have indented the task properly as shown above.
You've made the right first step by saving the output into a variable.
The next step is to use either when: or failed_when: statement in your next task, which will then switch based on the contents of the variable. There are a whole powerful set of statements for use in these, the Jinja2 builtin filters, but they are not really linked well into the Ansible documentation, or summarised nicely.
I use super explicitly named output variables, so they make sense to me later in the playbook :) I would probably write yours something like:
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: sonar_web_api_status_output
- name: do this thing if it is NOT up
shell: echo "OMG it's not working!"
when: sonar_web_api_status_output.stdout.find('UP') == -1
That is, the text "UP" is not found in the variable's stdout.
Other Jinja2 builtin filters I've used are:
changed_when: "'<some text>' not in your_variable_name.stderr"
when: some_number_of_files_changed.stdout|int > 0
The Ansible "Conditionals" docs page has some of this info. This blog post was also very informative.
As per documentation at https://docs.ansible.com/ansible/latest/modules/uri_module.html
Whether or not to return the body of the response as a "content" key in the dictionary result. Independently of this option, if the reported Content-type is "application/json", then the JSON is always loaded into a key called json in the dictionary results.
---
- name: Example of JSON body parsing with uri module
connection: local
gather_facts: true
hosts: localhost
tasks:
- name: Example of JSON body parsing with uri module
uri:
url: https://jsonplaceholder.typicode.com/users
method: GET
return_content: yes
status_code: 200
body_format: json
register: data
# failed_when: <optional condition based on JSON returned content>
- name: Print returned json dictionary
debug:
var: data.json
- name: Print certain element
debug:
var: data.json[0].address.city

Ansible: Remove whitespaces in json file

I have a json file content loaded in my ansible variables.
The json content (and the resulting file I write from it) has unnecessary file spaces and blank lines. I want to minify the json file by removing all that unnecessary stuff.
Is it possible to do something like {{ myjson_content| to_json_minify }} ?
Maybe this can be done through a regex ?
You can read it from json and convert back into json with separators option.
{{ my_json_content | from_json | to_json(separators=(',',':')) }}
Note: this is not documented, but if you look at the source code you will see that the filter accepts arbitrary keywords args which are later passed to the python json.dumps function. So you can basically pass to to_json any parameter accepted by json.dumps.
playbook.yml
---
- hosts: localhost
vars:
my_json_content:
'
{ "a" : 0,
"b": 1,
"c": 2}
'
tasks:
- debug:
msg: "json = {{ my_json_content }}"
- debug:
msg: "minified_json = {{ my_json_content | from_json | to_json(separators=(',',':')) }}"
$ ansible-playbook playbook.yml
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "json = { \"a\" : 0,\n\"b\": 1,\n\"c\": 2}\n"
}
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "minified_json = {\"a\":0,\"c\":2,\"b\":1}"
}