How can I loop through json object REST API body in Ansible? - json

When using Ansible I am able to execute when passed one-by-one like this:
---
- name: Using a REST API
become: false
hosts: localhost
gather_facts: false
tasks:
- debug:
msg: “Let’s get list of Interfaces”
- name: Adding a Bridge-Interface
uri:
url: https://router/rest/interface/bridge
method: PUT
validate_certs: false
url_username: ansible
url_password: ansible
force_basic_auth: yes
body_format: json
status_code: 201
body: '{"name":"bridge_ansible"}'
register: results
- debug:
var: results
I want to iterate through a set of commands so I thought of looping, but that does not work for me, I am using this code:
---
- name: Using a REST API
become: false
hosts: localhost
gather_facts: false
tasks:
- debug:
msg: “Let’s get list of Interfaces”
- name: Adding a Bridge-Interface
uri:
url: "{{item.url}}"
method: PUT
validate_certs: false
url_username: ansible
url_password: ansible
force_basic_auth: yes
body_format: json
status_code: 201
body: "{{item.body}}"
register: results
loop:
- {body:'{"name":"bridge_ansible"}', url:'https://router/rest/interface/bridge'}
- {body:'{"address":"6.6.6.6", "interface":"bridge_ansible"}', url:'https://router/rest/ip/address'}
- debug:
var: results
I get an error for this {body:'{"name":"bridge_ansible"}', url:'https://router/rest/interface/bridge'} in the json object { I think my syntax is not correct but cannot understand the correct thing. Can someone please help
ERROR! We were unable to read either as JSON nor YAML, these are the errors we got from each:
JSON: Expecting value: line 1 column 1 (char 0)
Syntax Error while loading YAML.
did not find expected ',' or '}'
The error appears to be in '/ansible-playbook/1-demo.yaml': line 23, column 19, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
loop:
- {body:'\{"name":"bridge_ansible"\}', url:'https://router/rest/interface/bridge'}
^ here
This one looks easy to fix. It seems that there is a value started
with a quote, and the YAML parser is expecting to see the line ended
with the same kind of quote. For instance:
when: "ok" in result.stdout
Could be written as:
when: '"ok" in result.stdout'
Or equivalently:
when: "'ok' in result.stdout"
We could be wrong, but this one looks like it might be an issue with
unbalanced quotes. If starting a value with a quote, make sure the
line ends with the same set of quotes. For instance this arbitrary
example:
foo: "bad" "wolf"
Could be written as:
foo: '"bad" "wolf"'
Thanks

Related

Ansible Making json_query / loop conditional with a when clause

I have some ansible code involving a json_query in a loop that I'm trying to make conditional. But I think there is something I'm not understanding about when clauses and loops.
The following code works fine, when the condition is met (IaC.status == 400), the problem is that when the condition is not met, the 2nd task still runs, the loop tries to process IaC2, which doesn't exist, and the task fails with: "Invalid data passed to 'loop', it requires a list, got this instead:
I think this is actually expected behaviour for conditional statements and loops: (https://docs.ansible.com/ansible/latest/user_guide/playbooks_conditionals.html#using-conditionals-in-loops), but the solution described to skip the entire task is to use an empty iterator, but I have no idea how to merge that in with the json_query() statement.
Does anyone know how I can truly make the 2nd task below conditional?
# The preceding code has made a RESTful API call to create a gitlab group, if the group
# exists the return status is 400, so I need to look it up instead. In which case the
# following code works fine. The problem is when IaC.status = 201 for some reason the loop
# in the 2nd task below tries to run, despite the when clause, and fails
# because the variable IaC2 doesn't exist
- name: or if IaC already exists
when: IaC.status == 400
uri:
url: https://{{new_hostname}}/api/v4/groups
method: GET
headers:
Authorization: "Bearer {{token.json.access_token}}"
body_format: json
body:
name: "IaC"
top_level_only: true
status_code: 200
validate_certs: yes
register: IaC2
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC.status == 400
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
You should condition your second task on the first task:
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC2 is defined
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
I cannot reproduce your case on ansible 2.9.10 where the task is skipped as expected. But you can also help with a default. I guess this would work:
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC2 is defined
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) | default([]) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
I have an answer of sorts. Firstly there were two things I didn't fully understand;
the register var (IaC2) is always created even if the uri task is skipped. And IaC2 has a totally different structure depending upon whether this task is executed or skipped.
The loop in the following task is always executed, so the when clause on the second task is pretty meaningless.
The actual problem was the jmesquery looking for the IaC2 variable to have a particular json structure, which it doesn't have in the case where the uri task is skipped.
So the solution that seems to work is checking to see if the uri task was skipped (when: IaC2.skipped is defined), and if it was skipped re-defining IaC2 in a empty list form that doesn't break the loop.
Code examples shown below.
So, it works, but I can't help thinking there must be an easier way of doing this?
- name: If IaC already exists (IaC.status == 400) then get the list of existing groups.
when: IaC.status == 400
uri:
url: https://{{new_hostname}}/api/v4/groups
method: GET
headers:
Authorization: "Bearer {{token.json.access_token}}"
body_format: json
body:
name: "IaC"
top_level_only: true
status_code: 200
validate_certs: no
register: IaC2
# To avoid breaking the following loop that searches the groups in the case that
# the group was created, re-define IaC2 into a form that won't break it.
- name: Not created group "IaC", so redefine IaC2 var so as not to break the following loop
when: IaC2.skipped is defined
set_fact:
IaC2: '{{ {"json": []} }}'
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
#when: IaC.status == 400 << Leave in or comment out, makes no difference
debug:
var: item
loop: "{{ (IaC2 | community.general.json_query(jmesquery)) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
- name: record the IaC group's id (400)
when: IaC.status == 400
set_fact:
IaC_group_id: "{{group_id.results[0].item}}"

Ansible read_csv module:How to provide path

I have a csv file as below
Hostname,Permission,User,Group,file
lbserver1,-rw-------,root,root,/tmp/dir1/4
lbserver2,drwx------,root,root,/tmp/dir1
lbserver3,-rw-------,root,root,/tmp/dir2/8
I need to use path as the key. My playbook is as below
- name: read csv
read_csv:
path: "/tmp/test.csv"
key: file
register: file_details
- name: test
debug:
msg: "{{file_details.dict./tmp/dir2/5.Permission}}"
I get error as
"msg": "template error while templating string: expected name or number. String: {{file_details.dict./tmp/dir2/5'.Permission}}"
I gave quotes as well as escape char for the paths, but still no luck. Please advise.
You can change the code like below:
- name: test
debug:
msg: "{{ file_details.dict['/tmp/dir2/5'].Permission }}"
But then it will raise an error if the key doesn't exist which is the case in your example. In that case you may use some default.
- name: test
debug:
msg: "{{ file_details.dict['/tmp/dir2/5'].Permission | default('undefined') }}"

Ansible "set_fact" repository url from json file using filters like "from_json"

Using Ansible "set_fact" module, I need to get repository url from json file using filters like "from_json". I tried in couple ways, and still doesn't get it how is should work.
- name: initial validation
tags: bundle
hosts: localhost
connection: local
tasks:
- name: register bundle version_file
include_vars:
file: '/ansible/playbook/workbench-bundle/bundle.json'
register: bundle
- name: debug registered bundle file
debug:
msg: '{{ bundle }}'
I get json that I wanted:
TASK [debug registered bundle file] ************************************************
ok: [127.0.0.1] => {
"msg": {
"ansible_facts": {
"engine-config": "git#bitbucket.org/engine-config.git",
"engine-monitor": "git#bitbucket.org/engine-monitor.git",
"engine-server": "git#bitbucket.org/engine-server.git",
"engine-worker": "git#bitbucket.org/engine-worker.git"
},
"changed": false
}
}
And then I'm trying to select each value by key name to use this value as URL to "npm install" each package in separate instances.
- name: set_fact some paramater
set_fact:
engine_url: "{{ bundle.('engine-server') | from_json }}"
And then I get error:
fatal: [127.0.0.1]: FAILED! => {"failed": true, "msg": "template error
while templating string: expected name or number. String: {{
bundle.('engine-server') }}"}
I many others ways like this loopkup, and it still fails with others errors. Can someone help to understand, how I can find each parameter and store him as "set_fact"? Thanks
Here is a sample working code to set a variable like in the question (although I don't see much sense in it):
- name: initial validation
tags: bundle
hosts: localhost
connection: local
tasks:
- name: register bundle version_file
include_vars:
file: '/ansible/playbook/workbench-bundle/bundle.json'
name: bundle
- debug:
var: bundle
- debug:
var: bundle['engine-server']
- name: set_fact some paramater
set_fact:
engine_url: "{{ bundle['engine-server'] }}"
The above assumes your input data (which you did not include) is:
{
"engine-config": "git#bitbucket.org/engine-config.git",
"engine-monitor": "git#bitbucket.org/engine-monitor.git",
"engine-server": "git#bitbucket.org/engine-server.git",
"engine-worker": "git#bitbucket.org/engine-worker.git"
}

ansible parse json array reply from api

I am trying to parse a json response from an API. The response in a browser looks like:
[{url: "abc.com/xyz"}]
I request it from ansible:
- name: Get url
uri:
url: my-url...
method: GET
force: yes
return_content: yes
#HEADER_Content-Type: "application/json"
register: json_response
I get a reply from ansible that looks like this (with debug):
- name: print reply
debug:
var: json_response
verbosity: 1
which gives:
ok: [server] => {
"json_response": {
... //removed for readability
"content": "({:url \"https://the-file-I-want\"})"
}
So it seems like some parsing happened already (note the colons :).
Accessing the content seem to work (with debug json_response['content']):
ok: [server] => {
"json_response['content']": "({:url \"https://the-file-I-want\"})"
}
But I cannot seems to access the json response url. If I try to take the first element of the array, I get "(" so it seems it is still a string.
- name: print reply2
debug:
var: json_response['content'][0]
verbosity: 1
from_json does not seem to work: fatal: [server]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined....
How do I parse a json reply like this one?
I created a json file response.json with the following contents:
{
content: ({:url \"https://the-file-I-want\"})
}
Then, in my playbook I loaded the file and to get the url you need, I created a custom jinja filter since Jinja2 does not have any filter for finding sub-string or regexp.
My custom filter named filter.py(you can name it anything) is in a dir called filter_plugins in the same directory as my playbook. My filter.py file is as follows:
import re
class FilterModule(object):
''' Custom filters are loaded by FilterModule objects '''
def filters(self):
return {'urlsubstr': self.urlsubstr}
def urlsubstr(self,content):
url = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_#.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', content)
return url[0]
After creating the custom filter, I got the url like this:
- hosts: localhost
vars:
json_response: "{{ lookup('file', 'response.json') | from_json }}"
tasks:
- debug: msg="{{ json_response.content | urlsubstr }}"
with_dict: "{{ json_response }}"
This is the output of running my playbook:
TASK [setup] *******************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => (item={'value': u'({:url "https://the-file-I-want"})', 'key': u'content'}) => {
"item": {
"key": "content",
"value": "({:url \"https://the-file-I-want\"})"
},
"msg": "https://the-file-I-want"
}
Hope this helps.
To make the response json use the to_json filter, then navigate to the url key. That should give the value you're looking for: https://.....
Here's the documentation:
http://docs.ansible.com/ansible/playbooks_filters.html#filters-for-formatting-data
The original response works on any case but it seems overkill unless there's a problem with the conversion to JSON, like in yours.
Nevertheless I think it might help me do something else I intend to do, which is why I was looking in here.

How to inspect a json response from Ansible URI call

I have a service call that returns system status in json format. I want to use the ansible URI module to make the call and then inspect the response to decide whether the system is up or down
{"id":"20161024140306","version":"5.6.1","status":"UP"}
This would be the json that is returned
This is the ansible task that makes a call:
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: data
Question is how can I access data and inspect it as per ansible documentation this is how we store results of a call. I am not sure of the final step which is to check the status.
This works for me.
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: result
until: result.json.status == "UP"
retries: 10
delay: 30
Notice that result is a ansible dictionary and when you set return_content=yes the response is added to this dictionary and is accessible using json key
Also ensure you have indented the task properly as shown above.
You've made the right first step by saving the output into a variable.
The next step is to use either when: or failed_when: statement in your next task, which will then switch based on the contents of the variable. There are a whole powerful set of statements for use in these, the Jinja2 builtin filters, but they are not really linked well into the Ansible documentation, or summarised nicely.
I use super explicitly named output variables, so they make sense to me later in the playbook :) I would probably write yours something like:
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: sonar_web_api_status_output
- name: do this thing if it is NOT up
shell: echo "OMG it's not working!"
when: sonar_web_api_status_output.stdout.find('UP') == -1
That is, the text "UP" is not found in the variable's stdout.
Other Jinja2 builtin filters I've used are:
changed_when: "'<some text>' not in your_variable_name.stderr"
when: some_number_of_files_changed.stdout|int > 0
The Ansible "Conditionals" docs page has some of this info. This blog post was also very informative.
As per documentation at https://docs.ansible.com/ansible/latest/modules/uri_module.html
Whether or not to return the body of the response as a "content" key in the dictionary result. Independently of this option, if the reported Content-type is "application/json", then the JSON is always loaded into a key called json in the dictionary results.
---
- name: Example of JSON body parsing with uri module
connection: local
gather_facts: true
hosts: localhost
tasks:
- name: Example of JSON body parsing with uri module
uri:
url: https://jsonplaceholder.typicode.com/users
method: GET
return_content: yes
status_code: 200
body_format: json
register: data
# failed_when: <optional condition based on JSON returned content>
- name: Print returned json dictionary
debug:
var: data.json
- name: Print certain element
debug:
var: data.json[0].address.city