Ansible Making json_query / loop conditional with a when clause - json

I have some ansible code involving a json_query in a loop that I'm trying to make conditional. But I think there is something I'm not understanding about when clauses and loops.
The following code works fine, when the condition is met (IaC.status == 400), the problem is that when the condition is not met, the 2nd task still runs, the loop tries to process IaC2, which doesn't exist, and the task fails with: "Invalid data passed to 'loop', it requires a list, got this instead:
I think this is actually expected behaviour for conditional statements and loops: (https://docs.ansible.com/ansible/latest/user_guide/playbooks_conditionals.html#using-conditionals-in-loops), but the solution described to skip the entire task is to use an empty iterator, but I have no idea how to merge that in with the json_query() statement.
Does anyone know how I can truly make the 2nd task below conditional?
# The preceding code has made a RESTful API call to create a gitlab group, if the group
# exists the return status is 400, so I need to look it up instead. In which case the
# following code works fine. The problem is when IaC.status = 201 for some reason the loop
# in the 2nd task below tries to run, despite the when clause, and fails
# because the variable IaC2 doesn't exist
- name: or if IaC already exists
when: IaC.status == 400
uri:
url: https://{{new_hostname}}/api/v4/groups
method: GET
headers:
Authorization: "Bearer {{token.json.access_token}}"
body_format: json
body:
name: "IaC"
top_level_only: true
status_code: 200
validate_certs: yes
register: IaC2
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC.status == 400
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id

You should condition your second task on the first task:
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC2 is defined
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
I cannot reproduce your case on ansible 2.9.10 where the task is skipped as expected. But you can also help with a default. I guess this would work:
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC2 is defined
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) | default([]) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id

I have an answer of sorts. Firstly there were two things I didn't fully understand;
the register var (IaC2) is always created even if the uri task is skipped. And IaC2 has a totally different structure depending upon whether this task is executed or skipped.
The loop in the following task is always executed, so the when clause on the second task is pretty meaningless.
The actual problem was the jmesquery looking for the IaC2 variable to have a particular json structure, which it doesn't have in the case where the uri task is skipped.
So the solution that seems to work is checking to see if the uri task was skipped (when: IaC2.skipped is defined), and if it was skipped re-defining IaC2 in a empty list form that doesn't break the loop.
Code examples shown below.
So, it works, but I can't help thinking there must be an easier way of doing this?
- name: If IaC already exists (IaC.status == 400) then get the list of existing groups.
when: IaC.status == 400
uri:
url: https://{{new_hostname}}/api/v4/groups
method: GET
headers:
Authorization: "Bearer {{token.json.access_token}}"
body_format: json
body:
name: "IaC"
top_level_only: true
status_code: 200
validate_certs: no
register: IaC2
# To avoid breaking the following loop that searches the groups in the case that
# the group was created, re-define IaC2 into a form that won't break it.
- name: Not created group "IaC", so redefine IaC2 var so as not to break the following loop
when: IaC2.skipped is defined
set_fact:
IaC2: '{{ {"json": []} }}'
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
#when: IaC.status == 400 << Leave in or comment out, makes no difference
debug:
var: item
loop: "{{ (IaC2 | community.general.json_query(jmesquery)) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
- name: record the IaC group's id (400)
when: IaC.status == 400
set_fact:
IaC_group_id: "{{group_id.results[0].item}}"

Related

How to Loop Parsed Json Response on Ansible

I'm working on a playbook and I'm waiting for this playbook to collect web requests and generate an output. Since I am making requests to multiple interfaces I need to split that. How could I do that?
tasks:
- name: Collecting Endpoints
uri:
url: "https://applicationserver.com/api/app_set"
method: GET
return_content: yes
validate_certs : no
body_format: json
register: Result
- name: Temp to variable.
set_fact:
Endpoint: "{{ Result.json | json_query('[].name[]') }}"
Here is my response from the first task https://paste.chapril.org/?142535f7072d47ed#4TahPDaDGPkP1NxxB3gzZP86SkusJ9Kk6jSfqmSy7wK7
According to the data I have collected from the first task, I want to request each one-by-one filter data from within.
This is the JSON response of any of my endpoints. https://paste.chapril.org/?44e8f5462b105285#E2J7LzRwWSCJXR8BetzB1pewnV2ye1Fr1g741VacwBJb
- name: Get Responses All.
uri:
url: "https://applicationserver.com/api/app_set/{{ item }}"
method: GET
return_content: yes
body_format: json
validate_certs : no
register: Resultsofall
loop: "{{ Endpoint }}"
After this task, I am receiving all Json datas of all endpoints.
How can I filter the data from the whole result and loop like that?
app_sets
Index 1 (Which is counting by endpoint)
AppSetName (Getting from the first task (json_query('[].name[]')
StatefulAppSetName (Getting from the first task( ('[].labels[].AttachedToStatefulAppSet[]'))
Apps (Getting from the second task json_query('[].json.pods.app')
Index 2 (Which is counting by endpoint)
AppSetName (Getting from the first task (json_query('[].name[]')
StatefulAppSetName (Getting from the first task( ('[].labels[].AttachedToStatefulAppSet[]'))
Apps (Getting from the second task json_query('[].json.pods.app')
You need to get your json_query right, which is based on jmespath. Assuming you're interested on the name of the your services in your json your query should be:
- name: Temp to variable.
set_fact:
Endpoint: "{{ Result.json | json_query('services[].app') }}"
If you are interested in services and pods you could de something like
- name: Temp to variable.
set_fact:
services_apps: "{{ Result.json | json_query('services[].app') }}"
pods_apps: "{{ Result.json | json_query('services[].app') }}"
- name: Get Responses All.
uri:
url: "https://applicationserver.com/api/app_set/{{ item }}"
method: GET
return_content: yes
body_format: json
validate_certs : no
register: Resultsofall
loop: "{{ (services_apps + pods_apps )| unique | select | list }}"
I recommend using https://jmespath.org/ to develop and test you queries.

How can I loop through json object REST API body in Ansible?

When using Ansible I am able to execute when passed one-by-one like this:
---
- name: Using a REST API
become: false
hosts: localhost
gather_facts: false
tasks:
- debug:
msg: “Let’s get list of Interfaces”
- name: Adding a Bridge-Interface
uri:
url: https://router/rest/interface/bridge
method: PUT
validate_certs: false
url_username: ansible
url_password: ansible
force_basic_auth: yes
body_format: json
status_code: 201
body: '{"name":"bridge_ansible"}'
register: results
- debug:
var: results
I want to iterate through a set of commands so I thought of looping, but that does not work for me, I am using this code:
---
- name: Using a REST API
become: false
hosts: localhost
gather_facts: false
tasks:
- debug:
msg: “Let’s get list of Interfaces”
- name: Adding a Bridge-Interface
uri:
url: "{{item.url}}"
method: PUT
validate_certs: false
url_username: ansible
url_password: ansible
force_basic_auth: yes
body_format: json
status_code: 201
body: "{{item.body}}"
register: results
loop:
- {body:'{"name":"bridge_ansible"}', url:'https://router/rest/interface/bridge'}
- {body:'{"address":"6.6.6.6", "interface":"bridge_ansible"}', url:'https://router/rest/ip/address'}
- debug:
var: results
I get an error for this {body:'{"name":"bridge_ansible"}', url:'https://router/rest/interface/bridge'} in the json object { I think my syntax is not correct but cannot understand the correct thing. Can someone please help
ERROR! We were unable to read either as JSON nor YAML, these are the errors we got from each:
JSON: Expecting value: line 1 column 1 (char 0)
Syntax Error while loading YAML.
did not find expected ',' or '}'
The error appears to be in '/ansible-playbook/1-demo.yaml': line 23, column 19, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
loop:
- {body:'\{"name":"bridge_ansible"\}', url:'https://router/rest/interface/bridge'}
^ here
This one looks easy to fix. It seems that there is a value started
with a quote, and the YAML parser is expecting to see the line ended
with the same kind of quote. For instance:
when: "ok" in result.stdout
Could be written as:
when: '"ok" in result.stdout'
Or equivalently:
when: "'ok' in result.stdout"
We could be wrong, but this one looks like it might be an issue with
unbalanced quotes. If starting a value with a quote, make sure the
line ends with the same set of quotes. For instance this arbitrary
example:
foo: "bad" "wolf"
Could be written as:
foo: '"bad" "wolf"'
Thanks

Extract part of JSON in ansible playbook

I want to extract a part from a json extra vars input and use this as a variable in further commands.
The extra vars being parsed towards ansible is:
{
"problemUrl": "https://xxxxx.xxxxxxxxx-xxxxx.xxxx/e/58b59a93-xxxx-xxxx-xxxx-91bb5ca1f41c/#problems/problemdetails;pid=-5484403941961857966_1631165040000V2",
}
I want to extract the part -5484403941961857966_1631165040000V2 and store it into a variable.
- name: get pid from URL
set_fact:
pidproblem: "{{ problemUrl | urlsplit('fragment') | regex_search('pid=(.+)', '\\1') }}"
- name: show pid
debug:
var: pidproblem[0]
- name: update problem with output
when: state == "OPEN"
uri:
url: https://xxxxx.xxxxxxxxx-xxxxx.xxxx/e/58b59a93-xxxx-xxxx-xxxx-91bb5ca1f41c/api/v2/problems/"{{ pidproblem[0] }}"/comments
method: POST
headers:
Content-Type: application/json; charset=utf-8
Authorization: Api-Token xxxxx
body_format: json
body: "{\"message\":\"TEST\",\"context\":\"TEST\"}"
Could the issue reside in the fact that the id is subsituded as "6551567569324750926_1631192580000V2" instead of 6551567569324750926_1631192580000V2?
"url": "https://xxxxx.xxxxxxxxx-xxxxx.xxxx/e/58b59a93-xxxx-xxxx-xxxx-91bb5ca1f41c/api/v2/problems/\"6551567569324750926_1631192580000V2\"/comments"
There is a urlsplit filter which can split a URL into known segments. We can use this to break down the URL and get the last fragment, i.e.
"{{ problemUrl | urlsplit('fragment') }}"
Gives...
problems/problemdetails;pid=-5484403941961857966_1631165040000V2
Now this gives us a more "manageable" string. We can do a regex_search (with groups) on this, to get the pid, like:
- name: get pid from URL
set_fact:
pid: "{{ problemUrl | urlsplit('fragment') | regex_search('pid=(-.+)', '\\1') }}"
- name: show pid
debug:
var: pid[0]
- name: update problem with output
uri:
url: "https://xxxxx.xxxxxxxxx-xxxxx.xxxx/e/58b59a93-xxxx-xxxx-xxxx-91bb5ca1f41c/api/v2/problems/{{ pid[0] }}/comments"
# other params
Not super-reliable as we don't know how your url can change, but you could use some regex filter to extract the pid value:
- hosts: localhost
vars:
problemUrl: '{ "problemUrl": "https://xxxxx.xxxxxxxxx-xxxxx.xxxx/e/58b59a93-xxxx-xxxx-xxxx-91bb5ca1f41c/#problems/problemdetails;pid=-5484403941961857966_1631165040000V2;other=false" }'
tasks:
- name: set_fact some paramater
set_fact:
pid: "{{ (problemUrl | from_json).problemUrl | regex_replace('.*pid=(?P<pid>[^;]*).*', '\\g<pid>') }}"
- name: "update"
debug:
msg: "{{ pid }}"

Ansible, error when reading values from JSON response

In my Ansible-Code I have a task which makes some requests to check if a Cluster is created. Response of the Request is a JSON file which contains the values "healthy" and "status" and some more.
This is my Code:
- name: Wait for Elastic Cluster to be ready
uri:
url: https://abcdefbla.{{ lookup('env','ENV') }}.some.url.com/api/v1/clusters/elasticsearch/{{elasticClusterDetails.elasticsearchId}}
method: GET
user: admin
password: "{{rootpw.stdout}}"
force_basic_auth: yes
return_content: yes
validate_certs: no
register: result
until: "{{ (result.stdout | from_json | first).healthy }}" == false and "{{ (result.stdout | from_json | first).status }}" == "started"
retries: 60
delay: 10
The from_json filter helps parsing the response, and the "first" tells that it should use the first key with that name.
When I run it, I get the following error:
ERROR! Syntax Error while loading YAML.
did not find expected key.
The error appears to have been in '/home/ubuntu/MLAAS-V2_USETHIS/mlaas-v2-aws/mls_ece_core/play.yml': line 295, column 65, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
register: result
until: "{{ (result.stdout | from_json | first).healthy }}" == false and "{{ (result.stdout | from_json | first).status }}" == "started"
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
As you see, I quoted the line as suggested. Any Ansible-Expert out there who can tell me what I am missing?!
I found the solution. There was an error with spacing for the URL object. Later, an error with JSON parsing came up which I also fixed. The following code worked for me:
- name: Wait for Elastic Cluster to be ready
uri:
url: https://abcdefbla.{{ lookup('env','ENV') }}.some.url.com/api/v1/clusters/elasticsearch/{{elasticClusterDetails.elasticsearchId}}
method: GET
user: admin
password: "{{rootpw.stdout}}"
force_basic_auth: yes
return_content: yes
validate_certs: no
register: result
until: resultA.json.healthy == false and "started" == "{{ resultA.json.status|lower }}"
retries: 60
delay: 10

How to inspect a json response from Ansible URI call

I have a service call that returns system status in json format. I want to use the ansible URI module to make the call and then inspect the response to decide whether the system is up or down
{"id":"20161024140306","version":"5.6.1","status":"UP"}
This would be the json that is returned
This is the ansible task that makes a call:
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: data
Question is how can I access data and inspect it as per ansible documentation this is how we store results of a call. I am not sure of the final step which is to check the status.
This works for me.
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: result
until: result.json.status == "UP"
retries: 10
delay: 30
Notice that result is a ansible dictionary and when you set return_content=yes the response is added to this dictionary and is accessible using json key
Also ensure you have indented the task properly as shown above.
You've made the right first step by saving the output into a variable.
The next step is to use either when: or failed_when: statement in your next task, which will then switch based on the contents of the variable. There are a whole powerful set of statements for use in these, the Jinja2 builtin filters, but they are not really linked well into the Ansible documentation, or summarised nicely.
I use super explicitly named output variables, so they make sense to me later in the playbook :) I would probably write yours something like:
- name: check sonar web is up
uri:
url: http://sonarhost:9000/sonar/api/system/status
method: GET
return_content: yes
status_code: 200
body_format: json
register: sonar_web_api_status_output
- name: do this thing if it is NOT up
shell: echo "OMG it's not working!"
when: sonar_web_api_status_output.stdout.find('UP') == -1
That is, the text "UP" is not found in the variable's stdout.
Other Jinja2 builtin filters I've used are:
changed_when: "'<some text>' not in your_variable_name.stderr"
when: some_number_of_files_changed.stdout|int > 0
The Ansible "Conditionals" docs page has some of this info. This blog post was also very informative.
As per documentation at https://docs.ansible.com/ansible/latest/modules/uri_module.html
Whether or not to return the body of the response as a "content" key in the dictionary result. Independently of this option, if the reported Content-type is "application/json", then the JSON is always loaded into a key called json in the dictionary results.
---
- name: Example of JSON body parsing with uri module
connection: local
gather_facts: true
hosts: localhost
tasks:
- name: Example of JSON body parsing with uri module
uri:
url: https://jsonplaceholder.typicode.com/users
method: GET
return_content: yes
status_code: 200
body_format: json
register: data
# failed_when: <optional condition based on JSON returned content>
- name: Print returned json dictionary
debug:
var: data.json
- name: Print certain element
debug:
var: data.json[0].address.city