Ansible: creating variables from JSON response - json

I am struggling on understanding how to get the response from one uri call to pass to another. I am creating a playbook for my storage system. It requires a SessionKey for all REST calls passed in a custom header "X-HP3PAR-WSAPI-SessionKey"
The initial authentication REST call works succeeds and I see the SessionKey. I am unable to get the SessionKey to be passed to the subsequent call.
Any suggestions?
---
- name: Connect to 3par
hosts: localhost
become: no
vars:
auth_3par_user: "3paruser"
auth_3par_password: "3parpass"
ip_address_3par: 1.1.1.1
rest_api_url_3par: "https://{{ ip_address_3par }}:8080/api/v1"
tasks:
- name: check if 3par WSAPI is running
uri:
url: "{{ rest_api_url_3par }}/credentials"
force_basic_auth: yes
method: POST
HEADER_Content-Type: application/json
body_format: json
body: "{ 'user': '{{ auth_3par_user }}', 'password': '{{ auth_3par_password }}' }"
status_code: 201
return_content: yes
validate_certs: no
register: output
- name: Parsing key
debug:
msg: "{{ output.json.key }}"
- name: GET 3par volumes
uri:
url: "{{ rest_api_url_3par }}/volumes"
method: GET
HEADER_Content-Type: "application/json"
headers:
"X-HP3PAR-WSAPI-SessionKey": "{{ output.json.key }}"
status_code: 200
return_content: yes
validate_certs: no
register: volume_output
- name: Parsing Volumes GET
debug:
var={{ volume_output.stdout | from_json }}
TASK [Parsing key] *************************************************************
task path: /home/chris/ansible/ansible_3par/playbooks/connect_3par.yml:26
ok: [localhost] => {
"msg": "0-f150f34f975693b3c95efa84b13bd605-00cc315a"
}
TASK [check if 3par WSAPI is running] ******************************************
task path: /home/chris/ansible/ansible_3par/playbooks/connect_3par.yml:30
ESTABLISH LOCAL CONNECTION FOR USER: chris
127.0.0.1 EXEC ( umask 22 && mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1513213268.76-220322452758647 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1513213268.76-220322452758647 )" )
127.0.0.1 PUT /tmp/tmpgOtWjh TO /home/chris/.ansible/tmp/ansible-tmp-1513213268.76-220322452758647/uri
127.0.0.1 EXEC LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/chris/.ansible/tmp/ansible-tmp-1513213268.76-220322452758647/uri; rm -rf "/home/chris/.ansible/tmp/ansible-tmp-1513213268.76-220322452758647/" > /dev/null 2>&1
fatal: [localhost]: FAILED! => {"changed": false, "connection": "close", "content": "{\"code\":6,\"desc\":\"invalid session key\"}",
Ansible version 2.0.0.2

You are mixing old and new syntax for headers in uri module.
And as you use an antiquated Ansible 2.0 version, headers parameter is completely ignored.
Upgrade to the current Ansible version.
For reference:
if you use Ansible < 2.1 define all headers using HEADER_*,
if you use Ansible >= 2.1 define all headers under headers.

Posting the corrected, functional code for reference after fixing headers and updating Ansible to latest version.
---
- name: Connect to 3par
hosts: localhost
become: no
vars:
auth_3par_user: "3paruser"
auth_3par_password: "3parpass"
ip_address_3par: 1.1.1.1
rest_api_url_3par: "https://{{ ip_address_3par }}:8080/api/v1"
tasks:
- name: check if 3par WSAPI is running
uri:
url: "{{ rest_api_url_3par }}/credentials"
method: POST
headers:
Content-Type: "application/json"
body_format: json
body: "{ 'user': '{{ auth_3par_user }}', 'password': '{{ auth_3par_password }}' }"
status_code: 201
return_content: yes
validate_certs: no
register: output
- name: Parsing key
debug:
msg: "{{ output.json.key }}"
- name: GET 3par volumes
uri:
url: "{{ rest_api_url_3par }}/volumes"
method: GET
headers:
Content-Type: "application/json"
X-HP3PAR-WSAPI-SessionKey: "{{ output.json.key }}"
Accept: "application/json"
status_code: 200
return_content: yes
validate_certs: no
register: volume_output
- name: Parsing Volumes GET
debug:
msg: "{{ volume_output }}"enter code here

Related

ansible - parsing json w jmesquery

I'm querying Splunk via the uri module w a certain search:
- name: splunk query
uri:
url: https://localhost:8089/servicesNS/admin/search/search/jobs/export?output_mode=json
method: POST
user: ...
password: ...
validate_certs: false
body_format: form-urlencoded
return_content: true
headers:
Content-Type: application/json
body:
- [ search, "{{ splunk_search }}" ]
vars:
- splunk_search: '| ...'
register: me
Splunk returns a 200, w content:
TASK [debug] ********************************************************************************************************************************************************************************************
ok: [host1] => {
"msg": "{\n \"title\": \"clientMessageId\",\n \"app\": \"whatever\"\n}\n{\n \"title\": \"a_title\",\n \"app\": \"some_app\"\n}\n{\n \"title\": \"another_title\",\n \"app\": \"my_app\"\n}\n{\n \"title\": \"title_for_app\",\n \"app\": \"another_app\"\n}"
}
However, I can't properly parse the output, I've tried:
- name: query result from content
debug:
msg: "{{ me.content | json_query('title') }}"
In this case, Ansible will return an empty string. I assume something is wrong with the formatting of the output.
Q: How can I configure Ansible to properly parse the json output, so that a new list is created based on the output.
Example:
my_list:
- title: clientMessageId
app: whatever
- title: a_title
app: some_app
...
What I did was change the url output_mode in which Splunk returns the content from json to csv.
url: https://localhost:8089/servicesNS/admin/search/search/jobs/export?output_mode=csv
Then, I wrote the output to a file and read it using the ansible read_csv module.
- name: write received content to csv file on controller
copy:
content: "{{ me.content }}"
dest: "./file.csv"
mode: '0775'
delegate_to: localhost
- name: read csv file
read_csv:
path: ./file.csv
delegate_to: localhost
register: read_csv
- name: set fact to create list
set_fact:
the_output: "{{ read_csv.list }}"

How to POST Json file with arguments using Ansible

All
I am using Ansible to POST data on a website. I have taken a very simple example of the JSON content
{
"test2":"{{test}}"
}
Below is snippet for ansible task responsible to POST data:
vars:
test: "somerandomtest"
tasks:
- name: Test
register: json_output
uri:
url: "http://localhost:3000"
validate_certs: no
follow_redirects: none
headers:
Authorization: Bearer 12344
Content-Type: application/json
method: POST
body_format: json
body: " {{ lookup('file','test.json') | from_json }} "
#body: '{"test2":" {{test}} "}'
timeout: 600
But on the web service I see the below data, where "test" is not replaced with "somerandomtest".
{'test2': '{{test}}'}
Not sure what I am doing wrong, or maybe I am using lookup the wrong way. If I replace the body with the content everything is fine, but the real JSON I work with is 400 lines, and I want to keep it separate.
Can anyone please let me know what can I do to change the JSON data using Ansible?
The file lookup doesn't evaluate file content for Jinja template markers (which would lead to unexpected behavior if a document just happened to contain {{).
If you want to read in a template, use the template lookup instead:
- hosts: localhost
gather_facts: false
tasks:
vars:
test: "somerandomtest"
tasks:
- name: Test
register: json_output
uri:
url: "https://httpbin.org/post"
validate_certs: no
follow_redirects: none
headers:
Content-Type: application/json
method: POST
body_format: json
body: " {{ lookup('template','test.json') }} "
return_content: true
timeout: 600
- debug:
var: json_output
This shows that the content sent to the remote server is:
{'test2': 'somerandomtest'}
Put the hash into a dictionary and format it to_json. For example
- hosts: localhost
vars:
test: somerandomtest
body:
test2: "{{ test }}"
tasks:
- debug:
msg: "{{ body|to_json }}"
gives the JSON
msg: '{"test2": "somerandomtest"}'
The template gives the same result
shell> cat test.json
{
"test2": "{{ test }}"
}
- debug:
msg: "{{ lookup('template','test.json')|to_json }}"

Multiline json string in ansible task

In an ansible task I am trying to break a long JSON string in a HTTP POST request into multiple lines based on:
In YAML, how do I break a string over multiple lines?
I have tried:
- name: "Test POST request"
uri:
url: "{{ HOST }}/api/"
method: POST
return_content: yes
body: >-
"{\"id\":\"{{ app_id }}\",
\"name\":\"prefix-{{ name }}\",
\"type\":\"ds\",
\"typeLogoUrl\":\"\",
\"access\":\"all\",
\"url\":\"{{ HOST_URL }}",
\"password\":\"\",
\"user\":\"\",
\"database\":\"\",
\"jsonData\":{\"a\":\"{{ a_var }}\",\"b\":true,\"c\":\"{{ c_var }}\"},
\"secureJsonFields\":{}}"
body_format: json
user: "{{ user }}"
password: "{{ password }}"
force_basic_auth: yes
headers:
Content-Type: application/json
But when I run it I get errors, summarized below:
[{\"classification\":\"DeserializationError\",\"message\":\"invalid character '\\\\n' in string literal\"}
Any suggestion on how to break this down into multiple without above error?
You specified body_format: json, so you can write your body in yaml
body:
id: "{{ org_id }}"
name: "prefix-{{ namespace }}"
type: datasource
typeLogoUrl: ""
access: proxy
url: "{{ HOST_URL }}"
password: ""
user:""
database: ""
jsonData:
a: "{{ a_var }}"
b: true
c: "{{ c_var }}"
secureJsonFields: ""

Ansible: loop on files to do POST requests

I'm trying to invoke some REST API and do some POST requests to the service with Ansible. Since the body (JSON) changes, I'm trying to do a loop on some files. Here is the playbook:
- hosts: 127.0.0.1
any_errors_fatal: true
tasks:
- name: do post requests
uri:
url: "https://XXXX.com"
method: POST
return_content: yes
body_format: json
headers:
Content-Type: "application/json"
X-Auth-Token: "XXXXXX"
body: "{{ lookup('file', "{{ item }}" ) }}"
with_file:
- server1.json
- server2.json
- proxy.json
But when I run the playbook, I get this error:
the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'item' is undefined
Where is the problem?
The main problem is that with_ directive should belong to a task dictionary (one indentation level up).
The second problem is that you should use either with_items with file lookup, or simply "{{ item }}" with with_files:
- name: do post requests
uri:
url: "https://XXXX.com"
method: POST
return_content: yes
body_format: json
headers:
Content-Type: "application/json"
X-Auth-Token: "XXXXXX"
body: "{{ item }}"
with_files:
- server1.json
- server2.json
- proxy.json
or
- name: do post requests
uri:
url: "https://XXXX.com"
method: POST
return_content: yes
body_format: json
headers:
Content-Type: "application/json"
X-Auth-Token: "XXXXXX"
body: "{{ lookup('file', item) }}"
with_items:
- server1.json
- server2.json
- proxy.json
Besides, {{ ... }} construct is not a required way to refer to each and every variable -- it is a construct opening a Jinja2 expressions inside which you use variables. For a single variable it indeed becomes: {{ variable }}, but once you open it, you don't need to do it again, so it's perfectly fine to write:
body: "{{ lookup('file', item) }}"

Ansible: access json field and use it as variable

I have to send some PUT requests to a server, at the URL: https://XXXX.com/id, and passing as the body some json files (item1.json, item2.json ...).
- name: invoke service
uri:
url: "https://XXXX.com/{{ item.id }}"
method: PUT
return_content: yes
body_format: json
headers:
Content-Type: "application/json"
X-Auth-Token: "XXXXXX"
body: "{{ lookup('file', item) }}"
with_items:
- item1.json
- item2.json
- item3.json
The id parameter of the url is within the respective json file. The structure of the json files is the following:
{
"address": "163.111.111.111",
"id": "ajsaljlsaaj",
"server": "dnnwqkwnlqkwnldkwqldn"
}
The code I have written seems not to work, and I get 'ansible.vars.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'id'.
So how can that field been accessed?
The issue is that in the following line:
url: "https://XXXX.com/{{ item.id }}"
The value of item is the JSON file name as defined in with_items, not the content of the JSON file.
The quickest fix is to open and parse the JSON file the same way you do in your body: declaration:
- name: invoke service
uri:
url: "https://XXXX.com/{{ ( lookup('file', item)|from_json ).id }}"
method: PUT
return_content: yes
body_format: json
headers:
Content-Type: "application/json"
X-Auth-Token: "XXXXXX"
body: "{{ lookup('file', item) }}"
with_items:
- item1.json
- item2.json
- item3.json
One nicer solution would be to use the with_file: directive instead of with_items.
with_file would automatically open and read the file content, so no need to call lookup anymore:
- name: Provision the Docker Swarm managers
hosts: localhost
tags: provision
gather_facts: False
become: True
tasks:
- name: invoke service
uri:
url: "https://XXXX.com/{{ (item|from_json).id }}"
method: PUT
return_content: yes
body_format: json
headers:
Content-Type: "application/json"
X-Auth-Token: "XXXXXX"
body: "{{ item }}"
with_file:
- item1.json
- item2.json
- item3.json