Ansible - issue with loop and json_query's - json

Question
I need to get the ID from the GET because it's needed in the URL in the PUT task to edit a specific "input" entry. I'm using the Ansible URI to talk to a REST API to manage this.
playbook
*host_vars/host.yml
*
---
inputs:
- title: "test_input_api"
type: "org.graylog2.inputs.syslog.udp.SyslogUDPInput"
global: false
configuration:
allow_override_date: false
bind_address: "0.0.0.0"
expand_structured_data: false
force_rdns: false
number_worker_threads: 8
override_source: null
port: 5999
recv_buffer_size: null
store_full_message: true
- title: "test_input_api_2"
type: "org.graylog2.inputs.syslog.udp.SyslogUDPInput"
global: false
configuration:
allow_override_date: false
bind_address: "0.0.0.0"
expand_structured_data: false
force_rdns: false
number_worker_threads: 8
override_source: null
port: 5998
recv_buffer_size: null
store_full_message: true
playbook.yml
---
- name: Configure system
hosts: graylog
connection: local
gather_facts: no
roles:
- graylog/inputs
roles/graylog/inputs/tasks/main.yml
---
- include_tasks: get_inputs.yml
- include_tasks: put_inputs.yml
roles/graylog/inputs/tasks/get_inputs.yml
---
- name: "API GET System Inputs"
uri:
url: http://{{ ansible_host }}:9000/api/system/inputs
url_username : "{{ system.users.triple_admin.api_token }}"
url_password: token
method: GET
return_content: yes
register: get_graylog_inputs
- name: Set Fact
set_fact:
get_input_id: "{{ get_graylog_inputs.content | from_json | json_query('inputs[?title == `{}`] | [0].id '.format(input.title)) }}"
loop: "{{ inputs }}"
loop_control:
loop_var: input
The registered var from the get show's the following
{
"json": {
"inputs": [
{
"attributes": {
"allow_override_date": "False",
"bind_address": "0.0.0.0",
"expand_structured_data": "False",
"force_rdns": "False",
"number_worker_threads": 8,
"override_source": "",
"port": 5999,
"recv_buffer_size": "",
"store_full_message": "True"
},
"content_pack": null,
"created_at": "2021-07-30T15:21:47.590Z",
"creator_user_id": "triple_admin",
"global": false,
"id": "6104170beca15547502665d6",
"name": "Syslog UDP",
"node": "ba52ad48-0b13-419d-b957-d47d8911b413",
"static_fields": {},
"title": "test_input_api",
"type": "org.graylog2.inputs.syslog.udp.SyslogUDPInput"
},
roles/graylog/inputs/tasks/put_inputs.yml
---
- name: "API PUT System Inputs"
uri:
url: http://{{ ansible_host }}:9000/api/system/inputs/{{ get_input_id }}
url_username : "{{ system.users.triple_admin.api_token }}"
url_password: token
headers:
X-Requested-By: X-Ansible
method: PUT
body_format: json
body: "{{ lookup('template', 'templates/post_template.j2') }}"
status_code: 201
return_content: yes
loop: "{{ inputs }}"
loop_control:
loop_var: input
"ansible_facts": {
"get_input_id": "61015085eca1554750236084",
"get_input_titles": "test_input_api"
},
"ansible_facts": {
"get_input_id": "610282d0eca155475024ac91",
"get_input_titles": "test_input_api_2"
Results of running the play
loop 1 - this needs to be matched to the title and therefor get id "61015085eca1554750236084"
"title": "test_input_api",
"url": "http://192.168.21.82:9000/api/system/inputs/610282d0eca155475024ac91",
loop 2
"title": "test_input_api_2",
"url": "http://192.168.21.82:9000/api/system/inputs/610282d0eca155475024ac91",
All help is welcome !

(not related) You don't need to json_decode the result get_graylog_inputs.content. If the server on the over side sends the correct Content-type: application/json header, you should have a get_graylog_inputs.json entry containing the already decoded json result.
You don't need to loop twice. Remove the set_fact loop (which is not correct anyway) in your first file and use the value from your register directly in the second loop.
You did not show any example of your input data so I have to guess a bit here from your jmespath expression... but you basically don't need json_query at all and can stick to generic core ansible filters.
Here is how I see the solution in the second file once you cleaned-up the first:
---
- name: "API PUT System Inputs"
vars:
get_input_id: "{{ get_graylog_inputs.json.inputs | selectattr('title', '==', input.title) | map(attribute='id') | first }}"
uri:
url: http://{{ ansible_host }}:9000/api/system/inputs/{{ get_input_id }}
url_username : "{{ system.users.triple_admin.api_token }}"
url_password: token
headers:
X-Requested-By: X-Ansible
method: PUT
body_format: json
body: "{{ lookup('template', 'templates/post_template.j2') }}"
status_code: 201
return_content: yes
loop: "{{ inputs }}"
loop_control:
loop_var: input
You will probably have to debug and tune the expression to get the input id as I could not do it myself against an example data structure.

Related

Pass JSON from output for another query Ansible

Please help me figure out what I'm wrong with. I'm getting a JSON from Ansible and filtering it, after which I want to save the output and reuse it. But, unfortunately, I get an error that this attribute does not exist. Where did I go wrong?
playbook code:
var:
query_general: "body.results[].{display_name: display_name, subnets: subnets[]}"
- name: parsing query
set_fact:
myvar: "{{ results | json_query(query_general) }}"
register: output
- name: qwe
set_fact:
scndjson: "{{ output.myvar[].display_name }}"
- name: print
debug:
msg: "{{ scndjson }}"
I tried the json_query second case as well, but that didn't work either.
in register:output i have:
[
{
"display_name": "1test",
"subnets": [
{
"gateway_address": "0.0.0.0/25",
"network": "0.0.0.0/25"
}
]
},
{
"display_name": "test",
"subnets": [
{
"gateway_address": "0.0.0.1/25",
"network": "0.0.0.1/25"
}
]
}
]
error:
The task includes an option with an undefined variable.
it can be: output, display_name, etc
UPD:
I corrected the yaml, there are no errors, but the data is not displayed.
tasks:
- name:
nsxt_rest:
hostname: anyhost
username: anyuser
password: anypass
validate_certs: false
method: get
path: /policy/api/v1/infra/segments
register: nsx_results
- debug:
var: nsx_query_general
vars:
nsx_query_general: "{{ nsx_results | json_query('body.results[].{display_name: display_name, subnets: subnets[]}') }}"
register: output
- debug:
var: secondjson
vars:
secondjson: "{{ output|json_query('[].display_name') }}"
Output from nsx_query_general:
{
"nsx_query_general": [
{
"display_name": "test",
"subnets": [
{
"gateway_address": "0.0.0.0/25",
"network": "0.0.0.0/25"
}
]
},
{
"display_name": "1test",
"subnets": [
{
"gateway_address": "0.0.0.1/25",
"network": "0.0.0.1/25"
}
]
}]}
Output from secondjson:
{
"secondjson": "",
"_ansible_verbose_always": true,
"_ansible_no_log": false,
"changed": false
}
Given the registered variable output
output:
- display_name: 1test
subnets:
- gateway_address: 0.0.0.0/25
network: 0.0.0.0/25
- display_name: test
subnets:
- gateway_address: 0.0.0.1/25
network: 0.0.0.1/25
Either use json_query
scndjson: "{{ output|json_query('[].display_name') }}"
, or map attribute
scndjson: "{{ output|map(attribute='display_name')|list }}"
Both declarations create the list
scndjson: [1test, test]
Example of a complete playbook
- hosts: localhost
vars:
output:
- display_name: 1test
subnets:
- gateway_address: 0.0.0.0/25
network: 0.0.0.0/25
- display_name: test
subnets:
- gateway_address: 0.0.0.1/25
network: 0.0.0.1/25
tasks:
- debug:
var: scndjson
vars:
scndjson: "{{ output|json_query('[].display_name') }}"
- debug:
var: scndjson
vars:
scndjson: "{{ output|map(attribute='display_name')|list }}"

Ansible: filter an item from a list of object in a json file

I have in ansible the following code:
---
- name: "Retrieve ID for group {{onboarding.repo.subGroup}}"
uri:
method: GET
return_content: yes
validate_certs: "{{gitlab_validate_certs}}"
url: "{{ gitlab_api_url }}/api/v4/groups?search={{onboarding.repo.subGroup}}"
headers:
Private-Token: "{{ gitlab_api_token }}"
register: uri_response
- debug:
msg: "{{ uri_response['content'] }}"
- set_fact:
groupID: "{{ uri_response['content'] | from_json | json_query([*].[?name=='{{onboarding.repo.subGroup}}'].id) }}"
The get method retrieve a json list like the following one:
[
{
"id":1,
"name":"name1"
},
{
"id":2,
"name":"name2"
},
...
]
With the set_fact module I am trying to get the id on an item that correspond to a specific name. I have tried many json_query syntax (for instance .[?name=='{{onboarding.repo.subGroup}}'].id or [?name=='{{onboarding.repo.subGroup}}'].id), without success. I saw many different examples online, but they were different because the list was the value of a key, like the following:
key:
[
{...},
{...},
...
]
And I saw different examples working with a query syntax like key.[?name=='{{onboarding.repo.subGroup}}'].id
I cannot find a way to adapt that query to my case. Can anyone help me on finding the right syntax?
Regards,
Giorgio
Q: "Get the id on an item that corresponds to a specific name."
A: Convert the list to a dictionary
content: "{{ uri_response.content|
items2dict(key_name='name', value_name='id') }}"
gives
content:
name1: 1
name2: 2
Use the dictionary
- debug:
msg: "{{ content[item]|default('undef') }}"
loop:
- name1
- name2
- name3
gives (abridged)
msg: '1'
msg: '2'
msg: undef
Example of a complete playbook
- hosts: localhost
vars:
uri_response:
content:
[
{
"id":1,
"name":"name1"
},
{
"id":2,
"name":"name2"
}
]
content: "{{ uri_response.content|items2dict(key_name='name', value_name='id') }}"
tasks:
- debug:
msg: "{{ content[item]|default('undef') }}"
loop:
- name1
- name2
- name3

Retrieve, manipulate and send back JSON in Ansible

We have to set and unset a VLAN configuration to a switch. Fortunately, Unify Controller provide an API, which works with Ansible in general.
However, it is not possible with the API to change a single element, Ansible has to get the current configuration, find and change the desired element and send back the configuration. (I pushed the desired config change. This yielded to the situation that the switch had nothing but my config change, I felt like the Facebook guys...)
---
- hosts: adm01.local
gather_facts: false
tasks:
- name: Use vars from Vault
include_vars: unifi_info.yaml
- name: Get Cookie from Unifi
uri:
url: https://{{ url }}/api/login
method: POST
body_format: json
body: {"username":"{{ username }}","password":"{{ password }}"}
validate_certs: false
register: login
- name: Print returned data to ensure it worked
debug:
msg: "{{ login }}"
- name: Get current config
uri:
url: https://{{ url }}/api/s/default/stat/device/
method: GET
body_format: json
headers:
Cookie: "{{ login.cookies_string }}"
validate_certs: false
register: switchConf
- name: Print SwitchConf
debug:
msg: "{{ switchConf }}"
- name: Set port 2 to CLIENTVLAN
uri:
url: https://{{ url }}/api/s/default/rest/device/60acc79964542d80774123b5/
method: PUT
body_format: json
# DISABLE VLAN: "60aca5ee64542d807741239d"
# CLIENTVLAN: ""60accb9b64542d80774123d1"
#body: {"port_overrides": [{"port_idx": 2, "portconf_id":"60accb9b64542d80774123d1"}]}
headers:
Cookie: "{{ login.cookies_string }}"
validate_certs: false
register: portConf
- name: Print portConf
debug:
msg: "{{ portConf }}"
SwitchConf prints (the relevant part, only):
"msg": {
"access_control_allow_credentials": "true",
"access_control_expose_headers": "Access-Control-Allow-Origin,Access-Control-Allow-Credentials",
"changed": false,
"connection": "close",
"content_type": "application/json;charset=UTF-8",
"cookies": {},
"cookies_string": "",
"date": "Wed, 20 Oct 2021 18:22:24 GMT",
"elapsed": 0,
"failed": false,
"json": {
"data": [
{
(...)
"port_overrides": [
{
"port_idx": 1,
"portconf_id": "A"
},
{
"port_idx": 2,
"portconf_id": "B"
},
{
"port_idx": 3,
"portconf_id": "B"
},
{
"port_idx": 5,
"portconf_id": "B"
},
{
"port_idx": 6,
"portconf_id": "B"
},
{
"port_idx": 7,
"portconf_id": "B"
},
(...)
],
(...)
}
]
},
"msg": "OK (unknown bytes)",
"redirected": false,
"status": 200,
"transfer_encoding": "chunked",
"url": "https://adm01.local:8443/api/s/default/stat/device/",
"vary": "accept-encoding,origin,accept-encoding",
"x_frame_options": "DENY"
}
}
So, the question is: how can Ansible be used to fetch the JSON datagram portConf->data->port_overrides and change the "portconf_id" to "X" with "port_idx"==2, so that post_overrides can be used to POST it back?
The dictionaries in YAML are immutable. You have to create a new one to change anything. Given the simplified data
portConf:
status: 200
json:
data:
- port_overrides:
- port_idx: 1
portconf_id: A
- port_idx: 2
portconf_id: B
- port_idx: 3
portconf_id: C
and the required changes
diff:
port_idx: 2
portconf_id: X
the task below makes the changes in json.data
- set_fact:
_json: "{{ {'data': [{'port_overrides': _port_change}]} }}"
vars:
_port_remove: "{{ portConf.json.data.0.port_overrides|
rejectattr('port_idx', 'eq', diff.port_idx)|
list }}"
_port_change: "{{ (_port_remove + [diff])|sort(attribute='port_idx') }}"
gives
_json:
data:
- port_overrides:
- port_idx: 1
portconf_id: A
- port_idx: 2
portconf_id: X
- port_idx: 3
portconf_id: C
and the next task combines the changes
- set_fact:
portConf: "{{ portConf|combine({'json': _json}) }}"
gives
portConf:
json:
data:
- port_overrides:
- port_idx: 1
portconf_id: A
- port_idx: 2
portconf_id: X
- port_idx: 3
portconf_id: C
status: 200

Ansible: How print number instead string in JSON - module uri

Hi,
I need to print a variable as a number instead a string.
Example:
- name: Create input
uri:
url: "https://{{ url_graylog }}/api/system/inputs"
...
body_format: json
body:
title: "{{ name }}"
configuration:
bind_address: "0.0.0.0"
port: "{{ port }}" <-- its print as string, I need number
global: true
I tried
port: {{ port }} <-- not work
port: "{{ port | int }}" <-- not work
Any idea? Thanks!
Actually it seems not possible to convert jinja template into integer since it always return string to Ansible. Detailed explanation here :
https://github.com/ansible/ansible/issues/9362#issuecomment-302432118
However, I found a workaround consisting of using folded string bloc in yaml. In your case the Ansible task should look like this :
- name: Create inputenter code here
uri:
url: "https://{{ url_graylog }}/api/system/inputs"
...
body_format: json
body: >
{
"title": "{{ name }}",
"configuration": {
"bind_address": "0.0.0.0",
"port": {{ port | int }}
}
}
global: true
It is a little bit less readable but will produce a non-quoted value for port. The body sent looks like this :
...
"body": {
"configuration": {
"bind_address": "0.0.0.0",
"port": 25565
},
"title": "My title"
},
...
Hope it helped !
You can set in ansible.cfg:
jinja2_native = True
Create your variable as such:
variableName:
name: "demo"
capacityUnits: 1
Then body like this:
body: "{{ variableName | to_json }}"
Then the int will persist:
"body": "{\"name\": \"demo\", \"capacityUnits\": 1}"

ansible parse json with several keys with the same name to one list variable

I have an issue to parse a json using ansible
I have a task that connected to rancher and get a json file
task:
- uri:
url: http://rancher.local:8080/v1/hosts
method: GET
user: ##################
password: ################
body_format: json
register: hosts_json
- name: test
set_fact:
rancher_env_hosts: "{{ item.hostname }}"
#when: item.hostname == "*-i-*"
with_items: "{{hosts_json.json.data}}"
- name: output
debug:
msg: "hosts: {{rancher_env_hosts}}"
and I get the following json (after edit it to be more readable):
{
"json": {
"data": [
{
"hostname": "rancher-i-host-02",
"id": "adsfsa"
},
{
"hostname": "rancher-i-host-01",
"id": "gfdgfdg"
},
{
"hostname": "rancher-q-host-01",
"id": "dfgdg"
},
{
"hostname": "rancher-q-host-02",
"id": "dfgdg"
}
]
}
}
When I start the playbook I get only the last host name in the variable and not all the list of hostname. can I register all the list to the same variable?
In addition, I also added a line with the a comment "#" in order to get only the host names that match the string "-i-" bit it's not working. any idea?
This is what filters (and this) for:
- set_fact:
hosts_all: "{{ hosts_json.json.data | map(attribute='hostname') | list }}"
hosts_i: "{{ hosts_json.json.data | map(attribute='hostname') | map('regex_search','.*-i-.*') | select('string') | list }}"
host_all will contain all hostnames, host_i will contain only .*-i-.* matching hostnames.
Try this
- uri:
url: http://rancher.local:8080/v1/hosts
method: GET
user: ##################
password: ################
body_format: json
register: hosts_json
- name: init fact
set_fact:
rancher_env_hosts: "[]"
- name: test
set_fact:
rancher_env_hosts: "{{rancher_env_hosts}} + [ {{item.hostname}} ]"
when: item.hostname | search(".*-i-.*")
with_items: "{{hosts_json.json.data}}"
- name: output
debug:
msg: "hosts: {{rancher_env_hosts}}"
About search you can read here http://docs.ansible.com/ansible/playbooks_tests.html
UPD:
About adding values to array here: Is it possible to set a fact of an array in Ansible?