Looking to extract href value for each flavor from below json when rel==self and I just can't work it out
{
"flavors": [
{
"id": "1",
"links": [
{
"href": "http://127.0.0.1:8764/v2.1/flavors/1",
"rel": "self"
},
{
"href": "http://127.0.0.1:8764/flavors/1",
"rel": "bookmark"
}
],
"name": "m1.tiny"
},
{
"id": "2",
"links": [
{
"href": "http://127.0.0.1:8764/v2.1/flavors/2",
"rel": "self"
},
{
"href": "http://127.0.0.1:8764/flavors/2",
"rel": "bookmark"
}
],
"name": "m1.small"
},
{
"id": "3",
"links": [
{
"href": "http://127.0.0.1:8764/v2.1/flavors/3",
"rel": "self"
},
{
"href": "http://127.0.0.1:8764/flavors/3",
"rel": "bookmark"
}
],
"name": "m1.medium"
}
]
}
I have tried below but just cant get the jmesquery: 'flavors.links[?contains(rel, self)]' query to work?
- name:
uri:
url: 'http://192.168.1.182:8774/v2.1/flavors'
method: GET
body_format: json
headers:
X-Auth-Token: "{{ ansible_facts.auth_token }}"
Content-Type: "application/json; charset=UTF-8"
return_content: yes
register: this
- debug:
msg: "{{ this }}"
- name: save the Json data to a Variable as a Fact
set_fact:
jsondata: "{{ this }}"
- debug:
msg: "{{ jsondata }}"
- name: save the Json data to a Variable as a Fact
set_fact:
flavors: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'json.flavors'
- debug:
msg: "{{ flavors }}"
- name: LINKS
set_fact:
links: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'flavors.links[?contains(rel, `self`)]'
- debug:
msg: "{{ links }}"
I can print the values for all href using below...
- name: save the Json data to a Variable as a Fact
set_fact:
href: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'json.flavors[*].links[].href'
- debug:
msg: "{{ href }}"
Any help appreciated?
Actually, I have eventually worked it out...
- name: save the Json data to a Variable as a Fact
set_fact:
href: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: json.flavors[].links[?rel==`self`].href
- debug:
msg: "{{ href }}"
Related
Im trying to pull values from a json file but I do not want values that are null? How can I easily skip null values?
I am just trying to find the easiest and simplest solution to parsing JSON into YALM and not have formatting errors in YAML. Maybe I am over complicating it.
JSON:
{
"configuration": {
"protocols" : {
"bgp" : {
"group" : [
{
"name" : "IBGP",
"neighbor" : [
{
"name" : "192.168.1.2",
"description" : "router-2"
}
]
},
{
"name" : "EBGP",
"neighbor" : [
{
"name" : "192.168.2.2",
"description" : "router-3",
"local-address" : "192.168.2.1",
"peer-as" : "65555"
}
]
}
]
}
}
}
}
Play:
- name: parse bgp
set_fact:
bgp: "{{ read | to_json | from_json | json_query(bgp_var) }}"
vars:
bgp_var: 'configuration.protocols.bgp.group[].{group_name:name,neighbors:[{peer_ip:neighbor[].name | [0], peer_description:neighbor[].description | [0], peer_as:neighbor[]."peer-as" | [0] }]}'
- copy:
content: >-
routing_protocols:
bgp:
{{ bgp | to_nice_yaml(indent=2, width=1337, sort_keys=False) | indent(4) }}
dest: "{{ inventory_hostname }}_routing_protocols.yaml"
Returns:
routing_protocols:
bgp:
- group_name: IBGP
neighbors:
- peer_as: null # < Is there a way to skip this anytime a value is null?
peer_description: router-2
peer_ip: 192.168.1.2
- group_name: EBGP
neighbors:
- peer_as: '65555'
peer_description: router-3
peer_ip: 192.168.2.2
you could use this playbook with the parameter omit:
tasks:
- name: set var
set_fact:
bgp: >-
{{ bgp | d([]) +
[ {'group_name': _gpname,
'neighbors': [{'peer_description': _desc, 'peer_ip': _peerip, 'peer_as': _peeras}]
}
] }}
loop: "{{ configuration.protocols.bgp.group }}"
vars:
_gpname: "{{ item.name }}"
_desc: "{{ item.neighbor.0.description }}"
_peeras: "{{ item.neighbor[0]['peer-as'] | d(omit)}}"
_peerip: "{{ item.neighbor.0.name }}"
- name: displ
debug:
msg: "{{ bgp }}"
result:
ok: [localhost] => {
"msg": [
{
"group_name": "IBGP",
"neighbors": [
{
"peer_description": "router-2",
"peer_ip": "192.168.1.2"
}
]
},
{
"group_name": "EBGP",
"neighbors": [
{
"peer_as": "65555",
"peer_description": "router-3",
"peer_ip": "192.168.2.2"
}
]
}
]
}
You can make it in one go in JMESPath, but that would require a double query on the property neighbour, as JMESPath cannot omit a property as Ansible can (for that, see #Frenchy's answer).
So, you would have to have list of list, into which, the first sub list would consist of the neighbour that would contain the key peer-as, and the second sub list, the one that would not contain it. Then you will have to flatten this list of list.
So, here is the copy task:
- copy:
content: >-
{{
read
| to_json
| from_json
| json_query(query)
| to_nice_yaml(indent=2)
}}
dest: "{{ inventory_hostname }}_routing_protocols.yaml"
vars:
query: >-
{
routing_protocols: {
bgp: configuration.protocols.bgp.group[].{
group_name: name,
neighbors: [
neighbor[?"peer-as"].{
peer_description: description,
peer_ip: name,
peer_as: "peer-as"
},
neighbor[?"peer-as" == null].{
peer_description: description,
peer_ip: name
}
]|[]
}
}
}
After this task, you will end with a YAML file containing your expected:
routing_protocols:
bgp:
- group_name: IBGP
neighbors:
- peer_description: router-2
peer_ip: 192.168.1.2
- group_name: EBGP
neighbors:
- peer_as: '65555'
peer_description: router-3
peer_ip: 192.168.2.2
I have the below json data file:
[
{
"?xml": {
"attributes": {
"encoding": "UTF-8",
"version": "1.0"
}
}
},
{
"domain": [
{
"name": "mydom"
},
{
"domain-version": "12.2.1.3.0"
},
{
"server": [
{
"name": "AdminServer"
},
{
"ssl": {
"name": "AdminServer"
}
},
{
"listen-port": "12400"
},
{
"listen-address": "mydom.myserver1.mybank.com"
}
]
},
{
"server": [
{
"name": "SERV01"
},
{
"log": [
{
"name": "SERV01"
},
{
"file-name": "/web/bea_logs/domains/mydom/SERV01/SERV01.log"
}
]
},
{
"listen-port": "12401"
},
{
"listen-address": "mydom.myserver1.mybank.com"
},
{
"server-start": [
{
"java-vendor": "Sun"
},
{
"java-home": "/web/bea/platform1221/jdk"
}
]
}
]
},
{
"server": [
{
"name": "SERV02"
},
{
"log": [
{
"name": "SERV02"
},
{
"file-name": "/web/bea_logs/domains/mydom/SERV02/SERV02.log"
}
]
},
{
"listen-port": "12401"
},
{
"listen-address": "mydom.myhost2.mybank.com"
},
{
"server-start": [
{
"java-home": "/web/bea/platform1221/jdk"
} ]
}
]
}
]
}
]
I wish to display all the server names and their respective port numbers.
Below is my failed attempt to display all the server names viz
AdminServer
SERV01
SERV02
My playbook:
tasks:
- name: Read the JSON file content in a variable
shell: "cat {{ playbook_dir }}/tmpfiles/{{ Latest_Build_Number }}/testme.json"
register: result
- name: Server Names
set_fact:
servernames: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: '*.domain.server[*].name'
- name: Server Names and Ports
set_fact:
serverinfo: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: '*.server[*].[name, port]'
- name: Print all server names
debug:
msg: "{{ item}}"
with_items:
- "{{ servernames }}"
I also tried the below:
jmesquery: 'domain.server[*].name'
There is no error but no data in the output as well. Output below:
TASK [Print all server names] *********************************************************************************
Monday 21 February 2022 03:07:47 -0600 (0:00:00.129) 0:00:03.590 *******
ok: [localhost] => (item=) => {
"msg": ""
}
Can you please suggest how can I get the desired data?
lot of solutions, one
with jmespath, you could try this:
tasks:
- name: Read the JSON file content in a variable
shell: "cat testme.json"
register: result
- name: jsondata
set_fact:
jsondata: "{{ result.stdout | from_json }}"
- name: Server Names
set_fact:
servernames: "{{ servernames | default([]) + [dict(name=item[0], port=item[1])] }}"
loop: "{{ jsondata | json_query(jmesquery0) | zip(jsondata | json_query(jmesquery1)) | list }}"
vars:
jmesquery0: '[].domain[].server[].name'
jmesquery1: '[].domain[].server[]."listen-port"'
- name: debug result
debug:
msg: "{{ servernames }}"
result:
ok: [localhost] => {
"msg": [
{
"name": "AdminServer",
"port": "12400"
},
{
"name": "SERV01",
"port": "12401"
},
{
"name": "SERV02",
"port": "12401"
}
]
}
Due to the nature of your data being in lists, you'll have to resort to conditionals in order to get rid of the empty objects and single item lists that would otherwise pollute your data:
[].domain[?server].server, to get the objects having a property server
[?name].name | [0] to get the name
[?"listen-port"]."listen-port" | [0] to get the port
So, a valid JMESPath query on your data would be
[].domain[?server]
.server[]
.{
name: [?name].name | [0],
port: [?"listen-port"]."listen-port" | [0]
}
And in Ansible, with that single JMESPath query, given that the file is on the controller:
- debug:
var: >-
lookup(
'file',
playbook_dir ~ '/tmpfiles/' ~ Latest_Build_Number ~ '/testme.json'
)
| from_json
| json_query('
[].domain[?server]
.server[]
.{
name: [?name].name | [0],
port: [?"listen-port"]."listen-port" | [0]
}
')
vars:
Latest_Build_Number: 1
This would yield
TASK [debug] *************************************************************************
ok: [localhost] =>
? |-
lookup(
'file',
playbook_dir ~ '/tmpfiles/' ~ Latest_Build_Number ~ '/testme.json'
) | from_json | json_query('
[].domain[?server]
.server[]
.{
name: [?name].name | [0],
port: [?"listen-port"]."listen-port" | [0]
}
')
: - name: AdminServer
port: '12400'
- name: SERV01
port: '12401'
- name: SERV02
port: '12401'
If the file is on the nodes and not on the controller, then, you can either slurp the files first or resort to cat, as you did before applying the same JMESPath query.
I have been trying to update the values in a dictionary in destination json with the values in the dictionary in source JSON file. Below is the example of source and destination JSON file:
Source file:
[
{
"key": "MYSQL",
"value": "456"
},
{
"key": "RDS",
"value": "123"
}
]
Destination File:
[
{
"key": "MYSQL",
"value": "100"
},
{
"key": "RDS",
"value": "111"
},
{
"key": "DB1",
"value": "TestDB"
},
{
"key": "OS",
"value": "EX1"
}
]
Expectation in destination file after running Ansible playbook:
[
{
"key": "MYSQL",
"value": "**456**"
},
{
"key": "RDS",
"value": "**123**"
},
{
"key": "DB1",
"value": "TestDB"
},
{
"key": "OS",
"value": "EX1"
}
]
Below is the playbook I have tried so far, but this only updates the value if it is hard coded:
- hosts: localhost
tasks:
- name: Parse JSON
shell: cat Source.json
register: result
- name: Save json data to a variable
set_fact:
jsondata: "{{result.stdout | from_json}}"
- name: Get key names
set_fact:
json_key: "{{ jsondata | map(attribute='key') | flatten }}"
- name: Get Values names
set_fact:
json_value: "{{ jsondata | map(attribute='value') | flatten }}"
# Trying to update the destination file with only the values provided in source.json
- name: Replace values in json
replace:
path: Destination.json
regexp: '"{{ item }}": "100"'
replace: '"{{ item }}": "456"'
loop:
- value
The main goal is to update the value in destination.json with the value provided in source.json.
In Ansible, the couple key/value tends to be handled with the filters dict2items and items2dict. And your use case can be handled by those filters.
Here would be the logic:
Read both files
Convert both files into dictionaries, with dict2items
Combine the two dictionaries, with the combine filter
Convert the dictionary back into a list with items2dict
Dump the result in JSON back into the file
Given the playbook:
- hosts: localhost
gather_facts: no
tasks:
- shell: cat Source.json
register: source
- shell: cat Destination.json
register: destination
- copy:
content: "{{
destination.stdout | from_json | items2dict |
combine(
source.stdout | from_json | items2dict
) | dict2items | to_nice_json
}}"
dest: Destination.json
We end up with Destination.json containing:
[
{
"key": "MYSQL",
"value": "456"
},
{
"key": "RDS",
"value": "123"
},
{
"key": "DB1",
"value": "TestDB"
},
{
"key": "OS",
"value": "EX1"
}
]
Without to knowing the structure of your destination file it's difficult to use a regex.
I suggest you to load your destination file in a variable, do the changes and save the content of variable to file.
This solution does the job:
- hosts: localhost
tasks:
- name: Parse JSON
set_fact:
source: "{{ lookup('file', 'source.json') | from_json }}"
destination: "{{ lookup('file', 'destination.json') | from_json }}"
- name: create new json
set_fact:
json_new: "{{ json_new | d([]) + ([item] if _rec == [] else [_rec]) | flatten }}"
loop: "{{ destination }}"
vars:
_rec: "{{ source | selectattr('key', 'equalto', item.key) }}"
- name: save new json
copy:
content: "{{ json_new | to_nice_json }}"
dest: dest_new.json
Result -> dest_new.json:
ok: [localhost] => {
"msg": [
{
"key": "MYSQL",
"value": "456"
},
{
"key": "RDS",
"value": "123"
},
{
"key": "DB1",
"value": "TestDB"
},
{
"key": "OS",
"value": "EX1"
}
]
}
I'm trying to filter a list of attributes from the below output. selection and value attributes need to be listed.
Output:
ok: [192.0.0.3] => {
"allow": "GET, PUT",
"attempts": 1,
"invocation": {
}
},
"json": {
"body": [
{
"selection": "fqdn",
"value": "airbus3.org"
},
{
"selection": "fqdn",
"value": "airbus4.org"
}
],
"key": "ntp_servers",
"meta": {
"transaction": "/api/transaction"
}
},
}
I'm trying to use the below task to filter but it's not working.
- set_fact:
ntp_details: "{{ item.selection }}"
with_items: "{{ reg_ntp.json.body | json_query('[*]') }}"
Please advise..
For example
- set_fact:
ntp_details: "{{ reg_ntp.json.body|json_query('[].selection') }}"
should give
ntp_details:
- fqdn
- fqdn
I use elb_application_lb_info module to get info about my application load balancer. Here is the code I am using for it:
- name: Test playbook
hosts: tag_elastic_role_logstash
vars:
aws_access_key: AKIARWXXVHXJS5BOIQ6P
aws_secret_key: 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
tasks:
- name: Gather information about all ELBs
elb_application_lb_info:
aws_access_key: AKIXXXXXXXXXXXXXXXXXXX
aws_secret_key: gG6a586XXXXXXXXXXXXXXXXXX
region: ap-southeast-2
names:
- LoadBalancer
register: albinfo
- debug:
msg: "{{ albinfo }}"
This is working fine and I got the following output:
"load_balancers": [
{
"idle_timeout_timeout_seconds": "60",
"routing_http2_enabled": "true",
"created_time": "2021-01-26T23:58:27.890000+00:00",
"access_logs_s3_prefix": "",
"security_groups": [
"sg-094c894246db1bd92"
],
"waf_fail_open_enabled": "false",
"availability_zones": [
{
"subnet_id": "subnet-0195c9c0df024d221",
"zone_name": "ap-southeast-2b",
"load_balancer_addresses": []
},
{
"subnet_id": "subnet-071060fde585476e0",
"zone_name": "ap-southeast-2c",
"load_balancer_addresses": []
},
{
"subnet_id": "subnet-0d5f856afab8f0eec",
"zone_name": "ap-southeast-2a",
"load_balancer_addresses": []
}
],
"access_logs_s3_bucket": "",
"deletion_protection_enabled": "false",
"load_balancer_name": "LoadBalancer",
"state": {
"code": "active"
},
"scheme": "internet-facing",
"type": "application",
"load_balancer_arn": "arn:aws:elasticloadbalancing:ap-southeast-2:117557247443:loadbalancer/app/LoadBalancer/27cfc970d48501fd",
"access_logs_s3_enabled": "false",
"tags": {
"Name": "loadbalancer_test",
"srg:function": "Storage",
"srg:owner": "ISCloudPlatforms#superretailgroup.com",
"srg:cost-centre": "G110",
"srg:managed-by": "ISCloudPlatforms#superretailgroup.com",
"srg:environment": "TST"
},
"routing_http_desync_mitigation_mode": "defensive",
"canonical_hosted_zone_id": "Z1GM3OXH4ZPM65",
"dns_name": "LoadBalancer-203283612.ap-southeast-2.elb.amazonaws.com",
"ip_address_type": "ipv4",
"listeners": [
{
"default_actions": [
{
"target_group_arn": "arn:aws:elasticloadbalancing:ap-southeast-2:117557247443:targetgroup/test-ALBID-W04X8DBT450Q/c999ac1cda7b1d4a",
"type": "forward",
"forward_config": {
"target_group_stickiness_config": {
"enabled": false
},
"target_groups": [
{
"target_group_arn": "arn:aws:elasticloadbalancing:ap-southeast-2:117557247443:targetgroup/test-ALBID-W04X8DBT450Q/c999ac1cda7b1d4a",
"weight": 1
}
]
}
}
],
"protocol": "HTTP",
"rules": [
{
"priority": "default",
"is_default": true,
"rule_arn": "arn:aws:elasticloadbalancing:ap-southeast-2:117557247443:listener-rule/app/LoadBalancer/27cfc970d48501fd/671ad3428c35c834/5b5953a49a886c03",
"conditions": [],
"actions": [
{
"target_group_arn": "arn:aws:elasticloadbalancing:ap-southeast-2:117557247443:targetgroup/test-ALBID-W04X8DBT450Q/c999ac1cda7b1d4a",
"type": "forward",
"forward_config": {
"target_group_stickiness_config": {
"enabled": false
},
"target_groups": [
{
"target_group_arn": "arn:aws:elasticloadbalancing:ap-southeast-2:117557247443:targetgroup/test-ALBID-W04X8DBT450Q/c999ac1cda7b1d4a",
"weight": 1
}
]
}
}
]
}
],
"listener_arn": "arn:aws:elasticloadbalancing:ap-southeast-2:117557247443:listener/app/LoadBalancer/27cfc970d48501fd/671ad3428c35c834",
"load_balancer_arn": "arn:aws:elasticloadbalancing:ap-southeast-2:117557247443:loadbalancer/app/LoadBalancer/27cfc970d48501fd",
"port": 9200
}
],
"vpc_id": "vpc-0016dcdf5abe4fef0",
"routing_http_drop_invalid_header_fields_enabled": "false"
}
]
I need to fetch "dns_name" which is dns name of the load balancer and pass it in another play as a variable.
I tried with json_query but got the error. Here is the code:
- name: save the Json data to a Variable as a Fact
set_fact:
jsondata: "{{ albinfo.stdout | from_json }}"
- name: Get ALB dns name
set_fact:
dns_name: "{{ jsondata | json_query(jmesquery) }}"
vars:
jmesquery: 'load_balancers.dns_name'
- debug:
msg: "{{ dns_name }}"
And here is the error:
"msg": "The task includes an option with an undefined variable. The error was: Unable to look up a name or access an attribute in template string ({{ albinfo.stdout | from_json }}).\nMake sure your variable name does not contain invalid characters like '-': the JSON object must be str, bytes or bytearray
Any idea how to extract "dns_name" from the json above?
Here is the way to get the dns_name from above json output:
- name: Get Application Load Balancer DNS Name
set_fact:
rezultat: "{{ albinfo | json_query('load_balancers[*].dns_name') }}"
- debug:
msg: "{{ rezultat }}"