parse json response using ansible - json

I need to parse the json response from an API request using ansible and then take action on items returned in the response. Here is part of my ansible playbook -
```- name: Invoke Import API
uri:
url: "{{ atl_bitbucket_dataset_url }}/rest/api/1.0/migration/imports"
user: admin
password: "{{ atl_bitbucket_admin_password }}"
method: POST
follow_redirects: yes
force_basic_auth: yes
creates: "{{ atl_product_home_shared }}/data/migration/import/lock.file"
body: "{ \"archivePath\": \"{{ atl_bitbucket_dataset_url | basename }}\" }"
body_format: json
return_content: yes
register: response
until: response.status == 200
retries: 6
delay: 15
failed_when: response.response.json.state != 'INITIALISING'
- name: get status of import
debug: var=response```
and here is the json response i got on a previous run
```TASK [bitbucket_dataset_restore : get status of import] ************************
ok: [localhost] => {
"response": {
"attempts": 1,
"cache_control": "no-cache, no-transform",
"changed": false,
"connection": "close",
"content": "{\"id\":1,\"initiator\":{\"name\":\"admin\",\"emailAddress\":\"admin#yourcompany.com\",\"id\":1,\"displayName\":\"AdminIstrator\",\"active\":true,\"slug\":\"admin\",\"type\":\"NORMAL\",\"links\":{\"self\":[{\"href\":\"http://bbdc-test-loadbala-t6vnlr2363vl-1404860112.us-west-2.elb.amazonaws.com/users/admin\"}]}},\"nodeId\":\"e72aa995-5016-4b2c-80e8-edba5eda3ab4\",\"progress\":{\"percentage\":0},\"startDate\":1574803309090,\"state\":\"INITIALISING\",\"type\":\"com.atlassian.bitbucket.migration.import\",\"updateDate\":1574803309090}",
"content_type": "application/json;charset=UTF-8",
"cookies": {},
"cookies_string": "",
"date": "Tue, 26 Nov 2019 21:21:49 GMT",
"elapsed": 1,
"failed": false,
"failed_when_result": False,
"json": {
"id": 1,
"initiator": {
"active": True,
"displayName": "AdminIstrator",
"emailAddress": "admin#yourcompany.com",
"id": 1,
"links": {
"self": [
{
"href": "http://bbdc-test-loadbala-t6vnlr2363vl-1404860112.us-west-2.elb.amazonaws.com/users/admin"
}
]
},
"name": "admin",
"slug": "admin",
"type": "NORMAL"
},
"nodeId": "e72aa995-5016-4b2c-80e8-edba5eda3ab4",
"progress": {
"percentage": 0
},
"startDate": 1574803309090,
"state": "INITIALISING",
"type": "com.atlassian.bitbucket.migration.import",
"updateDate": 1574803309090
},
"msg": "OK (unknown bytes)",
"redirected": False,
"status": 200,
"transfer_encoding": "chunked",
"url": "http://localhost:7990/rest/api/1.0/migration/imports",
"vary": "accept-encoding,x-auserid,cookie,x-ausername,accept-encoding",
"warnings": [
"The value True (type bool) in a string field was converted to u'True' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."
],
"x_arequestid": "#XXAH7Kx1281x1x0",
"x_asen": "SEN-500",
"x_auserid": "1",
"x_ausername": "admin",
"x_content_type_options": "nosniff"
}
}```
I want to retrieve the value "state": "INITIALISING" and the "id": 1. I have tried various methods response.response.json.state or response['response']['json']['state']. And it has not worked. Can anyone help me on how i can retrieve those values so i can use in other tasks further down?

I was able to fix this by adding is defined. So my code looks like this -
failed_when: output is defined and output.json is defined and output.json.state != 'INITIALISING'

Related

Cypress login to REST endpoint is unauthorized, Postman works - upload of JSON file

We have a REST service with an incoming REST endpoint that accepts data. It has no web interface (Swagger or so), only the API. With Postman I can POST a JSON file (response code is 202) to it and then read the uploaded data from another endpoint.
When I want to log in to the same endpoint with Cypress to upload a JSON file from the fixtures folder (with the same body as in the Postman request), then I get response code 401 instead – Unauthorized. I have the feeling that the cypress request is wrong because the logfile of the service does not write a message when I use the cypress POST but it does when I use the Postman POST.
First question: What could I be doing wrong in the cypress request?
Second question: Once the authentication works, how can I POST/upload/push the content of the JSON file to that endpoint? Because I have no webpage to interact with, I cannot use click button functions. The documentation mainly deals with interpreting a JSON response but not with sending it.
My cypress code:
it('logs in to connector through REST API', () => {
cy.request({
method: 'POST',
url: 'localhost:8095/connector/demands/v1/demandData',
failOnStatusCode:false,
form: true,
body: {
Username: 'user',
Password: 'pass',
}
})
})
import my-request from '../fixtures/my-request.json'
it('loads the JSON file', () => {
cy.fixture('my-request.json')
})
The structure of the JSON file to upload is not too simple, here is a shortened version:
{
"#metadata": {
"context": "{{A}}"
},
"pool": "{{B}}",
"action": "NEW",
"Type": "ANNOUNCEMENT",
"ON": "Order123",
"PON": "PO123",
"SNN": "SN123",
"direction": "OUT",
"mode": 3,
"pack": [
{
"out": {
"outKey": "OUT14",
"outQuantity": "3",
"dimension": {
"length": "303",
"width": "33",
"height": "903",
"unit": "mm"
},
"layers": "3",
"weight": "3000",
"weightUnit": "grm",
"in": [
{
"inKey": "IN12",
"inQuantity": "3",
"article": {
"articleKey": "article3",
"quantity": "300",
"PON": "Art_PO300",
"SNN": "Art_SN300"
}
}
]
},
"p1": "pack3",
"p2": "pack4",
"store": true
},
{
"out": {
"outKey": "OUT23",
"outQuantity": "5",
"dimension": {
"length": "505",
"width": "55",
"height": "905",
"unit": "mm"
},
"layers": "5",
"weight": "5000",
"weightUnit": "grm",
"in": [
{
"inKey": "IN19",
"inQuantity": "5",
"article": {
"articleKey": "article5",
"quantity": "500",
"PON": "Art_PO500",
"SNN": "Art_SN500"
}
}
]
},
"p1": "pack5",
"p2": "pack5",
"store": true
}
]
}
Solution found. "form: true" must not be given because this overrides the content-type.
You can pass the contents of the fixture file(which is json) in the request body like this:
describe('Some Test Suite', function() {
// we can use these values to log in
const username = 'jane.lane'
const password = 'password123'
it('logs in to connector through REST API', () => {
cy.fixture('my-request.json').then(myFixture => {
cy.request({
method: 'POST',
url: 'localhost:8095/connector/demands/v1/demandData',
auth: {
username,
password,
},
failOnStatusCode: false,
form: true,
body: myFixture
})
})
})
})
For HTTP auth you have to use. You can check out this cypress recipe.
auth: {
username,
password,
}

How to update existing JSON with a smaller subset JSON in Ansible?

I have this task:
- name: Get Status page url
uri:
url: "{{ responseversion.json.results[0]._links.self }}?expand=version,body.storage"
method: GET
return_content: yes
body_format: json
force_basic_auth: yes
headers:
Authorization: Basic blablab
register: responseurl
- name: print body without updates
debug: var=responseurl
- debug:
msg: "{{ responseurl | combine({'json':{'body':{ 'version':{'minorEdit':true, 'number':responseversion.json.results[0].version.number+1} }}}) }}"
Basically I get a response, get the page "version" and "minorEdit" keys from response and replace them and their values in the original response. There are other keys besides "version", I need to keep them also, as well as the neighboring keys inside "version" key.
The targeted "version" key before update looks like this:
"version": {
"_expandable": {
"content": "/rest/api/content/164794022"
},
"_links": {
"self": "https://hostname.com/confluence/rest/experimental/content/164794022/version/4"
},
"by": {
"_expandable": {
"status": ""
},
"_links": {
"self": "https://hostname.com/confluence/rest/api/user?key=ff80808164c5cf240164c6e351d30002"
},
"displayName": "my name",
"profilePicture": {
"height": 48,
"isDefault": false,
"path": "/confluence/download/attachments/98107443/user-avatar",
"width": 48
},
"type": "known",
"userKey": "ff80808164c5cf240164c6e351d30002",
"username": "my name"
},
"hidden": false,
"message": "",
"minorEdit": false,
"number": 4,
"when": "2021-08-25T15:36:35.285+03:00"
}
The final debug task print the version key like this:
"json": {
"body": {
"version": {
"minorEdit": true,
"number": 5
}
}
}
You can see the JSON for replacement inline, inside the "combine" function call. Whole JSON key gets overriden.

How to read a value from json output

Basically, I need to read a value from a JSON output and use it in subsequent tasks. So, I tried with_items, loop, but nothing worked.
Sample JSON that I generated from a registered variable:
TASK [local_volume_mount : debug Info from device that is parted] **************
Monday 29 March 2021 21:33:39 +0000 (0:00:02.271) 0:00:02.417 **********
ok: [node1] => {
"partitioned_device_live_info": {
"changed": false,
"msg": "All items completed",
"results": [
{
"ansible_loop_var": "item",
"changed": false,
"disk": {
"dev": "/dev/nvme2n1",
"logical_block": 512,
"model": "SAMSUNG MZQLW960HMJP-00003",
"physical_block": 512,
"size": 937692504.0,
"table": "msdos",
"unit": "kib"
},
"failed": false,
"invocation": {
"module_args": {
"align": "optimal",
"device": "/dev/nvme2n1",
"flags": null,
"label": "msdos",
"name": null,
"number": 1,
"part_end": "100%",
"part_start": "0%",
"part_type": "primary",
"state": "present",
"unit": "KiB"
}
},
"item": [
{
"device": "/dev/nvme2n1",
"partitions": [
{
"end": "100%",
"number": 1,
"start": "0%",
"storage_class": "ssd-wkr-services"
}
]
},
{
"end": "100%",
"number": 1,
"start": "0%",
"storage_class": "ssd-wkr-services"
}
],
"partitions": [
{
"begin": 1024.0,
"end": 937692160.0,
"flags": [],
"fstype": "",
"name": "",
"num": 1,
"size": 937691136.0,
"unit": "kib"
}
],
"script": ""
}
]
}
}
So, from the above I need to read the partitions.num value and use it in the next task, but, I don't know how to do that.
Task:
- name: THIS IS BEING TESTED
debug:
var: "{{ item.partitions }}"
ignore_errors: no
#loop: "{{ partitioned_device_live_info.results }}"
with_items: "{{ partitioned_device_live_info.results }}"
Output of the above task is
'dict object' has no attribute 'partitions'
I want to store that value item.partitions.num in a variable and then use it in further tasks.
Figured out using msg, below is my change
- name: THIS IS BEING TESTED with MSG and with_items
debug:
msg: "{{ item.partitions[0].num }}"
with_items: "{{ partitioned_device_live_info.results }}"
You can always use regex in ansible to set facts. More details stackoverflow
Following are some use cases
- name: Long form task does not
ansible.builtin.replace:
path: /etc/hosts
regexp: '\b(localhost)(\d*)\b'
replace: '\1\2.localdomain\2 \1\2'
- name: Explicitly specifying positional matched groups in replacement
ansible.builtin.replace:
path: /etc/ssh/sshd_config
regexp: '^(ListenAddress[ ]+)[^\n]+$'
replace: '\g<1>0.0.0.0'
Setting fact
- name: set version in file after replacement
set_fact:
version_in_file: "{{ version_deployment_file | regex_search(docker_image_version) }}"
More details - ansible.builtin.replace

cant use fo query json because of \" in variables

I have this result from ansible:
{
"count": 745,
"next": "/api/v2/inventories/xxxx/hosts/?page=160",
"previous": "/api/v2/inventories/xxxx/hosts/?page=14",
"results": [
{
"id": 3932944,
"type": "host",
"url": "/api/v2/hosts/39329/",
"created": "2021-02-16T20:21:58.104406Z",
"modified": "2021-03-16T23:59:28.688226Z",
"name": "machine1",
"description": "imported",
"enabled": true,
"instance_id": "",
"variables": "{\"VirtualMachine\": \"100\", \"CloneFrom\": \"mrzzzzz\", \"BuildingMachine\": \"*\", , \"__DailyCost\": \"{\\\"type\\\":\\\"TimeRate\\\",\\\"cost\\\":{\\\"type\\\":\\\"money\\\",\\\"currencyCode\\\":\\\"dz\\\",\\\"amount\\\":1.274599998125},\\\"basis\\\":{\\\"type\\\":\\\"timeSpan\\\",\\\"unit\\\":\\\"DAYS\\\",\\\"amount\\\":1}}\", \"virtualMachineState\": \"On\", \"vmDescription\": \"testj\"}",
"has_active_failures": false,
"has_inventory_sources": true,
"last_job": 862532,
"last_job_host_summary": 6309369,
},
{
"id": 3932945,
"type": "host",
"url": "/api/v2/hosts/39329/",
"created": "2021-02-16T20:21:58.104406Z",
"modified": "2021-03-16T23:59:28.688226Z",
"name": "machine2",
"description": "imported",
"enabled": true,
"instance_id": "",
"variables": "{\"VirtualMachine\": \"100\", \"CloneFrom\": \"mrzzzzz\", \"BuildingMachine\": \"*\", , \"__DailyCost\": \"{\\\"type\\\":\\\"TimeRate\\\",\\\"cost\\\":{\\\"type\\\":\\\"money\\\",\\\"currencyCode\\\":\\\"dz\\\",\\\"amount\\\":1.274599998125},\\\"basis\\\":{\\\"type\\\":\\\"timeSpan\\\",\\\"unit\\\":\\\"DAYS\\\",\\\"amount\\\":1}}\", \"virtualMachineState\": \"On\", \"vmDescription\": \"testj\"}",
"has_active_failures": false,
"has_inventory_sources": true,
"last_job": 862532,
"last_job_host_summary": 6309369,
},
]
}
I can get the value of results.name when use json_query, but I can't get value of results.varaibles.VirtualMachine for example using json,query because of '"' , so How can I format variables to be a json format , In other world I want someting like :
...
"variables": {
"VirtualMachine": "100",
"CloneFrom": "mrzzzzz",
"BuildingMachine": "*",
"DailyCost": "{"type":"TimeRate","cost":
{"type":"money","currencyCode":"dz","amount":1.274599998125},
"basis":{"type":"timeSpan","unit":"DAYS","amount":1}}",
"virtualMachineState": "On",
"vmDescription": "testj"
}
...
thank for your help at advance.
Update:
I used this Idea posted by #mdaniel (thanks a lot):
{{whatever.results | map(attribute='variables') | map('from_json') | list}}
Iget output that I need, but when I tried to filter like that:
- name: get id, name, hq_os, hq_site and hq_zone
debug:
var: dictresult_format | json_query(jmesquery)
vars:
jmesquery: "[?VirtualMachine: `100`].{From: CloneFrom, description: vmDescription}"
it output nothing :
TASK [task1t : get data from variables*********
Wednesday 17 March 2021 11:28:53 -0400 (0:00:00.049) 0:00:06.842 *******
ok: [hosttower] => {
"dictresult_format | json_query(jmesquery)": ""
}
Can I missed somting, because that "{{whatever.results | map(attribute='variables') | map('from_json') | list}}" return a list not a json?

How i can delimite entity join level for JSON parse

I'm new to web application development with PHP, Symfony (Bundles) and Angular
we develop a web application, works over API rest building on symfony and we make request through http protocol, every response is coded in JSON.
Every JSON correspond to doctrine entity.
We use FosRest to parse entity to JSON, here my config:
fos_rest:
param_fetcher_listener: true
routing_loader:
default_format: json
include_format: json
view:
view_response_listener: 'force'
formats:
xml: false
json: true
templating_formats:
html: false
format_listener:
rules: ~
exception:
codes:
'Symfony\Component\Routing\Exception\ResourceNotFoundException': 404
'Doctrine\ORM\OptimisticLockException': HTTP_CONFLICT
messages:
'Symfony\Component\Routing\Exception\ResourceNotFoundException': true
allowed_methods_listener: true
access_denied_listener:
json: true
body_listener: true
service:
inflector: app.util.inflector
Doctrine config
doctrine:
dbal:
driver: pdo_sqlsrv #pdo_mysql
host: "%database_host%"
port: "%database_port%"
dbname: "%database_name%"
user: "%database_user%"
password: "%database_password%"
charset: UTF8
orm:
auto_generate_proxy_classes: "%kernel.debug%"
naming_strategy: doctrine.orm.naming_strategy.underscore
auto_mapping: true
metadata_cache_driver: apc
query_cache_driver: apc
The generated JSON API is correct, here is an example
{
"id": 1,
"username": "admin",
"email": "admin#admin.com",
"enabled": true,
"last_login": "2016-07-04T16:31:07-0500",
"name": "admin",
"first_lastName": "admin",
"second_lastName": "admin",
"id_program": {
"id": 1,
"abbreviation": "LRI",
"nomenclature": "LRI",
"description": "Licenciatura en Relaciones Internacionales",
"consecutive": "1",
"cve_dgp": "3453454355",
"first_period": "2016-06-09T10:30:00-0500",
"campus": "HARVARD",
"mod": "Escolarizado",
"number": "863676476",
"register_date": "2016-06-09T10:30:00-0500",
"ref_number": "0976555",
"require_pay": true,
"colour_rgb": "253545454",
"last_modification": "2016-06-17T12:51:28-0500",
"id_level": {
"id": 1,
"cve": "LIC",
"description": "Licenciatura",
"last_modification": "2016-06-11T10:58:44-0500",
"id_status": {
"id": 1008,
"value": "Inactivo",
"active": true,
"id_cat": {
"id": 1004,
"valur": "CatalogosEstatus",
"active": true
}
}
},
"id_front_credenctial_file": {
"id": 193,
"name": "FRENTE-LRI.jpg",
"extention": "jpeg",
"mime_type": "image/jpeg",
"md5": "e5643b567aa6e134ae1abcc23fdb545e",
"is_perfil": false
},
"id_back_file_credential": {
"id": 194,
"name": "REVES-LRI.jpg",
"extention": "jpeg",
"mime_type": "image/jpeg",
"md5": "d490d35f448d3fcb849cc708f7291bc5",
"is_perfil": false
},
"id_status": {
"id": 1007,
"value": "Activo",
"active": true,
"id_cat": {
"id": 1004,
"value": "CatalogosEstatus",
"active": true
}
}
},
"ldap": true,
"rolesdb": [{
"id": 8,
"id_rol": {
"id": 9,
"name": "ADMINISTRADOR",
"description": "Rol Administrador",
"active": true
}
}, {
"id": 9,
"id_rol": {
"id": 10,
"name": "SHOWCASE",
"description": "Rol showcase",
"active": true
}
}, {
"id": 10,
"id_rol": {
"id": 11,
"name": "DESARROLLADOR",
"description": "Rol desarrollador",
"active": true
}
}]}
The problem I have is when an entity has multiple relationships , as you can see in the JSON example , when parsing the entity being added all your relationships (each key that starts with id_ is a mapped relationship with Doctrine) and as our BD has grown considerably , there are times when a JSON has too much information that is hard to read from Angular side .
Is there a way to define (optimize ) until the parser level (JSON ) relations is made in an entity ?
or not include each join of the entity?
regards!!
with #MaxDepth you can steer how deep you want to serialize your hierarchy and with #Groups you can manage which fields should be serialized.
Both can be set in your Serialization Context and are explained in more detail in the FOS Rest Documentation
By that you should be able to achieve what you're looking for.