Parse json response from curl in github action - json

I have the curl request as shown below and also attached the response of curl request from the response how can I fetch the value which is under "status.azure.resource_name" and store it in some variable as I'm new to GitHub action facing some challenges if this is some programming api I could have resolved it
Request:
curl --location --request PUT $URL \
--header "$AUTH_HEADER" \
--header 'Content-Type: application/json' \
--data-raw "$PAYLOAD"
Response:
{
"labels": {},
"spec": {
"mysql": {
"version": "8.0",
"sku": {
"name": "GP_Gen5_4"
},
"storage_profile": {
"storage_mb": 5120
}
},
"key_vault": {
"access_policies": [
{
"name": "test",
"type": "group",
"project": null
}
]
}
},
"type": "azure-mysql",
"name": "mysql",
"id": "1234",
"created_at": "2012-03-04T10:00:05+00:00",
"updated_at": "2012-03-04T10:00:05+00:00",
"project": {
"id": "ae3dfa99",
"name": "Test",
"url": "www.google.com",
"geography": "in"
},
"links": {
"key_vault": {
"endpoint": {
"url": "",
"description": "test.",
"display_name": "test"
},
"azure_portal": {
"url": "test",
"description": "Link to the resource in the Azure portal.",
"display_name": "Key Vault Azure Portal"
}
},
"azure_portal": {
"url": "test",
"description": "Link to the resource in the Azure portal.",
"display_name": "Azure Portal"
},
"endpoint": {
"url": "test",
"description": "Azure resource endpoint.",
"display_name": "test"
}
},
"url": "test",
"tags": {
"cost_center_id": "471000",
"customer": "internal",
"product_group": "internal",
"environment_type": "test",
"budget_category": "",
"team": ""
},
"spiffe_id": "test",
"status": {
"ready": false,
"state": "reconciling",
"deployment": {
"steps": {}
},
"azure": {
"resource_name": "test",
"id": null,
"subscription_id": "test",
"resource_group": "test"
},
"key_vault": {
"access_policies": []
}
}
}

There are multiple ways to achieve that.
You can use directly jq and read variable in bash step.
You can use one of existing open source actions from marketplace:
- name: get nested property
id: format_script
uses: notiz-dev/github-action-json-property#release
with:
path: 'yourjson.json'
prop_path: 'status.azure.resource_name'
- run: echo ${{steps.format_script.outputs.prop}}
Download just one property from whole JSON if you don't need a file itself:
- uses: senmu/download-json-property-action#v1.0.0
with:
url: 'https://httpbin.org/json'
property_path: status.azure.resource_name

Related

Change json object with jq

how can I update servers object in my json using jq?
"tags": [],
"servers": [],
"components": {
"securitySchemes": {
"cookie": {
"type": "apiKey",
"in": "cookie",
"name": "store",
"description": "Store API"
}
},
I need to have servers like that:
"servers": [
{
"url": "http://localhost:5000",
"description": "Localhost server"
}
],
echo '{"tags": [],
"servers": [],
"components": {
"securitySchemes": {
"cookie": {
"type": "apiKey",
"in": "cookie",
"name": "store",
"description": "Store API"
}
}
}}' | jq '.servers = [{url:"http://localhost:5000", description: "Localhost server"}]'
Or, for a fixed json file (adding missing }):
jq '.servers = [{url:"http://localhost:5000", description: "Localhost server"}]' file.json

Fiware Context Provider hello world not working

I'm trying to register a Context Provider as source for several KPIs.
So far, it seems registering might be working, as GET http://{{orion}}/v2/registrations returns something similar to what I set in creation:
{
// each registering a new id is returned: "id": "60991a887032541f4539a71d",
"description": "City Inhabitants",
"dataProvided": {
"entities": [
{
"id": "city.inhabitants"
//,"type": "KeyPerformanceIndicator"
}
]
},
"provider": {
"http": {
"url": "http://myhost/v2/inhabitants"
}
//, "legacyForwarding": false //perhaps there's a bug, cause although unset or set to false, broker still returns true.
}
}
However GET http://{{orion}}/v2/entities/city.inhabitants results in:
{
"error": "BadRequest",
"description": "Service not found. Check your URL as probably it is wrong."
}
GET http://{{orion}}/v2/entities?type=KeyPerformanceIndicator returns [] and context-provider is not being invoked any way.
I'm coding a Node.js application from scratch, to better understand what's going on https://github.com/FIWARE/tutorials.Context-Providers, and using tutorial containers as actual Fiware Broker, so all stores/shelf/products are working but not my KPI.
Using the tutorial application where a microservice is listening on the http://context-provider:3000/random/weatherConditions POST endpoint.
The following registration:
curl -L -X POST 'http://localhost:1026/v2/registrations' \
-H 'Content-Type: application/json' \
--data-raw '{
"description": "Get Weather data for KPI 1",
"dataProvided": {
"entities": [
{
"id" : "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator"
}
],
"attrs": [
"temperature", "relativeHumidity"
]
},
"provider": {
"http": {
"url": "http://context-provider:3000/random/weatherConditions"
},
"legacyForwarding": false
},
"status": "active"
}'
Will forward and retrieve data for the following request:
curl -L -X GET 'http://localhost:1026/v2/entities/urn:ngsi-ld:KPI:010'
or
curl -L -X GET 'http://localhost:1026/v2/entities/?type=KeyPerformanceIndicator'
Returns:
{
"id": "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator",
"temperature": {
"type": "Number",
"value": 11,
"metadata": {}
},
"relativeHumidity": {
"type": "Number",
"value": 39,
"metadata": {}
}
}
The context broker is forwarding the request to the http://context-provider:3000/random/weatherConditions POST end-point with the following payload:
{
"entities": [
{
"id": "urn:ngsi-ld:KPI:010",
"type": "KeyPerformanceIndicator"
}
],
"attrs": [
"temperature",
"relativeHumidity"
]
}

Parse and filter complex JSON Response in Python (the easy compute method)

I have a list of dictionaries (basically JSON Response of an endpoint)
I would need to Parse this 16000 lines of json objects and fitler the documents/objects which match criteria that
whose leaf element/field : statusInfo/status in not "UP" and of those filtered objects, just return "name" , "serviceUrl","status"
example :
"ADMIN-V1" "http://aws-ec2.aws.com:4435" "Warning"
I have been researching about JSONPath module , but there is no good documentation about it, and I could not find any easier way.
Any guidance is highly appreciated.
here is a snippet from long 16000 lines of JSON response.
[
{
"id": "9c108ec5",
"name": "USER-V2",
"managementUrl": "http://aws-ec2.aws.com:5784/",
"healthUrl": "http://aws-ec2.aws.com:5784/health",
"serviceUrl": "http://aws-ec2.aws.com:5784/",
"statusInfo": {
"status": "UP",
"timestamp": 1566663146681,
"details": {
"description": " Eureka Discovery Client",
"status": "UP"
}
},
"source": "discovery",
"metadata": {},
"info": {
"component": "user",
"description": "User REST Resource",
"version": "2.2.1",
"git": {
"commit": {
"time": "07/27/2018 # 15:06:55 CDT",
"id": "b2a1b37"
},
"branch": "refs/tags/v2.2.1"
}
}
},
{
"id": "1a381f20",
"name": "ADMIN-V1",
"managementUrl": "http://aws-ec2.aws.com:4435/",
"healthUrl": "http://aws-ec2.aws.com:4435/health",
"serviceUrl": "http://aws-ec2.aws.com:4435/",
"statusInfo": {
"status": "Warning",
"timestamp": 1566663146682,
"details": {
"description": "Spring Cloud Eureka Discovery Client",
"status": "Warning"
}
},
"source": "discovery",
"metadata": {},
"info": {
"description": "Exchange Admin REST Resource",
"api": {
"version": "1.2.1",
"name": "admin",
"link": "https://app.swaggerhub.com/apis/AWSExchange/admin/1.2.1"
},
"implementation": "admin",
"version": "1.1.0",
"git": {
"commit": {
"time": "01/04/2019 # 15:36:48 UTC",
"id": "39d5551"
},
"branch": "refs/tags/v1.1.0"
}
}
}
]
If your json file contains one big array, you'll want to stream that file in truncating out the array. Then use fromstream/1 to rebuild the objects and filtering them out as you go.
I don't have a representative file to test out the performance myself, but give this a try:
$ jq --stream -n 'fromstream(1|truncate_stream(inputs))
| select(.statusInfo.status != "UP")
| .name, .serviceUrl, .statusInfo.status
' input.json

make requestBody fields optional in swaggerUI

Using the following versions for my flask python project:
connexion==2.3.0
swagger-ui-bundle==0.0.5
OAS 3.0
I want to make all the requestBody fields optional, right now I have defined the json as shown below and I have assumed if you do not provide required field in the requestBody, then it should be taken as false and if I do not provide any values in swagger UI for these fields it should NOT generate curl request with those fields with -d option.
"openapi": "3.0.0",
"info": {
"description": "Dev",
"version": "1.0.0",
"title": "DEV-API",
"contact": {
"email": "dev#email.com"
},
"license": {
"name": "Dev",
"url": "https://opensource.org/licenses/MIT"
}
},
"servers": [
{
"url": "http://xx.yy.zz.a:8080"
}
],
"tags": [
{
"name": "Custom Event Post Request",
"description": "Example API for posting custom events"
}
],
"paths": {
"/api/v1/calls/{id}/{event-name}": {
"post": {
"tags": [
"Post Event"
],
"summary": "Post Custom Event to a call",
"operationId": "post_call_custom_event_data",
"parameters": [
{
"name": "id",
"in": "path",
"schema": {
"type": "string"
},
"required": true
},
{
"name": "event-name",
"in": "path",
"schema": {
"type": "string"
},
"required": true
}
],
"requestBody": {
"content": {
"application/x-www-form-urlencoded":{
"schema": {
"type": "object",
"properties": {
"event1": {
"type":"boolean"
},
"event2":{
"type": "string"
},
"event3": {
"type":"string"
}
}
},
"encoding": {
"event2": {
"allowReserved": true
},
"event3": {
"allowReserved": true
}
}
}
}
},
"responses": {
"200": {
"description": "ok"
},
"400": {
"description": "Failed. Bad Post Data"
}
}
}
}
}
But it does generate with no Values as follows:
curl -X POST "http://xx.yy.zz.a:8080/api/v1/calls/5454/custom-event" -H "accept: /" -H "Content-Type: application/x-www-form-urlencoded" -d "event1=&event2=&document="
I am not sure how to make the requestBody fields/elements to appear as optional in swagger UI
Your API definition is correct. It's a limitation of Swagger UI that it always sends all form fields, including optional fields with empty values. This issue is tracked here:
https://github.com/swagger-api/swagger-ui/issues/5303

Post a json body with swagger

I would like to POST a json body with Swagger, like this :
curl -H "Content-Type: application/json" -X POST -d {"username":"foobar","password":"xxxxxxxxxxxxxxxxx", "email": "foo#bar.com"}' http://localhost/user/register
Currently, I have this definition :
"/auth/register": {
"post": {
"tags": [
"auth"
],
"summary": "Create a new user account",
"parameters": [
{
"name": "username",
"in": "query",
"description": "The username of the user",
"required": true,
"type": "string"
},
{
"name": "password",
"in": "query",
"description": "The password of the user",
"required": true,
"type": "string",
"format": "password"
},
{
"name": "email",
"in": "query",
"description": "The email of the user",
"required": true,
"type": "string",
"format": "email"
}
],
"responses": {
"201": {
"description": "The user account has been created",
"schema": {
"$ref": "#/definitions/User"
}
},
"default": {
"description": "Unexpected error",
"schema": {
"$ref": "#/definitions/Errors"
}
}
}
}
}
But the data are sent in the URL. Here the generated curl provided by Swagger :
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' 'http://localhost/user/register?username=foobar&password=password&email=foo%40bar.com'
I understand that the query keywork is not good, but I didn't find the way to POST a JSON body. I tried formData but it didn't work.
You need to use the body parameter:
"parameters": [
{
"in": "body",
"name": "body",
"description": "Pet object that needs to be added to the store",
"required": false,
"schema": {
"$ref": "#/definitions/Pet"
}
}
],
and #/definitions/Pet is defined as a model:
"Pet": {
"required": [
"name",
"photoUrls"
],
"properties": {
"id": {
"type": "integer",
"format": "int64"
},
"category": {
"$ref": "#/definitions/Category"
},
"name": {
"type": "string",
"example": "doggie"
},
"photoUrls": {
"type": "array",
"xml": {
"name": "photoUrl",
"wrapped": true
},
"items": {
"type": "string"
}
},
"tags": {
"type": "array",
"xml": {
"name": "tag",
"wrapped": true
},
"items": {
"$ref": "#/definitions/Tag"
}
},
"status": {
"type": "string",
"description": "pet status in the store",
"enum": [
"available",
"pending",
"sold"
]
}
},
"xml": {
"name": "Pet"
}
},
Ref: https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/test/resources/2_0/petstore.json#L35-L43
OpenAPI/Swagger v2 spec: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#parameter-object
For OpenAPI v3 spec, body parameter has been deprecated. To define the HTTP payload, one needs to use the requestBody instead, e.g. https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/test/resources/3_0/petstore.json#L39-L41
OpenAPI v3 spec: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#requestBodyObject