How to print nested JSON array data in a tabular format? - json

I want to read the status of clusters and servers inside it.
Below is the sample json file
"data": [{
"id": 7865,
"timeCreated": 1602589399294,
"timeUpdated": 1602748892149,
"name": "gw-ext-1",
"type": "CLUSTER",
"status": "RUNNING",
"multicastEnabled": false,
"primaryNodeId": 546,
"servers": [{
"id": 768,
"timeCreated": 1602589028419,
"timeUpdated": 1602747941321,
"name": "gw-jpg208765-1",
"type": "SERVER",
"serverType": "GATEWAY",
"status": "RUNNING",
"addresses": [{
"networkInterface": "eng123"
},
{
"networkInterface": "eng124"
}],
"clusterId": 098,
"clusterName": "gw-ext-1",
"currentClusteringPort": 897,
"runtimeInformation": {
"Information": {
"runtime": {
"name": "abctech",
"version": "1.6.8"
},
"specification": {
"vendor": "rrr",
"name": "rrrt",
"version": "1.8.89"
}
},
"osInformation": {
"name": "LX",
"version": "35",
"architecture": "klh"
},
"mExpirationDate": 098765589283662
}
},
{
"id": 876,
"timeCreated": 1602589007370,
"timeUpdated": 1602748894901,
"name": "gw-jpg208765-2",
"type": "SERVER",
"serverType": "GATEWAY",
"mVersion": "3.9.1",
"gaVersion": "3.9.1",
"agentVersion": "1.9.5",
"ExpirationDate": 32521996800000,
"ExpirationDate": 1665661007000,
"status": "DISCONNECTED",
"addresses": [{
"networkInterface": "engg"
},
{
"networkInterface": "engg"
}],
"clusterId": 768,
"clusterName": "gw-ext-1",
"serverPort": 987,
"currentClusteringPort": 987,
"runtimeInformation": {
"abcInfo": {
"runtime": {
"name": "abc",
"version": "1.2.3"
},
"specification": {
"vendor": "RRR",
"name": "RTR",
"version": "1.8.0"
}
},
"osInformation": {
"name": "LX",
"version": "4.78",
"architecture": "eng"
},
"ExpirationDate": 8765478999765
}
}],
"visibilityMap": {
"mapNodes": [{
"serverId": 765,
"visibleNodeIds": [765,
876],
"unknownNodeIps": []
},
{
"serverId": 876,
"visibleNodeIds": [765,
876],
"unknownNodeIps": []
}]
}
},
{
"id": 7865,
"timeCreated": 1602589399294,
"timeUpdated": 1602748892149,
"name": "gw-ext-2",
"type": "CLUSTER",
"status": "RUNNING",
"multicastEnabled": false,
"primaryNodeId": 546,
"servers": [{
"id": 768,
"timeCreated": 1602589028419,
"timeUpdated": 1602747941321,
"name": "gw-jpg208766-1",
"type": "SERVER",
"serverType": "GATEWAY",
"status": "RUNNING",
"addresses": [{
"networkInterface": "eng123"
},
{
"networkInterface": "eng124"
}],
"clusterId": 098,
"clusterName": "gw-ext-2",
"currentClusteringPort": 897,
"runtimeInformation": {
"Information": {
"runtime": {
"name": "abctech",
"version": "1.6.8"
},
"specification": {
"vendor": "rrr",
"name": "rrrt",
"version": "1.8.89"
}
},
"osInformation": {
"name": "LX",
"version": "35",
"architecture": "klh"
},
"mExpirationDate": 098765589283662
}
},
{
"id": 876,
"timeCreated": 1602589007370,
"timeUpdated": 1602748894901,
"name": "gw-jpg208766-2",
"type": "SERVER",
"serverType": "GATEWAY",
"mVersion": "3.9.1",
"gaVersion": "3.9.1",
"agentVersion": "1.9.5",
"ExpirationDate": 32521996800000,
"ExpirationDate": 1665661007000,
"status": "DISCONNECTED",
"addresses": [{
"networkInterface": "engg"
},
{
"networkInterface": "engg"
}],
"clusterId": 768,
"clusterName": "gw-ext-2",
"serverPort": 987,
"currentClusteringPort": 987,
"runtimeInformation": {
"abcInfo": {
"runtime": {
"name": "abc",
"version": "1.2.3"
},
"specification": {
"vendor": "RRR",
"name": "RTR",
"version": "1.8.0"
}
},
"osInformation": {
"name": "LX",
"version": "4.78",
"architecture": "eng"
},
"ExpirationDate": 8765478999765
}
}],
"visibilityMap": {
"mapNodes": [{
"serverId": 765,
"visibleNodeIds": [765,
876],
"unknownNodeIps": []
},
{
"serverId": 876,
"visibleNodeIds": [765,
876],
"unknownNodeIps": []
}]
}
}]
So in each cluster we have two servers and this json continues to have around 15 clusters.
I want to filter out the status of each cluster and server in below format
name cluster/server status
gw-ext-1 CLUSTER RUNNING
gw-jpg208765-1 SERVER RUNNING
gw-jpg208765-2 SERVER DISCONNECTED
similarly for other clusters also.
I tried few things but its not giving me the servers .. it gives only cluster's details
target_id=echo \$targetIdResponse | ${env.WORKSPACE}/jq -r '.data[] | [.name, .type, .status]'
OR
target_id=echo \$targetIdResponse | ${env.WORKSPACE}/jq -r '.data[] | [.name, .type, .status, .servers.name, .servers.type, .servers.status]'
where $targetIdResponse contains my json data
I want to know how i can filter the above json to get the required data.

You need to have the header array the required fields in a separate array and put them together in a tabular format using #tsv
jq -r '[ "name", "cluster/server", "status" ],
( .data[] | [.name, .type, .status] ),
( .data[].servers[] | [ .name, .type, .status ] ) | #tsv'
The requirement was modified since originally posted to have the server information exactly below the cluster information
jq -r '[ "name", "cluster/server", "status" ],
( .data[] | [.name, .type, .status], ( .servers[] | [.name, .type, .status] ) ) | #tsv'

Related

Cannot get jq to query json object [duplicate]

This question already has answers here:
How to use jq when the variable has reserved characters?
(3 answers)
Closed 6 months ago.
I have a JSON file that I am trying to query with jq. I am unable to retrieve the observations. I am trying to retieve each of the "observations using the following command and not able to get to the result:
cat sample3.json | jq .dataSets[0].series.0:0:0:0:0.observations.0[0]
I am able to retieve up to the series using:
cat sample3.json | jq .dataSets[0].series
But once I try to drill down further I am getting a compile error:
$ cat sample3.json | jq .dataSets[0].series.0:0:0:0:0
jq: error: syntax error, unexpected LITERAL, expecting end of file (Unix shell quoting issues?) at <top-level>, line 1:
.dataSets[0].series.0:0:0:0:0
jq: 1 compile error
I am not sure what I am doing wrong here....
The input file is:
{
"header": {
"id": "b8be2cd5-33bf-4687-9e81-eb032f6f8a71",
"test": false,
"prepared": "2022-09-01T13:30:57.013+02:00",
"sender": {
"id": "ECB"
}
},
"dataSets": [
{
"action": "Replace",
"validFrom": "2022-09-01T13:30:57.013+02:00",
"series": {
"0:0:0:0:0": {
"attributes": [
0,
null,
0,
null,
null,
null,
null,
null,
null,
null,
null,
null,
0,
null,
0,
null,
0,
0,
0,
0
],
"observations": {
"0": [
1.4529,
0,
0,
null,
null
],
"1": [
1.4472,
0,
0,
null,
null
],
"2": [
1.4591,
0,
0,
null,
null
]
}
}
}
}
],
"structure": {
"links": [
{
"title": "Exchange Rates",
"rel": "dataflow",
"href": "https://sdw-wsrest.ecb.europa.eu:443/service/dataflow/ECB/EXR/1.0"
}
],
"name": "Exchange Rates",
"dimensions": {
"series": [
{
"id": "FREQ",
"name": "Frequency",
"values": [
{
"id": "D",
"name": "Daily"
}
]
},
{
"id": "CURRENCY",
"name": "Currency",
"values": [
{
"id": "AUD",
"name": "Australian dollar"
}
]
},
{
"id": "CURRENCY_DENOM",
"name": "Currency denominator",
"values": [
{
"id": "EUR",
"name": "Euro"
}
]
},
{
"id": "EXR_TYPE",
"name": "Exchange rate type",
"values": [
{
"id": "SP00",
"name": "Spot"
}
]
},
{
"id": "EXR_SUFFIX",
"name": "Series variation - EXR context",
"values": [
{
"id": "A",
"name": "Average"
}
]
}
],
"observation": [
{
"id": "TIME_PERIOD",
"name": "Time period or range",
"role": "time",
"values": [
{
"id": "2022-08-29",
"name": "2022-08-29",
"start": "2022-08-29T00:00:00.000+02:00",
"end": "2022-08-29T23:59:59.999+02:00"
},
{
"id": "2022-08-30",
"name": "2022-08-30",
"start": "2022-08-30T00:00:00.000+02:00",
"end": "2022-08-30T23:59:59.999+02:00"
},
{
"id": "2022-08-31",
"name": "2022-08-31",
"start": "2022-08-31T00:00:00.000+02:00",
"end": "2022-08-31T23:59:59.999+02:00"
}
]
}
]
},
"attributes": {
"series": [
{
"id": "TIME_FORMAT",
"name": "Time format code",
"values": [
{
"name": "P1D"
}
]
},
{
"id": "BREAKS",
"name": "Breaks",
"values": []
},
{
"id": "COLLECTION",
"name": "Collection indicator",
"values": [
{
"id": "A",
"name": "Average of observations through period"
}
]
},
{
"id": "COMPILING_ORG",
"name": "Compiling organisation",
"values": []
},
{
"id": "DISS_ORG",
"name": "Data dissemination organisation",
"values": []
},
{
"id": "DOM_SER_IDS",
"name": "Domestic series ids",
"values": []
},
{
"id": "PUBL_ECB",
"name": "Source publication (ECB only)",
"values": []
},
{
"id": "PUBL_MU",
"name": "Source publication (Euro area only)",
"values": []
},
{
"id": "PUBL_PUBLIC",
"name": "Source publication (public)",
"values": []
},
{
"id": "UNIT_INDEX_BASE",
"name": "Unit index base",
"values": []
},
{
"id": "COMPILATION",
"name": "Compilation",
"values": []
},
{
"id": "COVERAGE",
"name": "Coverage",
"values": []
},
{
"id": "DECIMALS",
"name": "Decimals",
"values": [
{
"id": "4",
"name": "Four"
}
]
},
{
"id": "NAT_TITLE",
"name": "National language title",
"values": []
},
{
"id": "SOURCE_AGENCY",
"name": "Source agency",
"values": [
{
"id": "4F0",
"name": "European Central Bank (ECB)"
}
]
},
{
"id": "SOURCE_PUB",
"name": "Publication source",
"values": []
},
{
"id": "TITLE",
"name": "Title",
"values": [
{
"name": "Australian dollar/Euro"
}
]
},
{
"id": "TITLE_COMPL",
"name": "Title complement",
"values": [
{
"name": "ECB reference exchange rate, Australian dollar/Euro, 2:15 pm (C.E.T.)"
}
]
},
{
"id": "UNIT",
"name": "Unit",
"values": [
{
"id": "AUD",
"name": "Australian dollar"
}
]
},
{
"id": "UNIT_MULT",
"name": "Unit multiplier",
"values": [
{
"id": "0",
"name": "Units"
}
]
}
],
"observation": [
{
"id": "OBS_STATUS",
"name": "Observation status",
"values": [
{
"id": "A",
"name": "Normal value"
}
]
},
{
"id": "OBS_CONF",
"name": "Observation confidentiality",
"values": [
{
"id": "F",
"name": "Free"
}
]
},
{
"id": "OBS_PRE_BREAK",
"name": "Pre-break observation value",
"values": []
},
{
"id": "OBS_COM",
"name": "Observation comment",
"values": []
}
]
}
}
}
The .foo syntax cannot be used if the key name has anything but alphanumeric characters or the underscore, or if the first character of the key name is numeric.
Assuming you are using a recent version of jq,
you can always use the form: ."foo", which is actually an abbreviation of the basic form, .["foo"].
So assuming you're using a sufficiently recent version of jq, your query could begin with:
.dataSets[0].series."0:0:0:0:0"
If you are presenting the jq query on a command line, then you may have to escape the double-quotes appropriately, e.g. in a bash shell, by enclosing the jq query in single-quotes.

Unable to parse JSON data for Oracle Integration Cloud using JQ

need help to parse the JSON data received from Oracle Integration Cloud. The expected output is mentioned below alongwith the command i am trying to use.
JQ command
jq '[{id: .id},{integrations: [.integrations[]|{code: .code, version: .version, dependencies: .dependencies|{connections: .connections[]|{id: .id, status: .status}}, .dependencies|{lookups: .lookups}}]}]' output.json
Error :
jq: error: syntax error, unexpected FIELD (Unix shell quoting issues?) at , line 1:
[{id: .id},{integrations: [.integrations[]|{code: .code, version: .version, dependencies: .dependencies|{connections: .connections[]|{id: .id, status: .status}}, .dependencies|{lookups: .lookups}}]}]
Note : If i run below command to fetch only connections data it works fine
jq '[{id: .id},{integrations: [.integrations[]|{code: .code, version: .version, dependencies: .dependencies|{connections: .connections[]|{id: .id, status: .status}}}]}]' output.json
Expected Output:
[
{
"id": "SAMPLE_PACKAGE"
},
{
"integrations": [
{
"code": "HELLO_INTEGRATION",
"version": "01.00.0000",
"dependencies": {
"connections": {
"id": "HELLO_WORLD1",
"status": "CONFIGURED"
}
}
},
{
"code": "HELLO_INTEGRATIO_LOOKUP",
"version": "01.00.0000",
"dependencies": {
"connections": {
"id": "HELLO_WORLD1",
"status": "CONFIGURED"
},
"lookups": {
"name": "COMMON_LOOKUP_VARIABLES",
"status": "CONFIGURED"
}
}
},
{
"code": "HI_INTEGRATION",
"version": "01.00.0000",
"dependencies": {
"connections": {
"id": "HELLO_WORLD1",
"status": "CONFIGURED"
}
}
}
]
}
]
output.json file contains
{
"bartaType": "DEVELOPED",
"countOfIntegrations": 3,
"id": "SAMPLE_PACKAGE",
"integrations": [
{
"code": "HELLO_INTEGRATION",
"dependencies": {
"connections": [
{
"id": "HELLO_WORLD1",
"lockedFlag": false,
"name": "Hello World1",
"role": "SOURCE",
"status": "CONFIGURED",
"type": "rest",
"usage": 6
}
]
},
"description": "",
"eventSubscriptionFlag": false,
"filmstrip": [
{
"code": "HELLO_WORLD1",
"iconUrl": "/images/rest/rest_icon_46.png",
"name": "Hello World1",
"role": "SOURCE",
"status": "CONFIGURED"
}
],
"id": "HELLO_INTEGRATION|01.00.0000",
"lockedFlag": false,
"name": "HELLO_INTEGRATION",
"pattern": "Orchestration",
"patternDescription": "Map Data",
"payloadTracingEnabledFlag": true,
"publishFlag": false,
"scheduleApplicable": false,
"scheduleDefined": false,
"status": "ACTIVATED",
"style": "FREEFORM",
"styleDescription": "Orchestration",
"tempCopyExists": false,
"tracingEnabledFlag": true,
"version": "01.00.0000",
"warningMsg": "ACTIVATE_PUBLISH_NO_CONN"
},
{
"code": "HELLO_INTEGRATIO_LOOKUP",
"dependencies": {
"connections": [
{
"id": "HELLO_WORLD1",
"lockedFlag": false,
"name": "Hello World1",
"role": "SOURCE",
"status": "CONFIGURED",
"type": "rest",
"usage": 6
}
],
"lookups": [
{
"lockedFlag": false,
"name": "COMMON_LOOKUP_VARIABLES",
"status": "CONFIGURED",
"usage": 1
}
]
},
"description": "",
"eventSubscriptionFlag": false,
"filmstrip": [
{
"code": "HELLO_WORLD1",
"iconUrl": "/images/rest/rest_icon_46.png",
"name": "Hello World1",
"role": "SOURCE",
"status": "CONFIGURED"
}
],
"id": "HELLO_INTEGRATIO_LOOKUP|01.00.0000",
"lockedFlag": false,
"name": "HELLO_INTEGRATION_LOOKUP",
"pattern": "Orchestration",
"patternDescription": "Map Data",
"payloadTracingEnabledFlag": true,
"publishFlag": false,
"scheduleApplicable": false,
"scheduleDefined": false,
"status": "ACTIVATED",
"style": "FREEFORM",
"styleDescription": "Orchestration",
"tempCopyExists": false,
"tracingEnabledFlag": true,
"version": "01.00.0000",
"warningMsg": "ACTIVATE_PUBLISH_NO_CONN"
},
{
"code": "HI_INTEGRATION",
"dependencies": {
"connections": [
{
"id": "HELLO_WORLD1",
"lockedFlag": false,
"name": "Hello World1",
"role": "SOURCE",
"status": "CONFIGURED",
"type": "rest",
"usage": 6
}
]
},
"description": "",
"eventSubscriptionFlag": false,
"filmstrip": [
{
"code": "HELLO_WORLD1",
"iconUrl": "/images/rest/rest_icon_46.png",
"name": "Hello World1",
"role": "SOURCE",
"status": "CONFIGURED"
}
],
"id": "HI_INTEGRATION|01.00.0000",
"lockedFlag": false,
"name": "HI_INTEGRATION",
"pattern": "Orchestration",
"patternDescription": "Map Data",
"payloadTracingEnabledFlag": true,
"publishFlag": false,
"scheduleApplicable": false,
"scheduleDefined": false,
"status": "ACTIVATED",
"style": "FREEFORM",
"styleDescription": "Orchestration",
"tempCopyExists": false,
"tracingEnabledFlag": true,
"version": "01.00.0000",
"warningMsg": "ACTIVATE_PUBLISH_NO_CONN"
}
],
"isCloneAllowed": false,
"isViewAllowed": false,
"name": "SAMPLE_PACKAGE",
"type": "DEVELOPED"
}
The problem is that the lookups key is not always present so, you cannot use the [] on it. So, instead you can use the map function and provide a default before piping to the map function like below
[
{ id: .id },
{
integrations: [
.integrations[]|{
id: .id,
code: .code,
dependencies: {
connections: (.dependencies.connections//[]|map({id,status}))[0],
lookups: (.dependencies.lookups//[]|map({name,status}))[0]
}
}
]
}
]
The (.dependencies.lookups//[]|map({name,status}))[0] has the effect of passing an empty array to the map function which results in a null value when accessing the first element.
See in action https://jqplay.org/s/zQBkHtnzOd1
The provided JQ statement works fine for single elements in the array , but incase the array contains multiple elements it only fetches the first element. Also i updated the dependencies object to capture all the arrays ( connections,lookups,certificates,libraries,integrations)
Below is the modified one. Please suggest for any better options.
[
{ id: .id },
{
integrations: [
.integrations[]|{
id: .id,
code: .code,
dependencies: {
connections: (.dependencies.connections//[]|map({id,status})),
lookups: (.dependencies.lookups//[]|map({name,status})),
certificates: (.dependencies.certificates//[]|map({id,status})),
libraries: (.dependencies.libraries//[]|map({code,status,version})),
integrations: (.dependencies.integrations//[]|map({code,version}))
}
}
]
}
]|del(..|select(.==[]))
Note: To remove the empty arrays del function is added which is giving the below output :
[
{
"id": "SAMPLE_PACKAGE"
},
{
"integrations": [
{
"id": "HELLO_INTEGRATION|01.00.0000",
"code": "HELLO_INTEGRATION",
"dependencies": {
"connections": [
{
"id": "HELLO_WORLD1",
"status": "CONFIGURED"
},
{
"id": "HELLO_WORLD2",
"status": "CONFIGURED"
}
]
}
},
{
"id": "HELLO_INTEGRATIO_LOOKUP|01.00.0000",
"code": "HELLO_INTEGRATIO_LOOKUP",
"dependencies": {
"connections": [
{
"id": "HELLO_WORLD1",
"status": "CONFIGURED"
}
],
"lookups": [
{
"name": "COMMON_LOOKUP_VARIABLES",
"status": "CONFIGURED"
}
]
}
},
{
"id": "HI_INTEGRATION|01.00.0000",
"code": "HI_INTEGRATION",
"dependencies": {
"connections": [
{
"id": "HELLO_WORLD1",
"status": "CONFIGURED"
}
]
}
}
]
}
]

Kubernetes + jq - retrieving containers list per pod yields cartesian product

Im trying to use jq on kubernetes json output, to create new json object containing list of objects - container and image per pod, however im getting cartesian product.
my input data (truncated from sensitive info):
{
"apiVersion": "v1",
"items": [
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"creationTimestamp": "2021-06-30T12:45:40Z",
"name": "pod-1",
"namespace": "default",
"resourceVersion": "757679286",
"selfLink": "/api/v1/namespaces/default/pods/pod-1"
},
"spec": {
"containers": [
{
"image": "image-1",
"imagePullPolicy": "Always",
"name": "container-1",
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"volumeMounts": [
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"readOnly": true
}
]
},
{
"image": "image-2",
"imagePullPolicy": "Always",
"name": "container-2",
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"volumeMounts": [
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"readOnly": true
}
]
}
],
"dnsPolicy": "ClusterFirst",
"enableServiceLinks": true,
"priority": 0,
"restartPolicy": "Always",
"schedulerName": "default-scheduler",
"securityContext": {},
"serviceAccount": "default",
"serviceAccountName": "default",
"terminationGracePeriodSeconds": 30,
"tolerations": [
{
"effect": "NoExecute",
"key": "node.kubernetes.io/not-ready",
"operator": "Exists",
"tolerationSeconds": 300
},
{
"effect": "NoExecute",
"key": "node.kubernetes.io/unreachable",
"operator": "Exists",
"tolerationSeconds": 300
}
],
"volumes": [
{
"name": "default-token-b954f",
"secret": {
"defaultMode": 420,
"secretName": "default-token-b954f"
}
}
]
},
"status": {
"conditions": [
{
"lastProbeTime": null,
"lastTransitionTime": "2021-06-30T12:45:40Z",
"status": "True",
"type": "Initialized"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2021-06-30T12:45:40Z",
"message": "containers with unready status: [container-1 container-2]",
"reason": "ContainersNotReady",
"status": "False",
"type": "Ready"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2021-06-30T12:45:40Z",
"message": "containers with unready status: [container-1 container-2]",
"reason": "ContainersNotReady",
"status": "False",
"type": "ContainersReady"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2021-06-30T12:45:40Z",
"status": "True",
"type": "PodScheduled"
}
],
"containerStatuses": [
{
"image": "image-1",
"imageID": "",
"lastState": {},
"name": "container-1",
"ready": false,
"restartCount": 0,
"started": false,
"state": {
"waiting": {
"message": "Back-off pulling image \"image-1\"",
"reason": "ImagePullBackOff"
}
}
},
{
"image": "image-2",
"imageID": "",
"lastState": {},
"name": "container-2",
"ready": false,
"restartCount": 0,
"started": false,
"state": {
"waiting": {
"message": "Back-off pulling image \"image-2\"",
"reason": "ImagePullBackOff"
}
}
}
],
"qosClass": "BestEffort",
"startTime": "2021-06-30T12:45:40Z"
}
}
],
"kind": "List",
"metadata": {
"resourceVersion": "",
"selfLink": ""
}
}
my command:
jq '.items[] | { "name": .metadata.name, "containers": [{ "name": .spec.containers[].name, "image": .spec.containers[].image }]} '
desired output:
{
"name": "pod_1",
"containers": [
{
"name": "container_1",
"image": "image_1"
},
{
"name": "container_2",
"image": "image_2"
}
]
}
output I get:
{
"name": "pod-1",
"containers": [
{
"name": "container-1",
"image": "image-1"
},
{
"name": "container-1",
"image": "image-2"
},
{
"name": "container-2",
"image": "image-1"
},
{
"name": "container-2",
"image": "image-2"
}
]
}
Could anyone explain what am I doing wrong?
Best Regards, Piotr.
The problem is "name": .spec.containers[].name and "image": .spec.containers[].image:
Both expressions generate a sequence of each value for name and image which will than be combined.
Simplified example of why you get a Cartesian product:
jq -c -n '{name: ("A", "B"), value: ("C", "D")}'
outputs:
{"name":"A","value":"C"}
{"name":"A","value":"D"}
{"name":"B","value":"C"}
{"name":"B","value":"D"}
You get the desired output using this jq filter on your input:
jq '
.items[]
| {
"name": .metadata.name,
"containers": .spec.containers
| map({name, image})
}'
output:
{
"name": "pod-1",
"containers": [
{
"name": "container-1",
"image": "image-1"
},
{
"name": "container-2",
"image": "image-2"
}
]
}

Converting array to objects within an object

I have been battling with this for sometime, ive looked a few articles in regards to this topic. I can't seem to get exactly what I'm aiming for. I have this original json data that I would like to transform a bit.
Group the objects based on the label.env (for now only using one env), then inserting those objects from the original json into an object named projects.
orginial_json
{
"createTime": "2020-06-25T14:16:45.720Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "infra",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname",
"parent": {
"id": "123456577",
"type": "folder"
},
"projectId": "projectname-324234",
"projectNumber": "962363417856"
}
{
"createTime": "2020-06-24T19:45:42.851Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "ad",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname2",
"parent": {
"id": "4352564765437",
"type": "folder"
},
"projectId": "projectname2-4567",
"projectNumber": "3243456324"
}
Im using jq -s 'group_by(.labels.env) | .[] | { (.[0].labels.env) : { projects: .[] | . } }'
to nest the original json data into objects grouped by env\projects, which works but seems to duplicate it instead of putting each object within projects.
{
"prod": {
"projects": {
"createTime": "2020-06-25T14:16:45.720Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "infra",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname",
"parent": {
"id": "770593713153",
"type": "folder"
},
"projectId": "projectname-324234",
"projectNumber": "962363417856"
}
}
}
{
"prod": {
"projects": {
"createTime": "2020-06-24T19:45:42.851Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "ad",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname2",
"parent": {
"id": "4352564765437",
"type": "folder"
},
"projectId": "projectname2-4567",
"projectNumber": "3243456324"
}
}
}
What I'm aiming for is
{
"prod": {
"projects": {
"createTime": "2020-06-25T14:16:45.720Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "infra",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname",
"parent": {
"id": "770593713153",
"type": "folder"
},
"projectId": "projectname-324234",
"projectNumber": "962363417856"
},
{
"createTime": "2020-06-24T19:45:42.851Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "ad",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname2",
"parent": {
"id": "4352564765437",
"type": "folder"
},
"projectId": "projectname2-4567",
"projectNumber": "3243456324"
}
}
}
Allowing for some minor discrepancies in the Q, the following variant of your jq program seems to do what you're trying to achieve:
group_by(.labels.env)
| .[]
| .[0].labels.env as $key
| { ($key) : { projects: . } }

Stellar cross currency transaction doesn't work

I'm unable to execute payment transaction in Stellar testnet.
Initially i use find_path:
curl -X POST https://test.stellar.org:9002 -d '
{
"method": "static_path_find",
"params": [
{
"source_account": "g4e5v2ERpKvdBZrchn6DrWUtvezkXsu5wo",
"destination_account": "gJ3e65GzqERgeeS5oXsv8NGfdZWzm93ej6",
"source_currencies" : [ { "currency" : "SGD", "issuer" : "gDfapfG5hDHuYkbVpugtupkGYVTKXwd59r"} ],
"destination_amount": {
"currency": "MYR",
"value": "1",
"issuer": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS"
}
}
]
}'
The result I get:
{
"result": {
"alternatives": [
{
"paths_computed": [
[
{
"currency": "MYR",
"issuer": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"type": 48
},
{
"account": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"type": 1
}
]
],
"source_amount": {
"currency": "SGD",
"issuer": "gDfapfG5hDHuYkbVpugtupkGYVTKXwd59r",
"value": "0.375"
}
}
],
"destination_account": "gJ3e65GzqERgeeS5oXsv8NGfdZWzm93ej6",
"destination_currencies": [
"MYR",
"STR"
],
"status": "success"
}
}
Then I trying to broadcast transaction in the network:
curl -X POST https://test.stellar.org:9002 -d '
{
"method": "submit",
"params": [
{
"secret": "sfk7dUd4N8cxVFVowsZL4DQgonNP9k8WFBgaV2NmwEQKGzWuqXT",
"tx_json": {
"TransactionType": "Payment",
"Account": "g4e5v2ERpKvdBZrchn6DrWUtvezkXsu5wo",
"Destination": "gJ3e65GzqERgeeS5oXsv8NGfdZWzm93ej6",
"Amount": {
"currency": "MYR",
"value": "1",
"issuer": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS"
},
"Paths": [
[
{
"currency": "MYR",
"issuer": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"type": 48
},
{
"account": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"type": 1
}
]
]
}
}
]
}'
And finally I'm getting unsuccessful result when transaction passed through consensus:
{
"engine_result": "tecPATH_PARTIAL",
"engine_result_code": 101,
"engine_result_message": "Path could not send full amount.",
"ledger_hash": "82BCF42941EF00BEA9E575B696CD88EC7E14FDE8CFD39FDA6A76B5710FDD0755",
"ledger_index": 704612,
"meta": {
"AffectedNodes": [
{
"ModifiedNode": {
"FinalFields": {
"Account": "g4e5v2ERpKvdBZrchn6DrWUtvezkXsu5wo",
"Balance": "199999960",
"Flags": 0,
"OwnerCount": 1,
"Sequence": 5
},
"LedgerEntryType": "AccountRoot",
"LedgerIndex": "D3433DEE556200D8AB72DED7156A597A273103D2BC50F526193EB9C4D8C8B735",
"PreviousFields": {
"Balance": "199999970",
"Sequence": 4
},
"PreviousTxnID": "6ADF48B45D95B3ABD20FD78F95F7E4F20D91FF5D169ECC5EFD6DB0CA78BEA486",
"PreviousTxnLgrSeq": 704445
}
}
],
"TransactionIndex": 0,
"TransactionResult": "tecPATH_PARTIAL"
},
"status": "closed",
"transaction": {
"Account": "g4e5v2ERpKvdBZrchn6DrWUtvezkXsu5wo",
"Amount": {
"currency": "MYR",
"issuer": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"value": "1"
},
"Destination": "gJ3e65GzqERgeeS5oXsv8NGfdZWzm93ej6",
"Fee": "10",
"Flags": 2147483648,
"Paths": [
[
{
"currency": "MYR",
"issuer": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"type": 48
},
{
"account": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"type": 1
}
]
],
"Sequence": 4,
"SigningPubKey": "4B640AC0C64AEF1C97EAAD0629CADDCA95FBA5F095BD20BD6282BEC450A125B1",
"TransactionType": "Payment",
"TxnSignature": "7BF5772EB916F20F56C5F148E99C3C3DD77C2916EAA47AB2D7FA8306F2B4A9D3FA4B789BDB79899A634B0C33F50A6ACB447CA6A9633C1C73BAB754FBFE74C105",
"date": 485412700,
"hash": "846AB6502D3B7A3137B2B538E6616B9C4459D2A67C48E58AF6B64F4659FF0016"
},
"type": "transaction",
"validated": true
}
Why? What I did wrong? I really can't understand...
Anyway I found the problem.
I forgot to add required SendMax parameter when submitting transaction.
curl -X POST https://test.stellar.org:9002 -d '
{
"method": "submit",
"params": [
{
"secret": "sfk7dUd4N8cxVFVowsZL4DQgonNP9k8WFBgaV2NmwEQKGzWuqXT",
"tx_json": {
"TransactionType": "Payment",
"Account": "g4e5v2ERpKvdBZrchn6DrWUtvezkXsu5wo",
"Destination": "gJ3e65GzqERgeeS5oXsv8NGfdZWzm93ej6",
"Amount": {
"currency": "MYR",
"value": "1",
"issuer": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS"
},
"Paths": [
[
{
"currency": "MYR",
"issuer": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"type": 48
},
{
"account": "gsQuUKcxM68nEX9tKR71UH5hZXwrvT2mpS",
"type": 1
}
]
],
"SendMax" : { "currency" : "SGD", "value" : "0.375", "issuer": "gDfapfG5hDHuYkbVpugtupkGYVTKXwd59r" }
}
}
]
}'