POWERSHELL - How to access multilevel child elements in JSON file with condtion - json

can someone please send me solution or link for PowerShell 5 and 7 how can I access child elements if specific condition is fulfilled for JSON file which I have as output.json. I haven't find it on the net.
I want to retrieve value of the "children" elements if type element has value FILE and to put that into some list. So final result should be [test1.txt,test2.txt]
Thank you!!!
{
"path": {
"components": [
"Packages"
],
"parent": "",
"name": "Packages",
},
"children": {
"values": [
{
"path": {
"components": [
"test1.txt"
],
"parent": "",
"name": "test1.txt",
},
"type": "FILE",
"size": 405
},
{
"path": {
"components": [
"test2.txt"
],
"parent": "",
"name": "test2.txt",
},
"type": "FILE",
"size": 409
},
{
"path": {
"components": [
"FOLDER"
],
"parent": "",
"name": "FOLDER",
},
"type": "DIRECTORY",
"size": 1625
}
]
"start": 0
}
}

1.) The json is incorrect, I assumt that this one is the correct one:
{
"path": {
"components": [
"Packages"
],
"parent": "",
"name": "Packages"
},
"children": {
"values": [
{
"path": {
"components": [
"test1.txt"
],
"parent": "",
"name": "test1.txt"
},
"type": "FILE",
"size": 405
},
{
"path": {
"components": [
"test2.txt"
],
"parent": "",
"name": "test2.txt"
},
"type": "FILE",
"size": 409
},
{
"path": {
"components": [
"FOLDER"
],
"parent": "",
"name": "FOLDER"
},
"type": "DIRECTORY",
"size": 1625
}
],
"start": 0
}
}
2.) The structure is not absolute clear, but for your example this seems to me to be the correct solution:
$element = $json | ConvertFrom-Json
$result = #()
$element.children.values | foreach {
if ($_.type -eq 'FILE') { $result += $_.path.name }
}
$result | ConvertTo-Json
Be aware, that the used construct $result += $_.path.name is fine if you have up to ~10k items, but for very large items its getting very slow and you need to use an arraylist. https://adamtheautomator.com/powershell-arraylist/

Related

Cannot get jq to query json object [duplicate]

This question already has answers here:
How to use jq when the variable has reserved characters?
(3 answers)
Closed 6 months ago.
I have a JSON file that I am trying to query with jq. I am unable to retrieve the observations. I am trying to retieve each of the "observations using the following command and not able to get to the result:
cat sample3.json | jq .dataSets[0].series.0:0:0:0:0.observations.0[0]
I am able to retieve up to the series using:
cat sample3.json | jq .dataSets[0].series
But once I try to drill down further I am getting a compile error:
$ cat sample3.json | jq .dataSets[0].series.0:0:0:0:0
jq: error: syntax error, unexpected LITERAL, expecting end of file (Unix shell quoting issues?) at <top-level>, line 1:
.dataSets[0].series.0:0:0:0:0
jq: 1 compile error
I am not sure what I am doing wrong here....
The input file is:
{
"header": {
"id": "b8be2cd5-33bf-4687-9e81-eb032f6f8a71",
"test": false,
"prepared": "2022-09-01T13:30:57.013+02:00",
"sender": {
"id": "ECB"
}
},
"dataSets": [
{
"action": "Replace",
"validFrom": "2022-09-01T13:30:57.013+02:00",
"series": {
"0:0:0:0:0": {
"attributes": [
0,
null,
0,
null,
null,
null,
null,
null,
null,
null,
null,
null,
0,
null,
0,
null,
0,
0,
0,
0
],
"observations": {
"0": [
1.4529,
0,
0,
null,
null
],
"1": [
1.4472,
0,
0,
null,
null
],
"2": [
1.4591,
0,
0,
null,
null
]
}
}
}
}
],
"structure": {
"links": [
{
"title": "Exchange Rates",
"rel": "dataflow",
"href": "https://sdw-wsrest.ecb.europa.eu:443/service/dataflow/ECB/EXR/1.0"
}
],
"name": "Exchange Rates",
"dimensions": {
"series": [
{
"id": "FREQ",
"name": "Frequency",
"values": [
{
"id": "D",
"name": "Daily"
}
]
},
{
"id": "CURRENCY",
"name": "Currency",
"values": [
{
"id": "AUD",
"name": "Australian dollar"
}
]
},
{
"id": "CURRENCY_DENOM",
"name": "Currency denominator",
"values": [
{
"id": "EUR",
"name": "Euro"
}
]
},
{
"id": "EXR_TYPE",
"name": "Exchange rate type",
"values": [
{
"id": "SP00",
"name": "Spot"
}
]
},
{
"id": "EXR_SUFFIX",
"name": "Series variation - EXR context",
"values": [
{
"id": "A",
"name": "Average"
}
]
}
],
"observation": [
{
"id": "TIME_PERIOD",
"name": "Time period or range",
"role": "time",
"values": [
{
"id": "2022-08-29",
"name": "2022-08-29",
"start": "2022-08-29T00:00:00.000+02:00",
"end": "2022-08-29T23:59:59.999+02:00"
},
{
"id": "2022-08-30",
"name": "2022-08-30",
"start": "2022-08-30T00:00:00.000+02:00",
"end": "2022-08-30T23:59:59.999+02:00"
},
{
"id": "2022-08-31",
"name": "2022-08-31",
"start": "2022-08-31T00:00:00.000+02:00",
"end": "2022-08-31T23:59:59.999+02:00"
}
]
}
]
},
"attributes": {
"series": [
{
"id": "TIME_FORMAT",
"name": "Time format code",
"values": [
{
"name": "P1D"
}
]
},
{
"id": "BREAKS",
"name": "Breaks",
"values": []
},
{
"id": "COLLECTION",
"name": "Collection indicator",
"values": [
{
"id": "A",
"name": "Average of observations through period"
}
]
},
{
"id": "COMPILING_ORG",
"name": "Compiling organisation",
"values": []
},
{
"id": "DISS_ORG",
"name": "Data dissemination organisation",
"values": []
},
{
"id": "DOM_SER_IDS",
"name": "Domestic series ids",
"values": []
},
{
"id": "PUBL_ECB",
"name": "Source publication (ECB only)",
"values": []
},
{
"id": "PUBL_MU",
"name": "Source publication (Euro area only)",
"values": []
},
{
"id": "PUBL_PUBLIC",
"name": "Source publication (public)",
"values": []
},
{
"id": "UNIT_INDEX_BASE",
"name": "Unit index base",
"values": []
},
{
"id": "COMPILATION",
"name": "Compilation",
"values": []
},
{
"id": "COVERAGE",
"name": "Coverage",
"values": []
},
{
"id": "DECIMALS",
"name": "Decimals",
"values": [
{
"id": "4",
"name": "Four"
}
]
},
{
"id": "NAT_TITLE",
"name": "National language title",
"values": []
},
{
"id": "SOURCE_AGENCY",
"name": "Source agency",
"values": [
{
"id": "4F0",
"name": "European Central Bank (ECB)"
}
]
},
{
"id": "SOURCE_PUB",
"name": "Publication source",
"values": []
},
{
"id": "TITLE",
"name": "Title",
"values": [
{
"name": "Australian dollar/Euro"
}
]
},
{
"id": "TITLE_COMPL",
"name": "Title complement",
"values": [
{
"name": "ECB reference exchange rate, Australian dollar/Euro, 2:15 pm (C.E.T.)"
}
]
},
{
"id": "UNIT",
"name": "Unit",
"values": [
{
"id": "AUD",
"name": "Australian dollar"
}
]
},
{
"id": "UNIT_MULT",
"name": "Unit multiplier",
"values": [
{
"id": "0",
"name": "Units"
}
]
}
],
"observation": [
{
"id": "OBS_STATUS",
"name": "Observation status",
"values": [
{
"id": "A",
"name": "Normal value"
}
]
},
{
"id": "OBS_CONF",
"name": "Observation confidentiality",
"values": [
{
"id": "F",
"name": "Free"
}
]
},
{
"id": "OBS_PRE_BREAK",
"name": "Pre-break observation value",
"values": []
},
{
"id": "OBS_COM",
"name": "Observation comment",
"values": []
}
]
}
}
}
The .foo syntax cannot be used if the key name has anything but alphanumeric characters or the underscore, or if the first character of the key name is numeric.
Assuming you are using a recent version of jq,
you can always use the form: ."foo", which is actually an abbreviation of the basic form, .["foo"].
So assuming you're using a sufficiently recent version of jq, your query could begin with:
.dataSets[0].series."0:0:0:0:0"
If you are presenting the jq query on a command line, then you may have to escape the double-quotes appropriately, e.g. in a bash shell, by enclosing the jq query in single-quotes.

Search object a element and return the element and parents

I want to make a search from object JSON but when a I found the element I want to be returned with his previous parents.
[
{
"name": "level1",
"children": [
{
"name": "level2",
"children": [
{
"name": "level3a",
"children": [
{
"name": "text",
"more": "info"
},
{
"name": "text Abc",
"more": "info"
}
]
},
{
"name": "level3b",
"children": [
{
"name": "text-C",
"more": "info"
},
{
"name": "search",
"more": "info"
}
]
},
{
"name": "level3c",
"children": [
{
"name": "info-C",
"more": "info"
},
{
"name": "search",
"more": "info"
}
]
}
]
}
]
},
{
"name": "level1",
"children": [
{
"name": "level2a",
"children": [
{
"name": "level3",
"children": [
{
"name": "text A",
"more": "info"
}
]
},
]
},
{
"name": "level2b",
"children": [
{
"name": "level3",
"children": [
{
"name": "text X",
"more": "info"
}
]
},
]
}
]
}
]
For example: if want to search "text" I want result like this
[
{
"name": "level1",
"children": [
{
"name": "level2",
"children": [
{
"name": "level3a",
"children": [
{
"name": "text",
"more": "info"
},
{
"name": "text Abc",
"more": "info"
}
]
},
{
"name": "level3b",
"children": [
{
"name": "text-C",
"more": "info"
}
]
}
]
}
]
},
{
"name": "level1",
"children": [
{
"name": "level2a",
"children": [
{
"name": "level3",
"children": [
{
"name": "text A",
"more": "info"
}
]
},
]
},
{
"name": "level2b",
"children": [
{
"name": "level3",
"children": [
{
"name": "text X",
"more": "info"
}
]
},
]
}
]
}
]
Things to considerer
only "search" in level 3 children by name property
if one o more childrens matches the search, return all of them
Basically I need to filter by the last level of object with elements matches with the search and return them with them parents.

jq return a json array in a very specifique way

I have this Json ( is a test database, no data is true here )
{
"pguid": "4EA979A2-E578-4DA3-89DB-24082F3092AA",
"lastEnrollTguid": "EA98B161-04D3-4F0A-920A-58DBFF3C2274",
"timestamp": 1016086888000,
"keys": [
{
"id": "gr",
"value": "1907971"
}
],
"biographics": [
{
"id": "localNascimento",
"value": "JOINVILLE SC"
},
{
"id": "dataNascimento",
"value": "1859-03-08"
},
{
"id": "mae",
"value": "ANTA MARCIA PINHEAD"
},
{
"id": "nome",
"value": "MIR PINHEAD"
}
],
"biometric": [
{
"source": "ORIGINAL",
"type": "FACE",
"format": "JPEG",
"properties": {
"width": 0,
"height": 0,
"resolution": 500,
"ratio": 0,
"matcherId": 0,
"extractorId": 0
},
"index": 10,
"content": "5215421547"
}
],
"labels": [
"SC",
"CIVIL",
"MASCULINO",
"JOINVILLE"
],
"history": {
"events": [
{
"type": "ENROLL",
"tguid": "3C1B0D1F-9143-4C24-A351-E88A19317AC9",
"timestamp": 1014086658288
},
{
"type": "UPDATE",
"tguid": "EA98B161-04D3-4F0A-920A-58DBFF3C2274",
"timestamp": 1016786888028
}
]
}
}
I want to retrive only de tguid in history array, and if exist a way to do this, use de index of the array to acomplish that.
Here I tryed to acomplish that ( and miserable failed in that )
example ( and it do not work ):
jq '.[].history.events.tguid[1]' /tmp/teste.json
I want to retrieve the pguid in a index to work with that.
Someone have any ideas?
try this
jq '.history.events | .[1].tguid' /tmp/teste.json
tnks everyone
jq '.[].history.events | .[0].tguid' /tmp/teste1.json

Removing entire object from JSON file based on key value pair

I been struggling removing set-off objects from Json file. I tried with jq json parser method but nothing has worked out. Could someone please help on this.
What am looking for is – Wherever the below key and value pair are present in a file, the entire object should be removed.
{"name": "exception"}
Input:
{
"results": [
{
"id": "a21f5193-881e-11eb-a0c1-3726f4a71fa9",
"retailerId": "1",
"category": "exception",
"context": {
"sourceEvents": [
"902bd449-881e-11eb-b603-29eb6c297e7d"
],
"entityType": "ORDER"
},
"eventStatus": "FAILED",
"attributes": [
{
"name": "exception",
"value": {
"code": 400,
"message": "Failed to execute http call",
"stackTrace": [
{
"fileName": "ReadOnlyFluentApiClient.java",
"className": "com.fluentretail.api.v2.client.ReadOnlyFluentApiClient"
}
],
"suppressed": [],
"suppressedExceptions": []
},
"type": "OBJECT"
},
{
"name": "lastRule",
"value": "ETOSUAT.base.ProposedFulfilmentWithoutInventory",
"type": "String"
},
{
"name": "lastRuleSet",
"value": "FindAndCreateDigitalFulfilment",
"type": "String"
},
{
"name": "message",
"value": "Failed to execute http call",
"type": "String"
}
],
"source": null,
"generatedBy": "Rubix User",
"generatedOn": "2021-03-18T19:17:51.517+0000"
},
{
"id": "a21f5193-881e-11eb-a0c1-3726f4a71fa9",
"retailerId": "1",
"category": "exception",
"context": {
"sourceEvents": [
"902bd449-881e-11eb-b603-29eb6c297e7d"
],
"entityType": "ORDER"
},
"eventStatus": "FAILED",
"attributes": [
{
"name": "exception",
"value": {
"code": 400,
"message": "Failed to execute http call",
"stackTrace": [
{
"fileName": "ReadOnlyFluentApiClient.java",
"className": "com.fluentretail.api.v2.client.ReadOnlyFluentApiClient"
}
],
"suppressed": [],
"suppressedExceptions": []
},
"type": "OBJECT"
},
{
"name": "lastRule",
"value": "ETOSUAT.base.ProposedFulfilmentWithoutInventory",
"type": "String"
},
{
"name": "lastRuleSet",
"value": "FindAndCreateDigitalFulfilment",
"type": "String"
},
{
"name": "message",
"value": "Failed to execute http call",
"type": "String"
}
],
"source": null,
"generatedBy": "Rubix User",
"generatedOn": "2021-03-18T19:17:51.517+0000"
}
]
}
Expected output is -
{
"results": [
{
"id": "a21f5193-881e-11eb-a0c1-3726f4a71fa9",
"retailerId": "1",
"category": "exception",
"context": {
"sourceEvents": [
"902bd449-881e-11eb-b603-29eb6c297e7d"
],
"entityType": "ORDER"
},
"eventStatus": "FAILED",
"attributes": [
{
"name": "lastRule",
"value": "ETOSUAT.base.ProposedFulfilmentWithoutInventory",
"type": "String"
},
{
"name": "lastRuleSet",
"value": "FindAndCreateDigitalFulfilment",
"type": "String"
},
{
"name": "message",
"value": "Failed to execute http call",
"type": "String"
}
],
"source": null,
"generatedBy": "Rubix User",
"generatedOn": "2021-03-18T19:17:51.517+0000"
},
{
"id": "a21f5193-881e-11eb-a0c1-3726f4a71fa9",
"retailerId": "1",
"category": "exception",
"context": {
"sourceEvents": [
"902bd449-881e-11eb-b603-29eb6c297e7d"
],
"entityType": "ORDER"
},
"eventStatus": "FAILED",
"attributes": [
{
"name": "lastRule",
"value": "ETOSUAT.base.ProposedFulfilmentWithoutInventory",
"type": "String"
},
{
"name": "lastRuleSet",
"value": "FindAndCreateDigitalFulfilment",
"type": "String"
},
{
"name": "message",
"value": "Failed to execute http call",
"type": "String"
}
],
"source": null,
"generatedBy": "Rubix User",
"generatedOn": "2021-03-18T19:17:51.517+0000"
}
]
}
del(..|select(type=="object" and .name=="exception"))
Try it at https://jqplay.org/s/il12Ribpdb
walk(if type=="object" and .name == "exception"
then empty else . end)
Equivalently:
walk(select(type=="object" and .name == "exception" | not))

jq sort by value of key

Given the following JSON (oversimplified for the sake of the example), I need to order the keys by their value. In this case, the order should be id > name > type.
{
"link": [{
"attributes": [{
"value": "ConfigurationElement",
"name": "type"
}, {
"value": "NAME1",
"name": "name"
}, {
"value": "0026a8b4-ced6-410e-9213-e3fcb28b3aab",
"name": "id"
}
],
"href": "href1",
"rel": "down"
}, {
"attributes": [{
"value": "0026a8b4-ced6-410e-9213-k23g15h2u1l5",
"name": "id"
}, {
"value": "ConfigurationElement",
"name": "type"
}, {
"value": "NAME2",
"name": "name"
}
],
"href": "href2",
"rel": "down"
}
],
"total": 2
}
EXPECTED RESULT:
{
"link": [{
"attributes": [{
"value": "0026a8b4-ced6-410e-9213-e3fcb28b3aab",
"name": "id"
}, {
"value": "NAME1",
"name": "name"
}, {
"value": "ConfigurationElement",
"name": "type"
}
],
"href": "href1",
"rel": "down"
}, {
"attributes": [{
"value": "0026a8b4-ced6-410e-9213-k23g15h2u1l5",
"name": "id"
}, {
"value": "NAME2",
"name": "name"
}, {
"value": "ConfigurationElement",
"name": "type"
}
],
"href": "href2",
"rel": "down"
}
],
"total": 2
}
I would be very grateful if anyone could help me out. I tried jq with -S and -s with sort_by(), but this example is way too complex for me to figure it out with my current experience with jq. Thank you a lot!
You can do:
jq '.link[].attributes|=sort_by(.name)'
The |= takes all the paths matched by .link[].attributes, i.e. each "attributes" array, and applies the filter sort_by(.name) to each of them, leaving everything else unchanged.