Filter JSON in Powershell - json

I have the following JSON in a Powershell variable:
{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
I want to get JSON where the value of criteria is 1. The result therefore should look as follows:
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
I made an attempt with the following code:
$json | Get-ObjectMembers | Select-Object | where { $_.value.criteria -eq 1 };
While this basically goes into the right direction, it is not what I want exactly, because the result looks like this:
{
"name": "asdf2",
"criteria": 1
}
See that the Object2 information is lost and one depth-level is lost.
How can I achieve the desired result as shown above?

In essence, you're looking to only retain properties of interest from your single input object or, to put it differently, to remove properties you're not interested in.
Here's a PSv4+ solution:
$json = #'
{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
'#
($json | ConvertFrom-Json).psobject.Properties.
Where({ $_.Value.criteria -eq 1 }).
ForEach({ [pscustomobject] #{ $_.Name = $_.Value } }) |
ConvertTo-Json
The above yields:
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}

The jq solution. with_entries temporarily turns a list of properties into key and value pairs. https://stedolan.github.io/jq/manual/#to_entries,from_entries,with_entries (section id's in html come in handy) (Json creators don't believe in arrays?)
$json = '{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}'
$json | jq 'with_entries(select(.value.criteria == 1))'
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}

Related

Powershell replace values in target.json with values from source.json

I have a big target json file (parameters_general.json) where all common settings for a deployment are set.
For each tier I have another json file (ex: parameters_dev.json, parameters_test.json, ....) Settings set in one needs tobe added to the general.json, or overwrite it when already in general.
ex: parameters_general.json
{
"general": {
"db_Policy": {
"type": "Periodic",
"databaseAccountOfferType": "Standard",
"periodicModeProperties": {
"backupIntervalInMinutes": 240,
"backupRetentionIntervalInHours": 8,
"backupStorageRedundancy": "Local"
}
},
"databases": [
{
"name": "CtrlWps",
"Containers": [
{
"name": "ControllerAuthentication",
"partitionKey": "id"
}
],
"ContainersTTL": []
},
{
"name": "CpoOcpi",
"Containers": [
{
"name": "Cpos",
"partitionKey": "cpoId"
},
{
"name": "OcpiCdrLastRecoveries",
"partitionKey": "id"
},
{
"name": "Routes",
"partitionKey": "ocpiCpoId"
}
],
"ContainersTTL": [
{
"name": "OcpiCdrs",
"partitionKey": "pk",
"ttl": 172800
},
{
"name": "OcppTransactionIds",
"partitionKey": "pk",
"ttl": 172800
},
{
"name": "Sessions",
"partitionKey": "pk",
"ttl": 172800
}
]
}
],
"system_engineers": [
],
}
}
If I want to updatethis with.
ex: parameters_test.json
{
"general": {
"system_engineers": [
{
"name": "hans",
"AppPrincipalId": "<id>",
"permissions": [
"get",
"list"
]
},
{
"name": "John do",
"AppPrincipalId": "<pid>",
"permissions": [
"all"
]
}
]
}
}
This works, the users are added to the empty "sytem_engineers" node in the parameters_general.json.
However, If I just want to change a setting on a lower node example:
ex: parameters_dev.json
{
"general": {
"databases": [
{
"name": "CtrlWps",
"Containers": [
{
"name": "ControllerAuthentication",
"partitionKey": "pk"
}
]
}
]
}
}
for replacing the partitionKey in one of the databases it replaces the whole "databases" part so I lose all other database configurations in the target.
The code I use is the following.
function ExtendJSON($base, $ext)
{
$propNames = $($ext | Get-Member -MemberType *Property).Name
foreach ($propName in $propNames) {
if ($base.PSObject.Properties.Match($propName).Count) {
if ($base.$propName.GetType().Name -eq "PSCustomObject")
{
$base.$propName = ExtendJSON $base.$propName $ext.$propName
}
else
{
$base.$propName = $ext.$propName
}
}
else
{
$base | Add-Member -MemberType NoteProperty -Name $propName -Value $ext.$propName
}
}
return $base
}
$tier = 'dev'
$parametersJsonGeneral = Get-Content -Path "./parameters/parameters_general.json" | ConvertFrom-Json
#Write-Output "#####################"
$parametersJsonTier = Get-Content -Path "./parameters/parameters_$($tier).json" | ConvertFrom-Json # overwrites existing values in $parametersJsonGeneral
ExtendJSON $parametersJsonGeneral $parametersJsonTier
Is there a way to loop over the settings from the lowest level up to the higher, and replace only these?
The proposed answer does only work one 1st level THIS WORKS
function merger ($target, $source) {
$source.psobject.Properties | ForEach-Object {
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject' -and $target."$($_.Name)" ) {
merger $target."$($_.Name)" $_.Value
}
else {
$target | Add-Member -MemberType $_.MemberType -Name $_.Name -Value $_.Value -Force
}
}
}
$Json1 ='{
"a": {
"b":"asda"
},
"c": "asdasd"
}
' | ConvertFrom-Json
$Json2 = '{
"a": {
"b":"d"
}
}
' | ConvertFrom-Json
merger $Json1 $Json2
However with this I loose data in $Json1
$Json1 ='{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": true,
"test2": false
}
}
]
},
"c": "asdasd"
}
' | ConvertFrom-Json
$Json2 = '{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": false
}
}
]
}
}
' | ConvertFrom-Json
merger $Json1 $Json2
{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": false
}
}
]
},
"c": "asdasd"
}
test2 is gone!
I don't have enough rep to post general comments yet, so I'll have to post it as an answer:
I tested the code in this answer to another question, and it did what you asked. The advantage of this answer is that it doesn't depend on importing a separate module.
There's another answer in the same question that provides link to a more general module that provides for different types of merges (Left Join, Inner Join, etc.) here, which links to here. However, one commenter said that it didn't work with powershell 7.0.

iterate on key to get the contents of the key

IF I have json that looks like
{
"items": [
{
"name": "issue1",
"spec": {
"config": {
"setting1": abc,
"setting2": {
"name": "xyz"
}
},
"files": {
"name": "cde",
"path": "/home"
},
"program": {
"name": "apache"
}
}
}
]
}
and I want to have iteration at .items[0].spec where the key config,files,program's contents can be shown.. something like
config:
{
"setting1": abc,
"setting2": {
"name": "xyz"
}
}
files:
{
"name": "cde",
"path": "/home"
}
program:
{
"name": "apache"
}
and there might be things more or less than config/files/programs for each items.
I know that I can get the list of keys by jq -r '.items[0].spec| to_entries[].key' but not sure about formating the output and getting the contents of .items[].spec.xxxxx like above.
I also got very close by jq -r '{test: .items[0].spec | with_entries(select(.value != null)) }' but I want to seperate each item per key
Also this got me even closer! but how do you get the json syntax of the object to display uncompressed?
jq -r '.items[].spec | keys[] as $k | "\($k):\n \(.[$k])"'
The "," operator is the magic ingredient you seem to be looking for:
.items[].spec | keys[] as $k | $k, .[$k]

Combining all key value pairs in one using jq filter or jq play

I want to transform JSON data using jq filter
Json data:
{
"main": [
{
"firstKey": "ABCD",
"id": "12345",
"data": [
{
"name": "first_id",
"value": "first_id_value"
},
{
"name": "second_id",
"value": "second_id_value"
},
{
"name": "third_id",
"value": "third_id_value"
}
]
}
]
}
Expected OUTPUT:
{
"firstKey": "ABCD",
"id": "12345",
"data.name.first_id": "first_id_value",
"data.name.second_id": "second_id_value",
"data.name.third_id": "third_id_value"
}
After many trials and errors, I was near to expected output using following filter expression
[.main[]|{"firstKey", "id"},foreach .data[] as $item (0; "data.name.\($item.name)" as $a|$item.value as $b| {($a): $b})][]
Used foreach as objects under "data" are dynamic. the number of objects can differ.
The output for the above expression is:
{
"firstKey": "ABCD",
"id": "12345"
}
{
"data.name.first_id": "first_id_value"
}
{
"data.name.second_id": "second_id_value"
}
{
"data.name.third_id": "third_id_value"
}
But I want the objects of data to be under the same braces as 'firstKey' and 'id'.
LINK to JqPlay
Any suggestions will be helpful.
Since your structure is so rigid, you can cheat and use the built-in from_entries, which takes a list of {key, value} pairs and constructs an object:
.main[] |
{firstKey, id} +
(.data | map({key: "data.name.\(.name)", value}) |
from_entries)

Modifying array of key value in JSON jq

In case, I have an original json look like the following:
{
"taskDefinition": {
"containerDefinitions": [
{
"name": "web",
"image": "my-image",
"environment": [
{
"name": "DB_HOST",
"value": "localhost"
},
{
"name": "DB_USERNAME",
"value": "user"
}
]
}
]
}
}
And I would like to inplace modify the value for the matched key like so:
jq '.taskDefinition.containerDefinitions[0].environment[] | select(.name=="DB_USERNAME") | .value="new"' json
I got the output
{
"name": "DB_USERNAME",
"value": "new"
}
But I want more like in-place modify or the whole json from the original with new value modified, like this:
{
"taskDefinition": {
"containerDefinitions": [
{
"name": "web",
"image": "my-image",
"environment": [
{
"name": "DB_HOST",
"value": "localhost"
},
{
"name": "DB_USERNAME",
"value": "new"
}
]
}
]
}
}
Is it possible to do with jq or any known workaround?
Thank you.
Updated
For anyone looking for editing multi-values,
here is the approach I use
JQ=""
for e in DB_HOST=rds DB_USERNAME=xxx; do
k=${e%=*}
v=${e##*=}
JQ+="(.taskDefinition.containerDefinitions[0].environment[] | select(.name==\"$k\") | .value) |= \"$v\" | "
done
jq '${JQ%??}' json
I think there should be more concise way, but this seems working fine.
It is enough to assign to the path, if you are using |=, e.g.
jq '
(.taskDefinition.containerDefinitions[0].environment[] |
select(.name=="DB_USERNAME") | .value) |= "new"
' infile.json
Output:
{
"taskDefinition": {
"containerDefinitions": [
{
"name": "web",
"image": "my-image",
"environment": [
{
"name": "DB_HOST",
"value": "localhost"
},
{
"name": "DB_USERNAME",
"value": "new"
}
]
}
]
}
}
Here is a select-free solution using |=:
.taskDefinition.containerDefinitions[0].environment |=
map(if .name=="DB_USERNAME" then .value = "new"
else . end)
Avoiding select within the expression on the LHS of |= makes the solution more robust w.r.t. the version of jq being used.
You might like to consider this alternative to using |=:
walk( if type=="object" and .name=="DB_USERNAME"
then .value="new" else . end)

jq json parser concate nested array object value

Hi I have the below JSON file with nested object:
{
"Maps": {
"Campus": [
{
"name": "nus",
"Building": [
{
"name": "sde1",
"Floor": [
{
"name": "floor1"
},
{
"name": "floor2"
}
]
},
{
"name": "sde2"
}
]
},
{
"name": "ntu",
"Building": [
{
"name": "ece1",
"Floor": [
{
"name": "floor1"
},
{
"name": "floor2"
}
]
},
{
"name": "ece2"
}
]
}
]
}
}
I want to use jq to parse the above JSON file and get the below format:
nus>sde1>floor1
nus>sde1>floor2
ntu>ece1>floor1
ntu>ece1>floor2
basically I have to concatenate the Campus Name with Building Name and Floor name and put a < symbol in between.
If the nested object field Floor is not exist, ignore the parse and continue the next child object.
How to achieve that? thanks.
You can use the following jq command:
jq '.Maps.Campus[]|"\(.name)>\(.Building[]|"\(.name)>\(.Floor[]?.name)")"' file.json
jq is smart enough to print the combinations of .name and .Building[].name since .Building is an array. The same action get's applied to .Building[].name and Floor[]?.name. ? because floor is not always set.
Here is a solution which uses jq variables
.Maps.Campus[]
| .name as $campus
| .Building[]
| .name as $bldg
| .Floor[]?
| .name as $floor
| "\($campus)>\($bldg)>\($floor)"