Powershell can't seem to correctly round-trip this JSON object:
{
"settings": {
"minimumApproverCount": 2,
"creatorVoteCounts": false,
"scope": [
{
"refName": "refs/heads/d14rel",
"matchKind": "Exact",
"repositoryId": "a290117c-5a8a-40f7-bc2c-f14dbe3acf6d"
}
]
}
}
Assuming $json is a string, this command:
$json | ConvertFrom-Json | ConvertTo-Json
produces the wrong JSON out of it:
{
"settings": {
"minimumApproverCount": 2,
"creatorVoteCounts": false,
"scope": [
"#{refName=refs/heads/d14rel; matchKind=Exact; repositoryId=a290117c-5a8a-40f7-bc2c-f14db
e3acf6d}"
]
}
}
Notice it gets the "scope" variable wrong. Is there a way to fix this?
Use the parameter Depth with value 3 or larger. The default 2 is not enough, deeper data are simply converted to strings.
$json | ConvertFrom-Json | ConvertTo-Json -Depth 3
Output
{
"settings": {
"minimumApproverCount": 2,
"creatorVoteCounts": false,
"scope": [
{
"refName": "refs/heads/d14rel",
"matchKind": "Exact",
"repositoryId": "a290117c-5a8a-40f7-bc2c-f14dbe3acf6d"
}
]
}
}
Related
I have a text file with below data which is Dynamically generated, so the contents of the repo.txt keep changing
repo.txt -> Content as below
repo1
repo2
repo_3
And a JSON file
template.json -> Content as below
{
"name": "new-report-test-1",
"resources": {
"repositories": [
{
"name": "repoa"
},
{
"name": "repo_b"
}
]
},
"filters": {
"severities": [
"High"
]
}
}
now i want to read the repo list from the repo.txt file and update the template.json to below
{
"name": "new-report-test-1",
"resources": {
"repositories": [
{
"name": "repoa"
},
{
"name": "repo_b"
},
{
"name": "repo1"
},
{
"name": "repo2"
},
{
"name": "repo_3"
}
]
},
"filters": {
"severities": [
"High",
"Medium",
"Critical"
]
}
}
I have this bash script that does the job thiugh I'm unsure how to translate it to PoweShell. Any inputs would be highly appreciated
cat ./tmpgen_reponames.txt | while read repoEntry do echo " Repository: $repoEntry" cat <<< $(jq --arg REPOID "$repoEntry" '[ .[] , {"name":$REPOID} ]' tmpgen_repoarray.json) > tmpgen_repoarray.json done
You can accomplish this in PowerShell using ConvertFrom-Json to convert your Json into objects then following an object-oriented approach adding new custom objects to the .resources.repositories property and adding the new values Medium and Critical to the .filters.severities property of your Json, lastly, converting it back to a Json string with ConvertTo-Json:
$json = Get-Content path\to\template.json -Raw | ConvertFrom-Json
$json.resources.repositories = #(
$json.resources.repositories
(Get-Content path\to\repo.txt).Trim() | ForEach-Object {
[pscustomobject]#{ Name = $_ }
}
)
$json.filters.severities = #(
$json.filters.severities
'Medium'
'Critical'
)
$json | ConvertTo-Json -Depth 99 | Set-Content path\to\newJson.Json
I have a big target json file (parameters_general.json) where all common settings for a deployment are set.
For each tier I have another json file (ex: parameters_dev.json, parameters_test.json, ....) Settings set in one needs tobe added to the general.json, or overwrite it when already in general.
ex: parameters_general.json
{
"general": {
"db_Policy": {
"type": "Periodic",
"databaseAccountOfferType": "Standard",
"periodicModeProperties": {
"backupIntervalInMinutes": 240,
"backupRetentionIntervalInHours": 8,
"backupStorageRedundancy": "Local"
}
},
"databases": [
{
"name": "CtrlWps",
"Containers": [
{
"name": "ControllerAuthentication",
"partitionKey": "id"
}
],
"ContainersTTL": []
},
{
"name": "CpoOcpi",
"Containers": [
{
"name": "Cpos",
"partitionKey": "cpoId"
},
{
"name": "OcpiCdrLastRecoveries",
"partitionKey": "id"
},
{
"name": "Routes",
"partitionKey": "ocpiCpoId"
}
],
"ContainersTTL": [
{
"name": "OcpiCdrs",
"partitionKey": "pk",
"ttl": 172800
},
{
"name": "OcppTransactionIds",
"partitionKey": "pk",
"ttl": 172800
},
{
"name": "Sessions",
"partitionKey": "pk",
"ttl": 172800
}
]
}
],
"system_engineers": [
],
}
}
If I want to updatethis with.
ex: parameters_test.json
{
"general": {
"system_engineers": [
{
"name": "hans",
"AppPrincipalId": "<id>",
"permissions": [
"get",
"list"
]
},
{
"name": "John do",
"AppPrincipalId": "<pid>",
"permissions": [
"all"
]
}
]
}
}
This works, the users are added to the empty "sytem_engineers" node in the parameters_general.json.
However, If I just want to change a setting on a lower node example:
ex: parameters_dev.json
{
"general": {
"databases": [
{
"name": "CtrlWps",
"Containers": [
{
"name": "ControllerAuthentication",
"partitionKey": "pk"
}
]
}
]
}
}
for replacing the partitionKey in one of the databases it replaces the whole "databases" part so I lose all other database configurations in the target.
The code I use is the following.
function ExtendJSON($base, $ext)
{
$propNames = $($ext | Get-Member -MemberType *Property).Name
foreach ($propName in $propNames) {
if ($base.PSObject.Properties.Match($propName).Count) {
if ($base.$propName.GetType().Name -eq "PSCustomObject")
{
$base.$propName = ExtendJSON $base.$propName $ext.$propName
}
else
{
$base.$propName = $ext.$propName
}
}
else
{
$base | Add-Member -MemberType NoteProperty -Name $propName -Value $ext.$propName
}
}
return $base
}
$tier = 'dev'
$parametersJsonGeneral = Get-Content -Path "./parameters/parameters_general.json" | ConvertFrom-Json
#Write-Output "#####################"
$parametersJsonTier = Get-Content -Path "./parameters/parameters_$($tier).json" | ConvertFrom-Json # overwrites existing values in $parametersJsonGeneral
ExtendJSON $parametersJsonGeneral $parametersJsonTier
Is there a way to loop over the settings from the lowest level up to the higher, and replace only these?
The proposed answer does only work one 1st level THIS WORKS
function merger ($target, $source) {
$source.psobject.Properties | ForEach-Object {
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject' -and $target."$($_.Name)" ) {
merger $target."$($_.Name)" $_.Value
}
else {
$target | Add-Member -MemberType $_.MemberType -Name $_.Name -Value $_.Value -Force
}
}
}
$Json1 ='{
"a": {
"b":"asda"
},
"c": "asdasd"
}
' | ConvertFrom-Json
$Json2 = '{
"a": {
"b":"d"
}
}
' | ConvertFrom-Json
merger $Json1 $Json2
However with this I loose data in $Json1
$Json1 ='{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": true,
"test2": false
}
}
]
},
"c": "asdasd"
}
' | ConvertFrom-Json
$Json2 = '{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": false
}
}
]
}
}
' | ConvertFrom-Json
merger $Json1 $Json2
{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": false
}
}
]
},
"c": "asdasd"
}
test2 is gone!
I don't have enough rep to post general comments yet, so I'll have to post it as an answer:
I tested the code in this answer to another question, and it did what you asked. The advantage of this answer is that it doesn't depend on importing a separate module.
There's another answer in the same question that provides link to a more general module that provides for different types of merges (Left Join, Inner Join, etc.) here, which links to here. However, one commenter said that it didn't work with powershell 7.0.
I want to transform JSON data using jq filter
Json data:
{
"main": [
{
"firstKey": "ABCD",
"id": "12345",
"data": [
{
"name": "first_id",
"value": "first_id_value"
},
{
"name": "second_id",
"value": "second_id_value"
},
{
"name": "third_id",
"value": "third_id_value"
}
]
}
]
}
Expected OUTPUT:
{
"firstKey": "ABCD",
"id": "12345",
"data.name.first_id": "first_id_value",
"data.name.second_id": "second_id_value",
"data.name.third_id": "third_id_value"
}
After many trials and errors, I was near to expected output using following filter expression
[.main[]|{"firstKey", "id"},foreach .data[] as $item (0; "data.name.\($item.name)" as $a|$item.value as $b| {($a): $b})][]
Used foreach as objects under "data" are dynamic. the number of objects can differ.
The output for the above expression is:
{
"firstKey": "ABCD",
"id": "12345"
}
{
"data.name.first_id": "first_id_value"
}
{
"data.name.second_id": "second_id_value"
}
{
"data.name.third_id": "third_id_value"
}
But I want the objects of data to be under the same braces as 'firstKey' and 'id'.
LINK to JqPlay
Any suggestions will be helpful.
Since your structure is so rigid, you can cheat and use the built-in from_entries, which takes a list of {key, value} pairs and constructs an object:
.main[] |
{firstKey, id} +
(.data | map({key: "data.name.\(.name)", value}) |
from_entries)
I have the following JSON in a Powershell variable:
{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
I want to get JSON where the value of criteria is 1. The result therefore should look as follows:
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
I made an attempt with the following code:
$json | Get-ObjectMembers | Select-Object | where { $_.value.criteria -eq 1 };
While this basically goes into the right direction, it is not what I want exactly, because the result looks like this:
{
"name": "asdf2",
"criteria": 1
}
See that the Object2 information is lost and one depth-level is lost.
How can I achieve the desired result as shown above?
In essence, you're looking to only retain properties of interest from your single input object or, to put it differently, to remove properties you're not interested in.
Here's a PSv4+ solution:
$json = #'
{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
'#
($json | ConvertFrom-Json).psobject.Properties.
Where({ $_.Value.criteria -eq 1 }).
ForEach({ [pscustomobject] #{ $_.Name = $_.Value } }) |
ConvertTo-Json
The above yields:
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
The jq solution. with_entries temporarily turns a list of properties into key and value pairs. https://stedolan.github.io/jq/manual/#to_entries,from_entries,with_entries (section id's in html come in handy) (Json creators don't believe in arrays?)
$json = '{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}'
$json | jq 'with_entries(select(.value.criteria == 1))'
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
I'm trying to ingest a JSON file into Powershell, append a block of JSON to an existing node (Components), then convert the PSCustomObject back to JSON and save the file. The JSON I'm playing with looks something like Figure 1.
As you see in my code, I run ConvertTo-Json to cast the data into a PSCustomObject, and I then append a new object to the Components node. If I view the object, $configFile in this case it all looks fine, but when I convert back to JSON the items in the Components node, are treated as strings and not evaluated into JSON (see last snippet). I imagine this is because ConvertTo-JSON treats arrays literally, but not 100% sure.
If someone can advise how to ensure the PSCustomObjects in the Components node get casted back to JSON properly I would be grateful, thank you.
Figure 1 - the original JSON:
{
"EngineConfiguration": {
"PollInterval": "00:00:15",
"Components": [
{
"Id": "ApplicationEventLog",
"FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"LogName": "Application",
"Levels": "1"
}
},
{
"Id": "SystemEventLog",
"FullName": "AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"LogName": "System",
"Levels": "7"
}
}
],
"Flows": {
"Flows":
[
"(ApplicationEventLog,SystemEventLog),CloudWatchLogs"
]
}
}
}
Figure 2 - my code:
#Requires -Version 3.0
$configFile = "C:\Program Files\Amazon\EC2ConfigService\Settings\AWS.EC2.Windows.CloudWatch.json"
$configToPSObject = ConvertFrom-Json "$(Get-Content $configFile)"
$configToPSObject.EngineConfiguration.Components += New-Object -Type PSObject -Property ([ordered]#{
"Id" = "IISRequestQueueSize"
"FullName" = "AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,AWS.EC2.Windows.CloudWatch"
"Parameters" = [PSCustomObject]#{
"CategoryName" = "HTTP Service Request Queues"
"CounterName" = "CurrentQueueSize"
"InstanceName" = "_Total"
"MetricName" = "IISRequestQueueSize"
"Unit" = ""
"DimensionName" = ""
"DimensionValue" = ""
}
})
$configJson = ConvertTo-Json -Depth 5 $configToPSObject
Set-Content -Path $configFile -Value $configJson
Figure 3 - the JSON output:
{
"EngineConfiguration": {
"PollInterval": "00:00:15",
"Components": [
"#{Id=ApplicationEventLog; FullName=AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch; Parameters=}",
"#{Id=SystemEventLog; FullName=AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch; Parameters=}",
"#{Id=IISRequestQueueSize; FullName=AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,AWS.EC2.Windows.CloudWatch; Parameters=}"
],
"Flows": {
"Flows":
"(ApplicationEventLog,SystemEventLog),CloudWatchLogs"
}
}
}
If I increase the depth to say, 8 or beyond, the JSON comes out as follows:
{
"EngineConfiguration": {
"PollInterval": "00:00:15",
"Components": [
"#{Id=ApplicationEventLog; FullName=AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch; Parameters=}",
"#{Id=SystemEventLog; FullName=AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,AWS.EC2.Windows.CloudWatch; Parameters=}",
"Id": "IISRequestQueueSize",
"FullName": "AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"CategoryName": "HTTP Service Request Queues",
"CounterName": "CurrentQueueSize",
"InstanceName": "_Total",
"MetricName": "IISRequestQueueSize",
"Unit": "",
"DimensionName": "",
"DimensionValue": ""
}
}
],
"Flows": {
"Flows": "(ApplicationEventLog,SystemEventLog),CloudWatchLogs"
}
}
}
The ConvertTo-Json cmdlet also has a Depth parameter, beyond which an object is treated with toString() instead of going deeper with recursion. So just setting that parameter to whatever max depth of objects you have should result in a correctly formed JSON.
$configJson = ConvertTo-Json $configToPSObject -Depth 8
# your JSON has depth of 5, get some extra
You have to supply the depth for the ConvertTo-Json commandlet.
Otherwise it only does the first level and leaves the subnodes as is and converts them to a string apparently.
$configJson = ConvertTo-Json $obj -Depth 3