Unable to set iteratively values from array object to json objects - json

I am a newbie to powershell, and I am trying to create function that copies an object from a json data, creates new object from it and assigns different values to the max parameter. So far, different versions of my implementation assigns only the last value from an array $ParameterValues to all the new objects created.
One solution, might be probabbly be to read the json data using a call by reference [ref]$jsonData. However, I am not even sure that is a thing in powershell.
--- Here is the sample json file:
"algorithms": {
"obj0": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 7
}
}
--- Function Select-Member helps select the object to copy
function Select-Member {
[CmdLetBinding()]
param(
[Parameter(Mandatory)]
[string]$Path,
[Parameter(Mandatory)]
[Object]$InputParameter
)
Write-Debug $InputParameter
$Path -split '/' | ForEach-Object { $selected = $InputParameter } { $selected = $selected.$_ } { $selected }
}
--- Function Set-Member pastes the new object copied using Select-Member or overwrites the values of the parameter selected
function Set-Member {
[CmdletBinding()]
param(
[Object]$Value,
[string[]]$Path,
[Object]$Object
)
$Head, $Next, $Tail = $Path
if (($null -eq $Next) -or (1 -gt $Next.Length)) {
Add-Member -Passthru -Force -MemberType NoteProperty `
-Input $Object -Name $Head -Value $Value
}
else {
Add-Member -Passthru -Force -MemberType NoteProperty `
-Input $Object -Name $Head `
-Value (Set-Member -Value $Value -Path ([string[]]$Next + $Tail) -Object ($Object.$Head))
}
}
--- test-func function uses Select-Member to select an object (obj0 for example), then Set-Member adds new copies (obj1, obj2, obj3) of the object selected and then it should assign a new value to Max iteratatively from $ParameterValues array.
function test-func {
[CmdletBinding()]
param (
[Parameter(Mandatory, Position)]
[string] $ObjectSelect,
[Parameter(Mandatory, Position)]
[string] $Parameter,
[Parameter(Mandatory, Position)]
[Object[]] $ParameterValues
)
$PathToPaste = ($ObjectSelect -Split '/', -2)[0] -Split '/'
$JsonData = Get-content "$JSON_FILE" -raw | convertFrom-Json
$ObjectSelected = Select-Member $ObjectSelect $JsonData
[string]$NewObjName
foreach ($ele in $ParameterValues){
$NewObjName = "obj" + $ele
Set-Member $ObjectSelected $NewObjName $JsonData.$PathToPaste
$PathToParameter = $PathToPaste, $NewObjName, $Parameter-Split'/'
Set-Member $ele $PathToParameter $JsonData
}
$JsonData | ConvertTo-Json -depth 32 | set-content "$JSON_FILE"
}
When I run the following command for example, test-func -ObjectSelect algorithms/obj0 -Parameter parameters/max -ParameterValues 1,2,3 to iteratatively assign each value from the $ParameterValues to max, it sets only the last value from the array to all the new objects created, obj1, obj2, obj3.
Here is the results I get. (Observe that the value max in the last three objects (obj1, obj2, obj3) are all identical = 3.
"algorithms": {
"obj0": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 7
}
},
"obj1": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 3
}
},
"obj2": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 3
}
},
"obj3": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 3
}
}
}
The expected results should be like this (Observe that the value of max are 1, 2, 3 for the last three objects (obj1, obj2, obj3) respectively.):
"algorithms": {
"obj0": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 7
}
},
"obj1": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 1
}
},
"obj2": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 2
}
},
"obj3": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 3
}
}
}
As of now, the only solution I have is using the following for-each loop in the test-func` function, which is definitely not professional. Notice it saves and reads again the json file on line 4 and 5. Like I said I am sure this is not professional as it will take up memory and time.
foreach ($ele in $ParameterValues){ $NewObjName = "obj" + $ele Set-Member $ObjectSelected $NewObjName $JsonData.$PathToPaste $JsonData | ConvertTo-Json -depth 32 | set-content "$JSON_FILE" $JsonData = Get-content "$JSON_FILE" -raw | convertFrom-Json $PathToParameter = $PathToPaste, $NewObjName, $Parameter-Split'/' Set-Member $ele $PathToParameter $JsonData }

The main issue here is that in the loop, you are still referencing the initial object. So the last $ele passed will change the values of all new objects created in that way.
You to use psobject.copy() to create a copy of the new object. But there is a catch, this only creates a shallow copy, that is if you have nested objects (where the properties contain other objects), only the top-level values are copied. The child objects will reference each other.
Since you are trying to update the value of one of the parameters. You will have to introduce something like this in the for-loop.
$ThisObject = $JsonData.($TargetPath[...]).psObject.Copy()
$ThisObject.$TargetProperty = $ParameterValues[$i]
$JsonData.($TargetPath[...])| Add-Member -Name "obj$($i+1)" -Value
$ThisObject -MemberType NoteProperty
Another solution will be to create a deep copy, especially if you have multiple updates to make at different depths. For more on that, you can visit this link

Related

How to remove nested JSON object members

I'm trying to remove the length value pair from the following JSON file:
{
"uuid": "6f74b1ba-0d7c-4c85-955b-2a4309f0e8df",
"records": {
"record1": [
{
"locale": "en_US",
"category": "alpha",
"contents": "My hovercraft is full of eels",
"length": 29
}
],
"record2": [
{
"locale": "cs_CZ",
"category": "alpha",
"contents": "Moje vznášedlo je plné úhořů",
"length": 28
}
]
}
}
Even though the length property is apparently found, it's not deleted, because the output file is identical to the input file.
I'm using the following code:
$infile = "C:\Temp\input.json"
$outfile = "C:\Temp\output.json"
$json = Get-Content $infile -Encoding UTF8 | ConvertFrom-Json
$records = $json.records
$records.PSObject.Properties | ForEach-Object {
if (($_.Value | Get-Member -Name "length")) {
Write-Host "length property found."
$_.Value.PSObject.Properties.Remove("length")
}
}
$json | ConvertTo-Json -Depth 3 | Out-File $outfile -Encoding UTF8
What am I doing wrong?
The record* properties are arrays, so you need a nested loop to process them:
foreach( $property in $records.PSObject.Properties ) {
foreach( $recordItem in $property.Value ) {
if( $recordItem | Get-Member -Name 'length' ) {
$recordItem.PSObject.Properties.Remove( 'length' )
}
}
}
For code clarity and performance I've replaced the ForEach-Object command by the foreach statement. Especially in nested loops, foreach helps to improve clarity as we no longer have to think about the context of the automatic $_ variable. Also the foreach statement is faster as it doesn't involve pipeline overhead.

How do I add a JSON element if and only if it does not already exist, using Powershell?

I have a JSON file I need to edit, conditionally. It may be an empty object:
{}
or it may contain other data.
I need to see if the data I'd like to add already exists, and if not, add it to that JSON file.
The content in question looks like this (entire JSON file):
{
{
"2020.3.19f1": {
"version": "2020.3.19f1",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2020.3.19f1\\Editor\\Unity.exe"
],
"manual": true
}
}
In this case, if "2020.3.19f" does not exist, I need to add that block.
I looked at these docs but really, lost. Any tips appreciated: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/convertto-json?view=powershell-7.2
This seems close but I'm lost on the syntax of checking for null or empty, and how that translates to PS: PowerShell : retrieve JSON object by field value
Edit: So, for example if the original file is:
{}
Then I need to overwrite that file with:
{
{
"2020.3.19f1": {
"version": "2020.3.19f1",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2020.3.19f1\\Editor\\Unity.exe"
],
"manual": true
}
}
And if the file already contained something, I need to keep that, just add the new block:
{
{
"2019.4.13f1": {
"version": "2019.4.13f1",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2019.3.13f1\\Editor\\Unity.exe"
],
"manual": true
},
{
"2020.3.19f1": {
"version": "2020.3.19f1",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2020.3.19f1\\Editor\\Unity.exe"
],
"manual": true
}
}
FWIW: I did find the condition I need:
$FileContent = Get-Content -Path "C:\Users\me\AppData\Roaming\UnityHub\editors.json" -Raw | ConvertFrom-Json
if ( ($FileContent | Get-Member -MemberType NoteProperty -Name "2020.3.19f1") -ne $null )
{
echo "it exists"
}
else
{
echo "add it"
# DO SOMETHING HERE TO CREATE AND (OVER)WRITE THE FILE
}
You can convert the json as object and add properties as you want.
$json = #"
{
"2020.3.19f2": {
"version": "2020.3.19f2",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2020.3.19f2\\Editor\\Unity.exe"
],
"manual": true
}
}
"#
$obj = ConvertFrom-Json $json
if (-not $obj.'2020.3.19f1') {
Add-Member -InputObject $obj -MemberType NoteProperty -Name '2020.3.19f1' -Value $(
New-Object PSObject -Property $([ordered]#{
version = "2020.3.19f1"
location = #("C:\Program Files\Unity\Hub\Editor\2020.3.19f1\Editor\Unity.exe")
manual = $true
})
) -Force
$obj | ConvertTo-Json
}

Powershell: High Performing conversion of Name and Value Array into Parseable format - How can I make this Faster

How can I make my code a lot more performant when I wish to make an easily/fast parseable object/PSCustomObject from a JSON payload $JSON?
An example of the structure of the PAYLOAD I receive is:
[
{
"name": "system.enablenetflow",
"value": "false"
},
{
"name": "system.deviceGroupId",
"value": "186,3060"
},
{
"name": "system.prefcollectorid",
"value": "144"
},
{
"name": "system.collectorplatform",
"value": "windows"
}
]
As you can see its in a very annoying format.
Note that the payloads I attempt to parse are much larger and variable in count from 500 of these Name/Value objects to 50000, rather than just the 4 listed above.
###########################################################################
MY GOAL
To have this turn into a key:value pair scenario for easier parsing later
NOT This:
With the JSON I have to do $JSON.where({$_.name -eq "system.enablenetflow"}).value
YES THIS:
I want the end state to be that the new variable $obj I create will let me get the value with $obj."system.enablenetflow"
###########################################################################
MY CURRENT ATTEMPT THAT IS SUPER SLOW
I did the following:
Create an Empty PSCustomObject and saved it as variable $obj
Did a foreach method on the $JSON variable which iterated through the JSON Array
Add-Member to $obj with setting the 'name' as PropertyName and 'value' as PropertyValue
Heres a sample of my code:
$obj = [PSCustomObject] #{}
$json.foreach({
$thisitem = $_
$obj | Add-member -NotePropertyName $($thisitem.name) -NotePropertyValue $($thisitem.name)
})
HOW CAN I MAKE THIS FASTER?
# Sample input JSON.
$json = #'
[
{
"name": "system.enablenetflow",
"value": "false"
},
{
"name": "system.deviceGroupId",
"value": "186,3060"
},
{
"name": "system.prefcollectorid",
"value": "144"
},
{
"name": "system.collectorplatform",
"value": "windows"
}
]
'#
# Initialize the (ordered) result hashtable.
$result = [ordered] #{}
# Note: In PowerShell (Core) 7+, add -AsHashTable to the ConvertFrom-Json
# call for additional performance gain, combined with -AsArray,
# in which case you don't need the `(...)` around the call anymore.
foreach ($element in (ConvertFrom-Json $json)) {
$result[$element.name] = $element.value
}
The above creates an (ordered) hashtable instead of a [pscustomobject] instance - especially if the latter are iteratively constructed via Add-Member calls.
Hashtables are lighter-weight and faster to construct than [pscustomobject] instances.
Using a foreach loop rather than processing the ConvertFrom-Json output in a pipeline via ForEach-Object also speeds up processing.
PowerShell allows you to use the familiar dot notation also with hashtables; so, for instance, after running the above, you'll get:
PS> $result.'system.collectorplatform'
windows
If you do need $result to be a [pscustomobject] instance, you can simply cast the fully populated hashtable to that type:
PS> $obj = [pscustomobject] $result; $obj.'system.collectorplatform'
windows

Manipulate JSON File with multiple objects using Powershell

I have a JSON File called index.json which looks like this :
{
"First": {
"href": "test/one two three.html",
"title": "title one"
},
"Second": {
"href": "test/test test/one two three four.html",
"title": "title two"
}
}
I want to write a powershell script to update the href of each object to replace the spaces with -.
The JSON file should looks like this:
{
"First": {
"href": "test/one-two-three.html",
"title": "title one"
},
"Second": {
"href": "test/test-test/one-two-three-four.html",
"title": "title two"
}
}
I got some help from this post:
Iterating through a JSON file PowerShell
I have already written a script to get all the href values, I dont know how to update the same in the original JSON file. My script looks like this:
function Get-ObjectMembers {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True, ValueFromPipeline=$True)]
[PSCustomObject]$obj
)
$obj | Get-Member -MemberType NoteProperty | ForEach-Object {
$key = $_.Name
[PSCustomObject]#{Key = $key; Value = $obj."$key"}
}
}
$a = Get-Content 'index.json' -raw | ConvertFrom-Json | Get-ObjectMembers
foreach($i in $a){
$i.Value.href.Replace(" ","-")
}
I've done it this way.
$path='index.json'
$Content= (Get-Content $path -raw | ConvertFrom-Json)
$memberNames=( $Content | Get-Member -MemberType NoteProperty).Name
foreach($memberName in $memberNames){
($Content.$memberName.href)=($Content.$memberName.href).Replace(" ","-")
}
$Content | ConvertTo-Json | Out-File $path -Append

Handling JSON Reference with ConvertFrom-Json

By default, PowerShell's ConvertFrom-Json cmdlet does not seem to handle references in JSON documents.
{
"foo": {"$ref": "#/bar"},
"bar": true
}
I don't really know about the official status of this JSON spec (see here and here), but is there a way to manage such a thing in PowerShell?
AFAIK there are no built-in ways to resolve JSON-references. I usually search for a C# solution for a problem that's not supported in PowerShell as those can be converted to PowerShell-code most of the time, but I couldn't find an easy way to do this in C# without custom code.
So I think you might need to write some functions yourself and use something like Invoke-Expression and reference types (which the converted object is as it is a PSCustomObject) to help you.
Proof of concept (contains many errors):
$json = #'
{
"foo": {"$ref": "#/bar"},
"bar": true
}
'#
$t = ConvertFrom-Json -InputObject $json
Write-Host "Before"
$t | Out-Host
function resolveReferences ($obj) {
#Loop through propeties
$obj.psobject.Properties | ForEach-Object {
#ref-values are PSCustomObjects with a $ref-property, so let's find the PSCustomObjects
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject') {
#Verify it was a ref-value by checking for $ref-property
if ($_.Value.'$ref') {
#Convert value to powershell-path like $t.foo
$refpath = $_.Value.'$ref'.Replace("#/",'$t.').Replace("/",".")
#Execute generated powershell-path to get the referenced object and replace reference with the referenced object
$_.Value = (Invoke-Expression -Command $refpath)
}
}
}
}
resolveReferences -obj $t
Write-host "After"
$t | Out-Host
Output:
Before
foo bar
--- ---
#{$ref=#/bar} True
After
foo bar
--- ---
True True
Which you can expand to fit your needs. Ex. support for array of objects:
$json = #'
{
"foo": {"$ref": "#/bar"},
"fooarray": [
{"item": {"$ref": "#/bar"}},
{"item": {"$ref": "#/hello"}}
],
"test": [1,2,3],
"bar": true,
"hello": "world"
}
'#
$t = ConvertFrom-Json -InputObject $json
Write-Host "Before"
$t | Out-Host
function resolveReferences ($obj) {
$obj.psobject.Properties | ForEach-Object {
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject') {
if ($_.Value.'$ref') {
$refpath = $_.Value.'$ref'.Replace("#/",'$t.').Replace("/",".")
$_.Value = (Invoke-Expression -Command $refpath)
}
} elseif ($_.TypeNameOfValue -eq 'System.Object[]') {
#If array, loop through objects (recursive search)
$_.Value | ForEach-Object { resolveReferences -obj $_ }
}
}
}
resolveReferences -obj $t
Write-host "After"
$t | Out-Host
Output:
#item properties in fooarray-objects are ref paths, it's just PS being to lazy to go deeper and show their values
Before
foo : #{$ref=#/bar}
fooarray : {#{item=}, #{item=}}
test : {1, 2, 3}
bar : True
hello : world
After
foo : True
fooarray : {#{item=True}, #{item=world}}
test : {1, 2, 3}
bar : True
hello : world