By default, PowerShell's ConvertFrom-Json cmdlet does not seem to handle references in JSON documents.
{
"foo": {"$ref": "#/bar"},
"bar": true
}
I don't really know about the official status of this JSON spec (see here and here), but is there a way to manage such a thing in PowerShell?
AFAIK there are no built-in ways to resolve JSON-references. I usually search for a C# solution for a problem that's not supported in PowerShell as those can be converted to PowerShell-code most of the time, but I couldn't find an easy way to do this in C# without custom code.
So I think you might need to write some functions yourself and use something like Invoke-Expression and reference types (which the converted object is as it is a PSCustomObject) to help you.
Proof of concept (contains many errors):
$json = #'
{
"foo": {"$ref": "#/bar"},
"bar": true
}
'#
$t = ConvertFrom-Json -InputObject $json
Write-Host "Before"
$t | Out-Host
function resolveReferences ($obj) {
#Loop through propeties
$obj.psobject.Properties | ForEach-Object {
#ref-values are PSCustomObjects with a $ref-property, so let's find the PSCustomObjects
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject') {
#Verify it was a ref-value by checking for $ref-property
if ($_.Value.'$ref') {
#Convert value to powershell-path like $t.foo
$refpath = $_.Value.'$ref'.Replace("#/",'$t.').Replace("/",".")
#Execute generated powershell-path to get the referenced object and replace reference with the referenced object
$_.Value = (Invoke-Expression -Command $refpath)
}
}
}
}
resolveReferences -obj $t
Write-host "After"
$t | Out-Host
Output:
Before
foo bar
--- ---
#{$ref=#/bar} True
After
foo bar
--- ---
True True
Which you can expand to fit your needs. Ex. support for array of objects:
$json = #'
{
"foo": {"$ref": "#/bar"},
"fooarray": [
{"item": {"$ref": "#/bar"}},
{"item": {"$ref": "#/hello"}}
],
"test": [1,2,3],
"bar": true,
"hello": "world"
}
'#
$t = ConvertFrom-Json -InputObject $json
Write-Host "Before"
$t | Out-Host
function resolveReferences ($obj) {
$obj.psobject.Properties | ForEach-Object {
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject') {
if ($_.Value.'$ref') {
$refpath = $_.Value.'$ref'.Replace("#/",'$t.').Replace("/",".")
$_.Value = (Invoke-Expression -Command $refpath)
}
} elseif ($_.TypeNameOfValue -eq 'System.Object[]') {
#If array, loop through objects (recursive search)
$_.Value | ForEach-Object { resolveReferences -obj $_ }
}
}
}
resolveReferences -obj $t
Write-host "After"
$t | Out-Host
Output:
#item properties in fooarray-objects are ref paths, it's just PS being to lazy to go deeper and show their values
Before
foo : #{$ref=#/bar}
fooarray : {#{item=}, #{item=}}
test : {1, 2, 3}
bar : True
hello : world
After
foo : True
fooarray : {#{item=True}, #{item=world}}
test : {1, 2, 3}
bar : True
hello : world
Related
From this comment:
How does one remove the initial data: from a JSON file?
data: [
{
...
},
{
...
}
]
To make it look like this and be able to parse it properly.
[
{
...
},
{
...
}
]
Get-Content reads a file line by line, which is wasted effort if the file's content is to be parsed as a whole as JSON.
The -Raw switch allows you to read the file as a whole, as a single, (usually) multi-line string.
The following solution uses -Raw and recognizes and removes any property name at the start of the file followed by :
(Get-Content -Raw -LiteralPath C:\path\path\file.json) -replace '^\w+:' |
ConvertFrom-Json
First import the content of the file to a variable:
$json = gc -Path C:\path\path\file.json
Then (assuming this is in the first line of the file), replace the content of that line:
$json[0] = $json[0] -replace '^data:', ''
This will now work.
$json | ConvertFrom-Json
Example:
# File content
data: [
{
"_id": "630528cd19c645e4f1c6be6d",
"index": 0,
"guid": "ee49651d-1296-4165-bada-6491ce6081a6"
}
]
PS> $json = Get-Content -path somefile.json
PS> $json[0] = $json[0] -replace '^data:', ''
PS> $json | ConvertFrom-Json
_id index guid
--- ----- ----
630528cd19c645e4f1c6be6d 0 ee49651d-1296-4165-bada-6491ce6081a6
Using the PowerShell ConvertFrom-Json parser
(and ConvertTo-Json if even required to go back to Json):
$Data = #'
data: [
{
a: "..."
},
{
a: "..."
}
]
'#
(ConvertFrom-Json "{$data}").Data |ConvertTo-Json -Depth 9
[
{
"a": "..."
},
{
"a": "..."
}
]
I'm trying to remove the length value pair from the following JSON file:
{
"uuid": "6f74b1ba-0d7c-4c85-955b-2a4309f0e8df",
"records": {
"record1": [
{
"locale": "en_US",
"category": "alpha",
"contents": "My hovercraft is full of eels",
"length": 29
}
],
"record2": [
{
"locale": "cs_CZ",
"category": "alpha",
"contents": "Moje vznášedlo je plné úhořů",
"length": 28
}
]
}
}
Even though the length property is apparently found, it's not deleted, because the output file is identical to the input file.
I'm using the following code:
$infile = "C:\Temp\input.json"
$outfile = "C:\Temp\output.json"
$json = Get-Content $infile -Encoding UTF8 | ConvertFrom-Json
$records = $json.records
$records.PSObject.Properties | ForEach-Object {
if (($_.Value | Get-Member -Name "length")) {
Write-Host "length property found."
$_.Value.PSObject.Properties.Remove("length")
}
}
$json | ConvertTo-Json -Depth 3 | Out-File $outfile -Encoding UTF8
What am I doing wrong?
The record* properties are arrays, so you need a nested loop to process them:
foreach( $property in $records.PSObject.Properties ) {
foreach( $recordItem in $property.Value ) {
if( $recordItem | Get-Member -Name 'length' ) {
$recordItem.PSObject.Properties.Remove( 'length' )
}
}
}
For code clarity and performance I've replaced the ForEach-Object command by the foreach statement. Especially in nested loops, foreach helps to improve clarity as we no longer have to think about the context of the automatic $_ variable. Also the foreach statement is faster as it doesn't involve pipeline overhead.
I am a newbie to powershell, and I am trying to create function that copies an object from a json data, creates new object from it and assigns different values to the max parameter. So far, different versions of my implementation assigns only the last value from an array $ParameterValues to all the new objects created.
One solution, might be probabbly be to read the json data using a call by reference [ref]$jsonData. However, I am not even sure that is a thing in powershell.
--- Here is the sample json file:
"algorithms": {
"obj0": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 7
}
}
--- Function Select-Member helps select the object to copy
function Select-Member {
[CmdLetBinding()]
param(
[Parameter(Mandatory)]
[string]$Path,
[Parameter(Mandatory)]
[Object]$InputParameter
)
Write-Debug $InputParameter
$Path -split '/' | ForEach-Object { $selected = $InputParameter } { $selected = $selected.$_ } { $selected }
}
--- Function Set-Member pastes the new object copied using Select-Member or overwrites the values of the parameter selected
function Set-Member {
[CmdletBinding()]
param(
[Object]$Value,
[string[]]$Path,
[Object]$Object
)
$Head, $Next, $Tail = $Path
if (($null -eq $Next) -or (1 -gt $Next.Length)) {
Add-Member -Passthru -Force -MemberType NoteProperty `
-Input $Object -Name $Head -Value $Value
}
else {
Add-Member -Passthru -Force -MemberType NoteProperty `
-Input $Object -Name $Head `
-Value (Set-Member -Value $Value -Path ([string[]]$Next + $Tail) -Object ($Object.$Head))
}
}
--- test-func function uses Select-Member to select an object (obj0 for example), then Set-Member adds new copies (obj1, obj2, obj3) of the object selected and then it should assign a new value to Max iteratatively from $ParameterValues array.
function test-func {
[CmdletBinding()]
param (
[Parameter(Mandatory, Position)]
[string] $ObjectSelect,
[Parameter(Mandatory, Position)]
[string] $Parameter,
[Parameter(Mandatory, Position)]
[Object[]] $ParameterValues
)
$PathToPaste = ($ObjectSelect -Split '/', -2)[0] -Split '/'
$JsonData = Get-content "$JSON_FILE" -raw | convertFrom-Json
$ObjectSelected = Select-Member $ObjectSelect $JsonData
[string]$NewObjName
foreach ($ele in $ParameterValues){
$NewObjName = "obj" + $ele
Set-Member $ObjectSelected $NewObjName $JsonData.$PathToPaste
$PathToParameter = $PathToPaste, $NewObjName, $Parameter-Split'/'
Set-Member $ele $PathToParameter $JsonData
}
$JsonData | ConvertTo-Json -depth 32 | set-content "$JSON_FILE"
}
When I run the following command for example, test-func -ObjectSelect algorithms/obj0 -Parameter parameters/max -ParameterValues 1,2,3 to iteratatively assign each value from the $ParameterValues to max, it sets only the last value from the array to all the new objects created, obj1, obj2, obj3.
Here is the results I get. (Observe that the value max in the last three objects (obj1, obj2, obj3) are all identical = 3.
"algorithms": {
"obj0": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 7
}
},
"obj1": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 3
}
},
"obj2": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 3
}
},
"obj3": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 3
}
}
}
The expected results should be like this (Observe that the value of max are 1, 2, 3 for the last three objects (obj1, obj2, obj3) respectively.):
"algorithms": {
"obj0": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 7
}
},
"obj1": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 1
}
},
"obj2": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 2
}
},
"obj3": {
"command": "...",
"parameters": {
"min": 2.7,
"max": 3
}
}
}
As of now, the only solution I have is using the following for-each loop in the test-func` function, which is definitely not professional. Notice it saves and reads again the json file on line 4 and 5. Like I said I am sure this is not professional as it will take up memory and time.
foreach ($ele in $ParameterValues){ $NewObjName = "obj" + $ele Set-Member $ObjectSelected $NewObjName $JsonData.$PathToPaste $JsonData | ConvertTo-Json -depth 32 | set-content "$JSON_FILE" $JsonData = Get-content "$JSON_FILE" -raw | convertFrom-Json $PathToParameter = $PathToPaste, $NewObjName, $Parameter-Split'/' Set-Member $ele $PathToParameter $JsonData }
The main issue here is that in the loop, you are still referencing the initial object. So the last $ele passed will change the values of all new objects created in that way.
You to use psobject.copy() to create a copy of the new object. But there is a catch, this only creates a shallow copy, that is if you have nested objects (where the properties contain other objects), only the top-level values are copied. The child objects will reference each other.
Since you are trying to update the value of one of the parameters. You will have to introduce something like this in the for-loop.
$ThisObject = $JsonData.($TargetPath[...]).psObject.Copy()
$ThisObject.$TargetProperty = $ParameterValues[$i]
$JsonData.($TargetPath[...])| Add-Member -Name "obj$($i+1)" -Value
$ThisObject -MemberType NoteProperty
Another solution will be to create a deep copy, especially if you have multiple updates to make at different depths. For more on that, you can visit this link
I have a JSON File called index.json which looks like this :
{
"First": {
"href": "test/one two three.html",
"title": "title one"
},
"Second": {
"href": "test/test test/one two three four.html",
"title": "title two"
}
}
I want to write a powershell script to update the href of each object to replace the spaces with -.
The JSON file should looks like this:
{
"First": {
"href": "test/one-two-three.html",
"title": "title one"
},
"Second": {
"href": "test/test-test/one-two-three-four.html",
"title": "title two"
}
}
I got some help from this post:
Iterating through a JSON file PowerShell
I have already written a script to get all the href values, I dont know how to update the same in the original JSON file. My script looks like this:
function Get-ObjectMembers {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True, ValueFromPipeline=$True)]
[PSCustomObject]$obj
)
$obj | Get-Member -MemberType NoteProperty | ForEach-Object {
$key = $_.Name
[PSCustomObject]#{Key = $key; Value = $obj."$key"}
}
}
$a = Get-Content 'index.json' -raw | ConvertFrom-Json | Get-ObjectMembers
foreach($i in $a){
$i.Value.href.Replace(" ","-")
}
I've done it this way.
$path='index.json'
$Content= (Get-Content $path -raw | ConvertFrom-Json)
$memberNames=( $Content | Get-Member -MemberType NoteProperty).Name
foreach($memberName in $memberNames){
($Content.$memberName.href)=($Content.$memberName.href).Replace(" ","-")
}
$Content | ConvertTo-Json | Out-File $path -Append
I am getting some json data from a API, however I don't need most of this data. I am trying to remove some fields so that the when I save this data as a json file it isn't so large. I doesnt seem to be removing any of the fields I am trying to remove.
Code:
$Response = Invoke-RestMethod -Uri "https://mtgjson.com/api/v5/AllPrintings.json" -Method GET
$Obj = ConvertFrom-Json $Response
$Obj.PSObject.Properties.Remove('booster')
$Obj.PSObject.Properties.Remove('cards')
$Obj | ConvertTo-Json | Out-File ./All-Sets-Data.json -Force
Json:
{
"data": {
"10E": {
"baseSetSize": 383,
"block": "Core Set",
"booster": "#{default=}",
"cards": "",
"code": "10E",
...
},
"2ED": {
"baseSetSize": 302,
"block": "Core Set",
"booster": "#{default=}",
"cards": "",
"code": "2ED",
...
},
"2XM": {
"baseSetSize": 332,
"booster": "#{default=}",
"cards": "",
"code": "2XM",
...
},
...
}
}
$Obj.data.'10E'.PSObject.Properties.Remove('booster')
$Obj.data.'10E'.PSObject.Properties.Remove('cards')
$Obj.data.'2ED'.PSObject.Properties.Remove('booster')
# and so on
The above code snippet should work. However, you can do all in a one step by calling the following (recursive) function RemoveProperty:
Function RemoveProperty {
param (
# A PSCustomObject
[Parameter( Mandatory, ValueFromPipeline )] $Object,
# A list of property names to remove
[Parameter( Mandatory )] [string[]]$PropList,
# recurse?
[Parameter()] [Switch]$Recurse
)
# Write-Host $Object -ForegroundColor Cyan
foreach ( $Prop in $PropList ) {
$Object.PSObject.Properties.Remove($prop)
}
# Write-Host $Object -ForegroundColor Green
if ( $Recurse.IsPresent ) {
foreach ($ObjValue in $Object.PSObject.Properties.Value) {
# Write-Host $ObjValue -ForegroundColor Yellow
if ( $ObjValue.GetType().Name -eq 'PSCustomObject' ) {
$ObjValue | RemoveProperty -PropList $PropList -Recurse
}
}
}
}
# sample usage:
$Obj = ConvertFrom-Json $Response
RemoveProperty -Object $Obj -PropList 'booster','cards' -Recurse
$Obj | ConvertTo-Json | Out-File ./All-Sets-Data.json -Force
(Please note that the RemoveProperty function contains some Write-Host in commented lines; originally used used for debugging purposes).