I am getting some json data from a API, however I don't need most of this data. I am trying to remove some fields so that the when I save this data as a json file it isn't so large. I doesnt seem to be removing any of the fields I am trying to remove.
Code:
$Response = Invoke-RestMethod -Uri "https://mtgjson.com/api/v5/AllPrintings.json" -Method GET
$Obj = ConvertFrom-Json $Response
$Obj.PSObject.Properties.Remove('booster')
$Obj.PSObject.Properties.Remove('cards')
$Obj | ConvertTo-Json | Out-File ./All-Sets-Data.json -Force
Json:
{
"data": {
"10E": {
"baseSetSize": 383,
"block": "Core Set",
"booster": "#{default=}",
"cards": "",
"code": "10E",
...
},
"2ED": {
"baseSetSize": 302,
"block": "Core Set",
"booster": "#{default=}",
"cards": "",
"code": "2ED",
...
},
"2XM": {
"baseSetSize": 332,
"booster": "#{default=}",
"cards": "",
"code": "2XM",
...
},
...
}
}
$Obj.data.'10E'.PSObject.Properties.Remove('booster')
$Obj.data.'10E'.PSObject.Properties.Remove('cards')
$Obj.data.'2ED'.PSObject.Properties.Remove('booster')
# and so on
The above code snippet should work. However, you can do all in a one step by calling the following (recursive) function RemoveProperty:
Function RemoveProperty {
param (
# A PSCustomObject
[Parameter( Mandatory, ValueFromPipeline )] $Object,
# A list of property names to remove
[Parameter( Mandatory )] [string[]]$PropList,
# recurse?
[Parameter()] [Switch]$Recurse
)
# Write-Host $Object -ForegroundColor Cyan
foreach ( $Prop in $PropList ) {
$Object.PSObject.Properties.Remove($prop)
}
# Write-Host $Object -ForegroundColor Green
if ( $Recurse.IsPresent ) {
foreach ($ObjValue in $Object.PSObject.Properties.Value) {
# Write-Host $ObjValue -ForegroundColor Yellow
if ( $ObjValue.GetType().Name -eq 'PSCustomObject' ) {
$ObjValue | RemoveProperty -PropList $PropList -Recurse
}
}
}
}
# sample usage:
$Obj = ConvertFrom-Json $Response
RemoveProperty -Object $Obj -PropList 'booster','cards' -Recurse
$Obj | ConvertTo-Json | Out-File ./All-Sets-Data.json -Force
(Please note that the RemoveProperty function contains some Write-Host in commented lines; originally used used for debugging purposes).
Related
I'm trying to remove the length value pair from the following JSON file:
{
"uuid": "6f74b1ba-0d7c-4c85-955b-2a4309f0e8df",
"records": {
"record1": [
{
"locale": "en_US",
"category": "alpha",
"contents": "My hovercraft is full of eels",
"length": 29
}
],
"record2": [
{
"locale": "cs_CZ",
"category": "alpha",
"contents": "Moje vznášedlo je plné úhořů",
"length": 28
}
]
}
}
Even though the length property is apparently found, it's not deleted, because the output file is identical to the input file.
I'm using the following code:
$infile = "C:\Temp\input.json"
$outfile = "C:\Temp\output.json"
$json = Get-Content $infile -Encoding UTF8 | ConvertFrom-Json
$records = $json.records
$records.PSObject.Properties | ForEach-Object {
if (($_.Value | Get-Member -Name "length")) {
Write-Host "length property found."
$_.Value.PSObject.Properties.Remove("length")
}
}
$json | ConvertTo-Json -Depth 3 | Out-File $outfile -Encoding UTF8
What am I doing wrong?
The record* properties are arrays, so you need a nested loop to process them:
foreach( $property in $records.PSObject.Properties ) {
foreach( $recordItem in $property.Value ) {
if( $recordItem | Get-Member -Name 'length' ) {
$recordItem.PSObject.Properties.Remove( 'length' )
}
}
}
For code clarity and performance I've replaced the ForEach-Object command by the foreach statement. Especially in nested loops, foreach helps to improve clarity as we no longer have to think about the context of the automatic $_ variable. Also the foreach statement is faster as it doesn't involve pipeline overhead.
This question already has answers here:
Unexpected ConvertTo-Json results? Answer: it has a default -Depth of 2
(2 answers)
Closed 11 months ago.
The owners key in my output (see OutputFile) I'm expecting as a line separated array, but it's outputting as a single-line space separated object/string
Script:
function Add-ApplicationOwner
{
param (
[string] $App,
[object] $OutputObject
)
# add values to our json output
$owners = (Get-AzureAdApplicationOwner -ObjectId $App).UserPrincipalName
$OutputObject | Add-Member -MemberType NoteProperty -Name owners -Value $owners
}
$inputFile = Get-Content -Path "AppInput.json" -Raw | ConvertFrom-Json
$outputFile = New-Object -TypeName PsObject
foreach ($object in $inputFile.PSObject.Properties)
{
$outputAppList = New-Object -TypeName PsObject
foreach ($app in $inputFile.New.PsObject.Properties)
{
# create app
$appRegistration = New-AzureADApplication -DisplayName "TestSPN1"
#add application info into json object
$outputAppValues = [PsCustomObject]#{
app_id = $appRegistration.AppId
}
#add application owners by object id
Add-ApplicationOwner -App $appRegistrationObjectId -OutputObject $outputAppValues
$outputAppList | Add-Member -MemberType NoteProperty -Name "TestSPN1" -Value $outputAppValues
}
# add all created apps into json output file
$outputFile | Add-Member -MemberType NoteProperty -Name "New Applications" -Value $outputAppList
}
$outputFile | ConvertTo-Json | Out-File "AzADAppRegistrationInfo.json" -Append
OutputFile:
{
"New Applications": {
"TestSPN1": {
"app_id": "dsfadfdafa-3afadfdafadsfasd-343",
"owners": "user1 user2 user3"
}
}
}
Desired Output:
{
"New Applications": {
"TestSPN1": {
"app_id": "dsfadfdafa-3afadfdafadsfasd-343",
"owners": [
"user1",
"user2",
"user3"
]
}
}
}
$owners Variable Examined:
$owners
user1
user2
user3
$owners.gettype()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
When I look at $outputFile.'New Applications' it's as expected
$outputFile.'New Applications' | convertto-json
{
"TestSPN1": {
"app_id": "asdfdsfad",
"owners": [
"user1",
"user2",
"user3"
]
}
}
When I look at $outputFile it's flattened
$outputFile | convertto-json
{
"New Applications": {
"TestSPN1": {
"app_id": "cc6dgfsdgdsgfsdgfdsa5562614",
"owners": "user1 user2 user3"
}
}
}
The most likable explanation for your issue is that -Depth is using the default Value. I have stored $outputFile.'New Applications' in the $json variable for below example.
[pscustomobject]#{
'New Applications' = $json
} | ConvertTo-Json
Results in:
WARNING: Resulting JSON is truncated as serialization has exceeded the set depth of 2.
{
"New Applications": {
"TestSPN1": {
"app_id": "asdfdsfad",
"owners": "user1 user2 user3"
}
}
}
Worth pointing out that the Warning Message is only displayed on newer versions of PowerShell (PS 7.1+ to be precise, thanks mklement0 for pointing it out). Windows PowerShell defaults to truncate the JSON without any warning.
However if we add 1 depth level (Default -Depth value is 2):
[pscustomobject]#{
'New Applications' = $json
} | ConvertTo-Json -Depth 3
Results in:
{
"New Applications": {
"TestSPN1": {
"app_id": "asdfdsfad",
"owners": [
"user1",
"user2",
"user3"
]
}
}
}
I have a JSON file I need to edit, conditionally. It may be an empty object:
{}
or it may contain other data.
I need to see if the data I'd like to add already exists, and if not, add it to that JSON file.
The content in question looks like this (entire JSON file):
{
{
"2020.3.19f1": {
"version": "2020.3.19f1",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2020.3.19f1\\Editor\\Unity.exe"
],
"manual": true
}
}
In this case, if "2020.3.19f" does not exist, I need to add that block.
I looked at these docs but really, lost. Any tips appreciated: https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/convertto-json?view=powershell-7.2
This seems close but I'm lost on the syntax of checking for null or empty, and how that translates to PS: PowerShell : retrieve JSON object by field value
Edit: So, for example if the original file is:
{}
Then I need to overwrite that file with:
{
{
"2020.3.19f1": {
"version": "2020.3.19f1",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2020.3.19f1\\Editor\\Unity.exe"
],
"manual": true
}
}
And if the file already contained something, I need to keep that, just add the new block:
{
{
"2019.4.13f1": {
"version": "2019.4.13f1",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2019.3.13f1\\Editor\\Unity.exe"
],
"manual": true
},
{
"2020.3.19f1": {
"version": "2020.3.19f1",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2020.3.19f1\\Editor\\Unity.exe"
],
"manual": true
}
}
FWIW: I did find the condition I need:
$FileContent = Get-Content -Path "C:\Users\me\AppData\Roaming\UnityHub\editors.json" -Raw | ConvertFrom-Json
if ( ($FileContent | Get-Member -MemberType NoteProperty -Name "2020.3.19f1") -ne $null )
{
echo "it exists"
}
else
{
echo "add it"
# DO SOMETHING HERE TO CREATE AND (OVER)WRITE THE FILE
}
You can convert the json as object and add properties as you want.
$json = #"
{
"2020.3.19f2": {
"version": "2020.3.19f2",
"location": [
"C:\\Program Files\\Unity\\Hub\\Editor\\2020.3.19f2\\Editor\\Unity.exe"
],
"manual": true
}
}
"#
$obj = ConvertFrom-Json $json
if (-not $obj.'2020.3.19f1') {
Add-Member -InputObject $obj -MemberType NoteProperty -Name '2020.3.19f1' -Value $(
New-Object PSObject -Property $([ordered]#{
version = "2020.3.19f1"
location = #("C:\Program Files\Unity\Hub\Editor\2020.3.19f1\Editor\Unity.exe")
manual = $true
})
) -Force
$obj | ConvertTo-Json
}
I have a script that I am using to read a JSON file, read the paths, calculate the MD5 hash and then export them in a CSV file. The JSON file contains multiple objects. In another post, I laid out the structure of the JSON file and I'll post that here. Please note that I am not allowed to change this JSON.
{
"destinationpath": "C:\\Destination\\Mobile Phones\\"
"sourcepath": "C:\\Source\\Mobile Phones\\"
"OnePlus" : {
"files": [
{
"source": "6T",
"destination": "Model\\6T",
"log": "logfiles",
"version": "Version.txt",
}
]
}
"Samsung" : {
"files": [
{
"source": "S20",
"destination": "Galaxy\\S20",
"log": "logfiles",
"version": "Version.txt",
}
]
}
}
The destinationpath: C:\\Destination\\Mobile Phones\\, when added to, for example, Samsung it becomes: destinationpath: C:\\Destination\\Mobile Phones\\Galaxy\\S20.
The script that I have so far is this:
$JSON = Get-Content -Path file.json | ConvertFrom-Json
$JSONdestination = $JSON.destination
$JSONsource = $JSON.source
foreach ($i in $JSON.psobject.properties) {
foreach($i2 in $i.Value ){
$IgnoredFiles = #(
$logfiles = $i2.files.log
$newFiles = $i2.files.version
)
$destination = $JSON + $i2.files.destination
$source = $JSONdestination + $i2.files.source
echo $destination
Get-ChildItem -Path $destination -Recurse -Exclude $IgnoredFiles | Get-FileHash -Algorithm MD5 -ErrorAction SilentlyContinue | Export-csv -Append -Path "C:\Users\home\Downloads\hashes.csv"
}
In the Get-ChildItem portion, I want to exclude the "log" and "version" files so that they are not counted in the Get-FileHash. I have tried doing it this way as well:
Get-ChildItem -Path $destination -Recurse -Exclude $i2.files.log, $i2.files.version | Get-FileHash -Algorithm MD5 -ErrorAction SilentlyContinue | Export-csv -Append -Path "C:\Users\home\Downloads\hashes.csv"
However, it doesn't work either and the log and version files are still taken into account in the exported csv.
The json you show us is invalid (missing commas and commas too many..)
Once fixed into
{
"destinationpath": "C:\\Destination\\Mobile Phones\\",
"sourcepath": "C:\\Source\\Mobile Phones\\",
"OnePlus": {
"files": [{
"source": "6T",
"destination": "Model\\6T",
"log": "*.log",
"version": "Version.txt"
}]
},
"Samsung": {
"files": [{
"source": "S20",
"destination": "Galaxy\\S20",
"log": "*.log",
"version": "Version.txt"
}]
}
}
you can do this:
$JSON = Get-Content -Path file.json | ConvertFrom-Json
$JSONdestination = $JSON.destinationpath
$JSONsource = $JSON.sourcepath
$result = foreach ($item in ($JSON | Select-Object * -ExcludeProperty destinationpath, sourcepath)) {
$item.PSObject.Properties.Value | ForEach-Object {
$destPath = Join-Path -Path $JSONdestination -ChildPath $_.files.destination
$logFolder = Join-Path -Path $JSONdestination -ChildPath $_.files.log
$exclude = $_.files.version
Write-Host "Processing files in '$destPath'" -ForegroundColor Cyan
Get-ChildItem -Path $destPath -Recurse -Exclude $exclude -ErrorAction SilentlyContinue |
Where-Object {$_.FullName -notlike "$logFolder*" } |
Get-FileHash -Algorithm MD5
}
}
$result | Export-Csv -Path "C:\Users\home\Downloads\hashes.csv" -NoTypeInformation
I have a JSON File called index.json which looks like this :
{
"First": {
"href": "test/one two three.html",
"title": "title one"
},
"Second": {
"href": "test/test test/one two three four.html",
"title": "title two"
}
}
I want to write a powershell script to update the href of each object to replace the spaces with -.
The JSON file should looks like this:
{
"First": {
"href": "test/one-two-three.html",
"title": "title one"
},
"Second": {
"href": "test/test-test/one-two-three-four.html",
"title": "title two"
}
}
I got some help from this post:
Iterating through a JSON file PowerShell
I have already written a script to get all the href values, I dont know how to update the same in the original JSON file. My script looks like this:
function Get-ObjectMembers {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True, ValueFromPipeline=$True)]
[PSCustomObject]$obj
)
$obj | Get-Member -MemberType NoteProperty | ForEach-Object {
$key = $_.Name
[PSCustomObject]#{Key = $key; Value = $obj."$key"}
}
}
$a = Get-Content 'index.json' -raw | ConvertFrom-Json | Get-ObjectMembers
foreach($i in $a){
$i.Value.href.Replace(" ","-")
}
I've done it this way.
$path='index.json'
$Content= (Get-Content $path -raw | ConvertFrom-Json)
$memberNames=( $Content | Get-Member -MemberType NoteProperty).Name
foreach($memberName in $memberNames){
($Content.$memberName.href)=($Content.$memberName.href).Replace(" ","-")
}
$Content | ConvertTo-Json | Out-File $path -Append