Remove Object from Array of Objects in Json/powershell - json

I am importing JSON into Powershell to replace some values, but I also need to remove some objects from there.
Import:
$fileJson = Get-Content -path/Template.json -Encoding UTF8
I have an array of object which looks like this:
{
"resources": [
{
"name": "name1"
"type": "type1"
"....": "....."
},
{
"name": "name2"
"type": "type2"
"....": "....."
},
{
"name": "name3"
"type": "type1"
"....": "....."
}
]
}
and I want to remove a specific object from this array of objects. For example I want to remove Object where "type" equals "type2".
I have already tried to replace values with .Replace, however I can only replace single values and not the complete object.
Is it possible to delete or skip entire object with condition?

Convert the JSON to a custom object:
$fileJson = Get-Content -path/Template.json -Encoding UTF8
$data = $fileJson |ConvertFrom-Json
Use Where-Object to filter the resources array:
$data.resources = #($data.resources |Where-Object type -ne type2)
Convert the now modified object back to JSON and write to disk:
$data |ConvertTo-Json |Set-Content ./path/to/updatedTemplate.json -Encoding UTF8

I suggest to convert json into PowerShell objects, then make desired changes. then convert back to json if needed.
Example
$root = Get-Content -Path "C:\source.json" | ConvertFrom-Json
$root.resources = $root.resources | where type -eq 'type2'
$root | ConvertTo-Json -Depth 5 | Out-File "C:\destination.json"

Related

Convert CSV file to JArray

I have a need to export the json results of a Invoke-WebRequest call to a .CSV file so that non-technical users can edit the data. This is what the json looks like when I save it to a .CSV file:
{
"Forecasts": [
{
"AccountNumber": "12345",
"Period": "2022-02-01T00:00:00",
"Field1": 3998.87968239,
"Field2": 2133.91206875
},
{
"AccountNumber": "12346",
"Period": "2022-03-01T00:00:00",
"Field1": 6741.483,
"Field2": 4007.857
}
]
}
I'm creating the .CSV file with this line of code:
$json.Forecasts | ForEach-Object { $_ | Export-Csv -path $fileName -NoTypeInformation -Append -Force }
And this is working good. After the user edits the data and saves and closes the .CSV file, I need to convert the .CSV data back into a JArray that looks like the original json structure (like above) so I can make another Invoke-WebRequest call (Patch), passing the JArray as the Body.
When I execute this line of code:
$editedJson = Get-Content -Raw -Path "$fileName" | ConvertFrom-CSV | Select-Object -Property AccountNumber,Period,Field1,Field2 | ConvertTo-json
The json looks like this:
[
{
"AccountNumber": "12345",
"Period": "2022-02-01T00:00:00",
"Field1": "3900.87968239",
"Field2": "2100.91206875"
},
{
"AccountNumber": "12346",
"Period": "2022-03-01T00:00:00",
"Field1": "6700.483",
"Field2": "4000.857"
}
]
As you can see, it's missing the "Forecasts" portion of the JArray and I don't know how to get the .CSV data back into this format. Can anyone help with this ?
You simply need to create an object with that property (Forecasts) and attach the imported CSV to said property:
# This line would be where you import the CSV:
# $csv = Import-Csv .....
$csv = #'
"AccountNumber","Period","Field1","Field2"
"12345","2/1/2022 12:00:00 AM","3998.87968239","2133.91206875"
"12346","3/1/2022 12:00:00 AM","6741.483","4007.857"
'# | ConvertFrom-Csv
#{
Forecasts = $csv
} | ConvertTo-Json
If you want to assure that the JSON representation of Forecasts is always an array (even when the CSV only has one row):
#{
Forecasts = [array]$csv
} | ConvertTo-Json

Powershell: High Performing conversion of Name and Value Array into Parseable format - How can I make this Faster

How can I make my code a lot more performant when I wish to make an easily/fast parseable object/PSCustomObject from a JSON payload $JSON?
An example of the structure of the PAYLOAD I receive is:
[
{
"name": "system.enablenetflow",
"value": "false"
},
{
"name": "system.deviceGroupId",
"value": "186,3060"
},
{
"name": "system.prefcollectorid",
"value": "144"
},
{
"name": "system.collectorplatform",
"value": "windows"
}
]
As you can see its in a very annoying format.
Note that the payloads I attempt to parse are much larger and variable in count from 500 of these Name/Value objects to 50000, rather than just the 4 listed above.
###########################################################################
MY GOAL
To have this turn into a key:value pair scenario for easier parsing later
NOT This:
With the JSON I have to do $JSON.where({$_.name -eq "system.enablenetflow"}).value
YES THIS:
I want the end state to be that the new variable $obj I create will let me get the value with $obj."system.enablenetflow"
###########################################################################
MY CURRENT ATTEMPT THAT IS SUPER SLOW
I did the following:
Create an Empty PSCustomObject and saved it as variable $obj
Did a foreach method on the $JSON variable which iterated through the JSON Array
Add-Member to $obj with setting the 'name' as PropertyName and 'value' as PropertyValue
Heres a sample of my code:
$obj = [PSCustomObject] #{}
$json.foreach({
$thisitem = $_
$obj | Add-member -NotePropertyName $($thisitem.name) -NotePropertyValue $($thisitem.name)
})
HOW CAN I MAKE THIS FASTER?
# Sample input JSON.
$json = #'
[
{
"name": "system.enablenetflow",
"value": "false"
},
{
"name": "system.deviceGroupId",
"value": "186,3060"
},
{
"name": "system.prefcollectorid",
"value": "144"
},
{
"name": "system.collectorplatform",
"value": "windows"
}
]
'#
# Initialize the (ordered) result hashtable.
$result = [ordered] #{}
# Note: In PowerShell (Core) 7+, add -AsHashTable to the ConvertFrom-Json
# call for additional performance gain, combined with -AsArray,
# in which case you don't need the `(...)` around the call anymore.
foreach ($element in (ConvertFrom-Json $json)) {
$result[$element.name] = $element.value
}
The above creates an (ordered) hashtable instead of a [pscustomobject] instance - especially if the latter are iteratively constructed via Add-Member calls.
Hashtables are lighter-weight and faster to construct than [pscustomobject] instances.
Using a foreach loop rather than processing the ConvertFrom-Json output in a pipeline via ForEach-Object also speeds up processing.
PowerShell allows you to use the familiar dot notation also with hashtables; so, for instance, after running the above, you'll get:
PS> $result.'system.collectorplatform'
windows
If you do need $result to be a [pscustomobject] instance, you can simply cast the fully populated hashtable to that type:
PS> $obj = [pscustomobject] $result; $obj.'system.collectorplatform'
windows

Convert PowerShell JSON to a CSV file

I have not used PowerShell much. I have an API request which returns JSON that I need to turn into a CSV or xlsx file.
$output = Get-SurveyParticipents `
-url "https://orxsurveys.limequery.com/admin/remotecontrol" `
-session $sessionKey `
-id "5133965" `
-start "0" `
-limit "2" `
-unused $False `
-attributes ["completed", "usesleft"]
Write-Host($output | ConvertTo-Json)
{
"id": 1,
"result": [
{
"tid": "6",
"token": "35ddmyQTlNpzLat",
"participant_info": "#{firstname=Hsdfng; lastname=Gsdfh; email=gosdfdsfz.com}"
},
{
"tid": "7",
"token": "nQ_S838LjYT4mR6",
"participant_info": "#{firstname=Ofdlga; lastname=Yadfdfa; email=olsdfdsfivska#axsdfdsfnce.com}"
}
],
"error": null
}
The participant_info doesn't look like a normal JSON structure.
Also I am unsure how to turn this JSON into a CSV. Something like:
$output | ConvertTo-Json | Export-Csv -Path "c:\Scripts"
You don't need to do both Export-Csv and ConvertTo-Json. Export-Csv will convert a stream of powershell objects into a file containing CSV records.
Assuming the objects in the "results" array in your json output are what you want, you might try something like:
$output.result | Export-Csv -NoTypeInformation foo.csv
You are correct to say that participant_info is not Json. You'll have to process this yourself. ConvertFrom-StringData might be helpful.

Replace column value with null in json file using Powershell

Here is how my json file looks like :
{
"count": 12,
"name": "Daily Ticket",
"columnNames": [
"User",
"Channel",
"Date",
"# of Closed Incidents",
"Open",
"Response",
"Remark",
"Closed"
],
"rows": [
[
"abc",
"Service Web",
"\u00272020-06-13 00:00:00\u0027",
"1",
"0",
"0",
"this is a text,please replace with null",
"1"
],
[
"xyz",
"Email",
"\u00272020-06-13 00:00:00\u0027",
"21",
"1",
"0",
"this is a text,please replace with null",
"7"
]
]
}
I want to replace all the values in columns of Remark with null and convert into a csv file using powershell. Please help to achieve this.
I want column names as header and rows as rows separated with comma in csv.
My output csv file should look like below one:
User,Channel,Date,# of Closed Incidents,Open,Response,Remark,Closed
abc,Service Web,\u00272020-06-13 00:00:00\u0027,1,0,0,,1
xyz,Email,\u00272020-06-13 00:00:00\u0027,1,0,0,,1
To convert this json into a CSV file is not that difficult.
Just load the JSON, convert it into an object and loop through the properties building an array of new objects you can save as CSV:
$json = Get-Content -Path 'D:\Test\DailyTicket.json' -Raw | ConvertFrom-Json
$headers = $json.columnNames
$result = foreach ($row in $json.rows) {
# just a precaution to not run into index errors when there are
# more items in the array than there are headers or vice-versa
$items = [math]::Min($row.Count, $headers.Count)
# create a new empty (ordered) hashtable
$hash = [ordered]#{}
for ($i = 0; $i -lt $items; $i++) {
# fill the hashtable, except for iten 'Remark'
$hash[$headers[$i]] = if ($headers[$i] -ne 'Remark') { $row[$i] } else { $null }
}
# If you insist on keeping the apostrophe characters in the date field in unicode format `\u0027`
# $hash['Date'] = $hash['Date'] -replace "'", '\u0027'
# output a PSObject to be collected in array $result
[PsCustomObject]$hash
}
# output on screen
$result | Format-Table -AutoSize
# output to CSV file
$result | Export-Csv -Path 'D:\Test\DailyTicket.csv' -NoTypeInformation
Resulting CSV file:
"User","Channel","Date","# of Closed Incidents","Open","Response","Remark","Closed"
"abc","Service Web","'2020-06-13 00:00:00'","1","0","0",,"1"
"xyz","Email","'2020-06-13 00:00:00'","21","1","0",,"7"

How do I update JSON file using PowerShell

I have one json file mytest.json like below I want to update values using PowerShell script
update.json
{
"update": [
{
"Name": "test1",
"Version": "2.1"
},
{
"Name": "test2",
"Version": "2.1"
}
]
}
I want to write a PowerShell script where if Name=="test1" I want to update Version= "3"
How can i do it using parameters?
Here is a way :
$a = Get-Content 'D:\temp\mytest.json' -raw | ConvertFrom-Json
$a.update | % {if($_.name -eq 'test1'){$_.version=3.0}}
$a | ConvertTo-Json -depth 32| set-content 'D:\temp\mytestBis.json'
According to #FLGMwt and #mikemaccana I improve the ConvertTo-Json with -depth 32 because the default depth value is 2 and for object deeper than 2 you will receive class informations in spite of objects.
I have also faced the same kind of issue. I was looking to change the records of the below JSON file
{
"SQS_QUEUE_URL": "https://que-url.com/server1",
"SQS_EVENTS_QUEUE_URL": "https://events-server.com/server1/development_events",
"REGION": "region1",
"BUCKET": "test-bucket",
"AE_WORK_PATH": "C:\\workpath\\path1",
"ENV": "env"
}
Finally, I managed to find the easiest way to generate a JSON file from Powershell.
$json = Get-Content "c:\users\bharat.gadade\desktop\test.json" | ConvertFrom-Json
$json.SQS_QUEUE_URL = "https://que-url.com/server2"
$json.SQS_EVENTS_QUEUE_URL = "https://events-server.com/Server2/development_events"
$json.REGION = "region1 "
$json.BUCKET = "test-bucket"
$json.AE_WORK_PATH = "C:\workpath\path1"
$json.ENV = "env"
$json | ConvertTo-Json | Out-File "c:\users\bharat.gadade\desktop\test.json"