Modifying JSON file using data from CSV in PowerShell - json

I'm trying to modify some specific values in a .json file based on two columns in a .csv file. If the current value in the .json file is identical to the one in the left column, I want to change it to the one in the right column.
This is my first time with PowerShell though, so I'm struggling to figure out how to go about doing this. I feel like my solution is not only wrong, but is using a double for loop when it might not need to. Here's what I have so far.
$jsonData = Get-Content -Path $jsonFile | ConvertFrom-Json
$csvData = Get-Content -Path $csvFile | Select-Object -Skip 1 # Skipping the header
foreach ($jsonItem in $jsonData.'Placeable List') {
foreach ($csvRow in $csvData) {
$splitRow = $csvRow -split ","
$lCol = $splitRow[0]
$rCol = $splitRow[1]
$currentItem = $jsonItem.'value'.'Appearance'.'value'
if ($currentItem -eq $lCol) {
$currentItem -eq $rCol
}
}
}

I managed to figure it out.
$csvData = Get-Content -Path $csvFile | Select-Object -Skip 1 # Skipping the header
$jsonData = Get-Content -Path $jsonFile -raw | ConvertFrom-Json
foreach($csvRow in $csvData) {
$splitRow = $csvRow -split ","
$lCol = $splitRow[0]
$rCol = $splitRow[1]
foreach($item in $jsonData.'Placeable List'.value) {
$item.Appearance | % {
if ($_.value -eq $lCol) {
$_.value = $rCol
}
}
}
}
$jsonData | ConvertTo-Json -depth 32 | Set-Content $jsonFile

Related

marge embedded JSON objects with powershell

I have following ConvertFrom-Json output from JSON file:
Id : 1
ItemName : TestFile
SharingInformation : {#{RecipientEmail=complianceadmin#dev.onmicrosoft.com; ResharePermission=Read}, #{RecipientEmail=test#dev.onmicrosoft.com; ResharePermission=Read}}
I would like to save this data to .csv file in following manner as columns:
Id : 1
ItemName : TestFile
Users : (read) test#dev.domain.com ; (write) test2#dev.domain.com
as columns.. Here you can find part of my actual PS code (which do not work properly when there is more than one embedded values):
$JSONFile = $ExctratedFile | ConvertFrom-Json
$psObjectForCsv = $JSONFile | ForEach-Object {
[PSCustomObject]#{
"id"=$_.Id
"ItemName"=$_.ItemName
"RecipientEmail"=$_.SharingInformation.RecipientEmail
"ResharePermission"=$_.SharingInformation.ResharePermission
}
}
$psObjectForCsv | Export-Csv -path $fileName -Force -NoTypeInformation
}
do you have any ideas how to achieve this?
Thank you for your help!
Regards
You can format the SharingInformation in any form you like.
Try
$JSONFile = $ExctratedFile | ConvertFrom-Json
$result = $JSONFile | ForEach-Object {
# output a formatted string "(permission) emailaddress"
$users = foreach ($shareInfo in $_.SharingInformation) {
'({0}) {1}' -f $shareInfo.ResharePermission, $shareInfo.RecipientEmail
}
[PSCustomObject]#{
id = $_.Id
ItemName = $_.ItemName
Users = $users -join "; "
}
}
$result | Export-Csv -Path $fileName -Force -NoTypeInformation

Powershell - Reading multiple json files in a directory and output the data

I'm having to whip up a process that will read multiple json files created by another process.
I have code that can read a single file, but we're needing to process these results in bulk.
Here's my current code:
$json = Get-ChildItem $filePath -recurse | Where-Object { $_.LastWriteTime -gt [DateTime] $filesNewerThan } | ConvertFrom-Json
$json.delegates | foreach-Object {
foreach ($File in $_.files)
{
[PSCustomObject]#{
LastName = $_.lastName
ZipCode = $File.zipCode
BirthDate = $File.birthdate
Address = $File.Address}
}
}
Right now I'm getting an error about an "invalid JSON primitive" which what I'm guessing is an issue where I don't have "Get-Content" specified in my code.
Wondering what my issue is with my code.
ConvertFrom-Json currently (as of PowerShell 7.0) doesn't support file-path input, only file-content input (the actual JSON string), which means that you need to involve Get-Content:
$json = Get-ChildItem -File $filePath -Recurse |
Where-Object { $_.LastWriteTime -gt [DateTime] $filesNewerThan } |
ForEach-Object { Get-Content -Raw -LiteralPath $_.FullName | ConvertFrom-Json }

Access Object From JSON File in Powershell

I have a JSON file that I am reading in Powershell. The structure of the file is below.
[
["computer1", ["program1", versionX]],
["computer2", ["program2", versionY]],
["computer3", ["program3", "versionX"],
["program1", "versionZ"]
],
]
What I want in the program is use $env:computername and compare it with the computerX in the JSON file. If found a match, then iterate through and get the values of programName and ProgramVersion.
However, I don't know how to search through the objects and find ALL items under that.
This is what I have so far.
$rawData = Get-Content -Raw -Path "file.json" | ConvertFrom-Json
$computername=$env:computername
$data = $rawData -match $computername
This gives me objects under it. But how do I iterate through and get individual values?
But don't know what I do after that.
To start you need to be using a valid JSON file
{
"computer1": {
"program1": "versionX"
},
"computer2": {
"program2": "versionY"
},
"computer3": {
"program3": "versionX",
"program1": "versionZ"
}
}
Then you can access the PSObject Properties
$rawData = Get-Content -Raw -Path "file.json" | ConvertFrom-Json
$rawData.PsObject.Properties |
Select-Object -ExpandProperty Name |
ForEach-Object { IF ($_ -eq $env:COMPUTERNAME) {
Write-Host "Computer Name : " $_
Write-Host "Value : " $rawData."$_"
}
}
EDIT for Computer, Program, and Version as separate values
psobject.Properties.Name will give all the program names.
psobject.Properties.Name[0] will give the first program name.
psobject.Properties.value[0] will give the first program version value.
You need to increment the value to get second value, you can also use -1 as a shortcut for the last value.
$rawData = Get-Content -Raw -Path "file.json" | ConvertFrom-Json
$rawData.PsObject.Properties |
Select-Object -ExpandProperty Name |
ForEach-Object { IF ($_ -eq $env:COMPUTERNAME) {
$Computer = $_
$Values = $rawData.$_
}
}
$Computer
$Values.psobject.Properties
$Values.psobject.Properties.Name
$Values.psobject.Properties.Name[0]
$Values.psobject.Properties.value[0]
$Values.psobject.Properties.Name[1]
$Values.psobject.Properties.value[1]
You could also use the program name
$Values.program1
$Values.program2
$Values.program3

Where-Object with complex evaluation

I have a PowerShell script where I read in a CSV file, and if the date in a certain column is greater than a parameter date, I output that row to a new file.
As of now, I read the CSV file and then pipe to a ForEach-Object where if the row "passes" I store it in an Arraylist. Then when all the rows are processed, I output the Arraylist to an output CSV file. My starting CSV file is 225MB with over a quarter million rows, meaning that this process is slow.
Is there a way I can add a filter function to my piping so that only the passing rows are passed to the output CSV in one fell swoop? The current Where-Object just uses things like -like, -contains... and not more complex forms of evaluation.
For best practices, I've got my code below:
Import-Csv -Delimiter "`t" -Header $headerCounter -Path $filePath |
Select-Object -Skip(1) |
ForEach-Object {
#Skip the header
if( $lineCounter -eq 1)
{
return
}
$newDate = if ([string]::IsNullOrEmpty($_.1) -eq $true)
{ [DateTime]::MinValue }
else { [datetime]::ParseExact($_.1,”yyyyMMdd”,$null) }
$updateDate = if ([string]::IsNullOrEmpty($_.2) -eq $true)
{ [DateTime]::MinValue }
else { [datetime]::ParseExact($_.2,”yyyyMMdd”,$null) }
$distanceDate = (Get-Date).AddDays($daysBack * -1)
if( $newDate -gt $distanceDate -or $updateDate -gt $distanceDate )
{
[void]$filteredArrayList.Add($_)
}
}
...
$filteredArrayList |
ConvertTo-Csv -Delimiter "`t" -NoTypeInformation |
select -Skip 1 |
% { $_ -replace '"', ""} |
out-file $ouputFile -fo -en unicode -Append
I've added ConvertToDate as a function to stop that confusing the Where block.
DistanceDate is out because it appears to be calculated only once.
ExportCsv is a little function that writes pipeline input to a file.
I haven't tested it, so bugs are quite likely unless I got lucky.
function ConvertToDate {
param(
[String]$DateString
)
if ($DateString -eq '') {
return [DateTime]::MinValue
} else {
return [DateTime]::ParseExact($DateString, ”yyyyMMdd”, $null)
}
}
filter ExportCsv {
param(
[Parameter(Position = 1)]
[String]$Path
)
$csv = $_ | ConvertTo-Csv -Delimiter "`t" | Select-Object -Last 1
$csv -replace '"' | Out-File $Path -Append -Encoding Unicode -Force
}
$distanceDate = (Get-Date).AddDays($daysBack * -1)
Import-Csv -Delimiter "`t" -Header $headerCounter -Path $filePath |
Select-Object -Skip 1 |
Where-Object { (ConvertToDate $_.1) -gt $distanceDate -or (ConvertToDate $_.2) -gt $distanceDate } |
ExportCsv $OutputFile
Sure, just add a function that takes a value from the pipeline and pipe the result of Import-Csv to it. Within the function you check whether you want to filter the current item or not. Here a simple example which uses a string list and filter all strings that starts with h:
$x = #('hello', 'world', 'hello', 'tree')
filter Filter-CsvByMyRequirements
{
Param(
[Parameter(Mandatory=$true,
ValueFromPipeline=$true)]
$InputObject
)
Process
{
if ($_ -match '^h.*')
{
$_
}
}
}
$x | Filter-CsvByMyRequirements | Write-Host
Output:
hello
hello

How can I combine fields in a .csv based off of a shared value in powershell?

I have two files in identical formats, one containing destination IP addresses and URLs, and one that contains only the destination IP addresses. I am attempting to write a powershell script to add the URL field from the first file to the second file for that row if the destination IP addresses are equal. Here is an example of the two files:
File Containing URLs:
Date;Time;Source;Destination;Port;User;URL
3/7/2016;0:00:07;168.254.25.6;10.0.1.27;80;jsmith;abcnet
File to add URLs to:
Date;Time;Source;Destination;Port;User;URL
3/7/2016;0:00:09;168.254.25.6;10.0.1.27;80;;
Whenever I run the code below, it appears to be caught in an infinite loop because it does not run to completion, but it throws no errors. My data set is thousands of lines long, but it works when I test it with a sample set that is only a few lines long.
$noURLs = Import-Csv C:\Path\to\noURLs.csv
$containsURLs = Import-Csv C:\Path\to\containsURLs.csv | Select-Object Destination, URL
$outputFile = "C:\Path\to\output.csv"
if(Test-Path $outputFile){
Remove-Item $outputFile
}
foreach($line in $noURLs){
$cpDest = $line.Destination
$destURL = $containsURLs | Where-Object {$_.Destination -eq $cpDest} | Select-Object -ExpandProperty URL | Select-Object -Unique
if($destURL -ne $null){
if( $destURL.Count -gt 1) {
$destURL = $destURL -join ';'
}
}
$line.URL = $destURL
}
$noURLs | Export-Csv $outputFile
I forgot to add a -unique switch to my select object, so for every one record in the first csv, it was looping through every single line of the second csv. Fixed code looks like this:
$noURLs = Import-Csv C:\Path\to\noURLs.csv
$containsURLs = Import-Csv C:\Path\to\containsURLs.csv | Select-Object -Unique Destination, URL
$outputFile = "C:\Path\to\output.csv"
if(Test-Path $outputFile){
Remove-Item $outputFile
}
foreach($line in $noURLs){
$cpDest = $line.Destination
$destURL = $containsURLs | Where-Object {$_.Destination -eq $cpDest} | Select-Object -ExpandProperty URL | Select-Object -Unique
if($destURL -ne $null){
if( $destURL.Count -gt 1) {
$destURL = $destURL -join ';'
}
}
$line.URL = $destURL
}
$noURLs | Export-Csv $outputFile -NoTypeInformation