Powershell replace values in target.json with values from source.json - json

I have a big target json file (parameters_general.json) where all common settings for a deployment are set.
For each tier I have another json file (ex: parameters_dev.json, parameters_test.json, ....) Settings set in one needs tobe added to the general.json, or overwrite it when already in general.
ex: parameters_general.json
{
"general": {
"db_Policy": {
"type": "Periodic",
"databaseAccountOfferType": "Standard",
"periodicModeProperties": {
"backupIntervalInMinutes": 240,
"backupRetentionIntervalInHours": 8,
"backupStorageRedundancy": "Local"
}
},
"databases": [
{
"name": "CtrlWps",
"Containers": [
{
"name": "ControllerAuthentication",
"partitionKey": "id"
}
],
"ContainersTTL": []
},
{
"name": "CpoOcpi",
"Containers": [
{
"name": "Cpos",
"partitionKey": "cpoId"
},
{
"name": "OcpiCdrLastRecoveries",
"partitionKey": "id"
},
{
"name": "Routes",
"partitionKey": "ocpiCpoId"
}
],
"ContainersTTL": [
{
"name": "OcpiCdrs",
"partitionKey": "pk",
"ttl": 172800
},
{
"name": "OcppTransactionIds",
"partitionKey": "pk",
"ttl": 172800
},
{
"name": "Sessions",
"partitionKey": "pk",
"ttl": 172800
}
]
}
],
"system_engineers": [
],
}
}
If I want to updatethis with.
ex: parameters_test.json
{
"general": {
"system_engineers": [
{
"name": "hans",
"AppPrincipalId": "<id>",
"permissions": [
"get",
"list"
]
},
{
"name": "John do",
"AppPrincipalId": "<pid>",
"permissions": [
"all"
]
}
]
}
}
This works, the users are added to the empty "sytem_engineers" node in the parameters_general.json.
However, If I just want to change a setting on a lower node example:
ex: parameters_dev.json
{
"general": {
"databases": [
{
"name": "CtrlWps",
"Containers": [
{
"name": "ControllerAuthentication",
"partitionKey": "pk"
}
]
}
]
}
}
for replacing the partitionKey in one of the databases it replaces the whole "databases" part so I lose all other database configurations in the target.
The code I use is the following.
function ExtendJSON($base, $ext)
{
$propNames = $($ext | Get-Member -MemberType *Property).Name
foreach ($propName in $propNames) {
if ($base.PSObject.Properties.Match($propName).Count) {
if ($base.$propName.GetType().Name -eq "PSCustomObject")
{
$base.$propName = ExtendJSON $base.$propName $ext.$propName
}
else
{
$base.$propName = $ext.$propName
}
}
else
{
$base | Add-Member -MemberType NoteProperty -Name $propName -Value $ext.$propName
}
}
return $base
}
$tier = 'dev'
$parametersJsonGeneral = Get-Content -Path "./parameters/parameters_general.json" | ConvertFrom-Json
#Write-Output "#####################"
$parametersJsonTier = Get-Content -Path "./parameters/parameters_$($tier).json" | ConvertFrom-Json # overwrites existing values in $parametersJsonGeneral
ExtendJSON $parametersJsonGeneral $parametersJsonTier
Is there a way to loop over the settings from the lowest level up to the higher, and replace only these?
The proposed answer does only work one 1st level THIS WORKS
function merger ($target, $source) {
$source.psobject.Properties | ForEach-Object {
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject' -and $target."$($_.Name)" ) {
merger $target."$($_.Name)" $_.Value
}
else {
$target | Add-Member -MemberType $_.MemberType -Name $_.Name -Value $_.Value -Force
}
}
}
$Json1 ='{
"a": {
"b":"asda"
},
"c": "asdasd"
}
' | ConvertFrom-Json
$Json2 = '{
"a": {
"b":"d"
}
}
' | ConvertFrom-Json
merger $Json1 $Json2
However with this I loose data in $Json1
$Json1 ='{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": true,
"test2": false
}
}
]
},
"c": "asdasd"
}
' | ConvertFrom-Json
$Json2 = '{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": false
}
}
]
}
}
' | ConvertFrom-Json
merger $Json1 $Json2
{
"a": {
"b": [
{
"name": "admin",
"appconfig": {
"test1": false
}
}
]
},
"c": "asdasd"
}
test2 is gone!

I don't have enough rep to post general comments yet, so I'll have to post it as an answer:
I tested the code in this answer to another question, and it did what you asked. The advantage of this answer is that it doesn't depend on importing a separate module.
There's another answer in the same question that provides link to a more general module that provides for different types of merges (Left Join, Inner Join, etc.) here, which links to here. However, one commenter said that it didn't work with powershell 7.0.

Related

Converting JSON to NDJSON with Powershell

I'm trying to convert a text file that has an array of json objects to an NDJSON formatted file for another team to consume.
I've almost got it, except for one problem. I have an array of objects nested inside the JSON (which then has nested arrays and objects inside of it, the structure gets pretty complex, I'll include a sample below) and for whatever reason, when I use ConvertFrom-JSON it drops this nested array and in my output, I end up with a blank string for that key, instead of the nested array object. I tried using the -Depth flag but when I do that my output file ends up blank, which doesn't make a ton of sense to me. I don't have a whole lot of experience with powershell, so I'm not really sure where I'm going wrong here.
Code:
$JSONSourceFile = Get-Content -Path "input/sample.json" | ConvertFrom-JSON
$NDJSONTargetFile = "output/sample.json"
New-Item $NDJSONTargetFile -ItemType file
for ( $i = 0 ; $i -lt $JSONSourceFile.Length ; $i++) {
$item = $JSONSourceFile.item($i)
$row = ($item | ConvertTo-JSON -Compress)
Add-Content $NDJSONTargetFile $row
}
Input File:
[
{
"id": "1",
"TransactionDttm": "2021-02-22T15:45:00:00.000-05:00",
"Array1": [
{
"UserID": "ak2354",
"Preferences": [
{
"CagegoryID": "01",
"CategoryName": "Reminder",
"Channels": [
{
"ChannelID": "1",
"ChannelName": "Email",
"Preference": "Y"
},
{
"ChannelID": "2",
"ChannelName": "Text",
"Preference": "N"
}
]
}
]
}
]
},
{
"id": "2",
"TransactionDttm": "2021-02-22T15:45:00:00.000-05:00",
"Array1": [
{
"UserID": "ak1234",
"Preferences": [
{
"CagegoryID": "01",
"CategoryName": "Reminder",
"Channels": [
{
"ChannelID": "1",
"ChannelName": "Email",
"Preference": "Y"
},
{
"ChannelID": "2",
"ChannelName": "Text",
"Preference": "Y"
}
]
}
]
}
]
},
{
"id": "3",
"TransactionDttm": "2021-02-22T15:45:00:00.000-05:00",
"Array1": [
{
"UserID": "ak5678",
"Preferences": [
{
"CagegoryID": "01",
"CategoryName": "Reminder",
"Channels": [
{
"ChannelID": "1",
"ChannelName": "Email",
"Preference": "N"
},
{
"ChannelID": "2",
"ChannelName": "Text",
"Preference": "N"
}
]
}
]
}
]
}
]
And then when I convert it to the output, this is what I get:
{"id":"1","TransactionDttm":"2021-02-22T15:45:00:00.000-05:00","Array1":[{"UserID":"ak2354","Preferences":""}]}
{"id":"2","TransactionDttm":"2021-02-22T15:45:00:00.000-05:00","Array1":[{"UserID":"ak1234","Preferences":""}]}
{"id":"3","TransactionDttm":"2021-02-22T15:45:00:00.000-05:00","Array1":[{"UserID":"ak5678","Preferences":""}]}
Thanks to the comment from Doug Maurer I figured it out, I was adding -Depth to my ConvertFrom-Json command when it should have been in my ConvertTo-Json command. This is the final script and what it gives me:
$JSONSourceFile = Get-Content -Path "input/sample.json" | ConvertFrom-JSON
$NDJSONTargetFile = "output/sample.json"
New-Item $NDJSONTargetFile -ItemType file
for ( $i = 0 ; $i -lt $JSONSourceFile.Length ; $i++) {
$item = $JSONSourceFile.item($i)
$row = ($item | ConvertTo-JSON -Compress -Depth 20)
Add-Content $NDJSONTargetFile $row
}
and the output:
{"id":"1","TransactionDttm":"2021-02-22T15:45:00:00.000-05:00","Array1":[{"UserID":"ak2354","Preferences":[{"CagegoryID":"01","CategoryName":"Reminder","Channels":[{"ChannelID":"1","ChannelName":"Email","Preference":"Y"},{"ChannelID":"2","ChannelName":"Text","Preference":"N"}]}]}]}
{"id":"2","TransactionDttm":"2021-02-22T15:45:00:00.000-05:00","Array1":[{"UserID":"ak1234","Preferences":[{"CagegoryID":"01","CategoryName":"Reminder","Channels":[{"ChannelID":"1","ChannelName":"Email","Preference":"Y"},{"ChannelID":"2","ChannelName":"Text","Preference":"Y"}]}]}]}
{"id":"3","TransactionDttm":"2021-02-22T15:45:00:00.000-05:00","Array1":[{"UserID":"ak5678","Preferences":[{"CagegoryID":"01","CategoryName":"Reminder","Channels":[{"ChannelID":"1","ChannelName":"Email","Preference":"N"},{"ChannelID":"2","ChannelName":"Text","Preference":"N"}]}]}]}

Powershell - issue with matching an object within a list with another object within another list of object

yep, i dunno if i can make a longer title ... ;)
well, here is what i got :
some lists of objects (lets call them "objA[0..z]") in "listA[1..z]"
some lists of objects (lets call them "objB[0..z]") in "listB[1..z]"
Here is a Dataset (modified from answer below)
$lista1 = #'
[
{
"computerName": "123",
"userName": "Antoine",
"otherData1": "test1.xls",
"otherData2": "yyy"
},
{
"computerName": "123",
"userName": "Antoine",
"otherData1": "test2.xls",
"otherData2": "yyy"
},
{
"computerName": "456",
"userName": "Benoit",
"otherData1": "file1.txt",
"otherData2": "yyy"
}
]
'#
$lista2 = #'
[
{
"computerName": "789",
"userName": "Cyril",
"otherData1": "file1.pps",
"otherData2": "yyy"
},
{
"computerName": "789",
"userName": "Cyril",
"otherData1": "file2.pps",
"otherData2": "yyy"
},
{
"computerName": "321",
"userName": "Damien",
"otherData1": "doc.docx",
"otherData2": "yyy"
}
]
'#
$listb1 = #'
[
{
"computerName": "789",
"userName": "Cyril",
"otherData1": "file2.pps",
"otherData2": "yyy"
},
{
"computerName": "321",
"userName": "Damien",
"otherData1": "doc.docx",
"otherData2": "bbb"
}
]
'#
$listb2 = #'
[
{
"computerName": "123",
"userName": "Antoine",
"otherData1": "test1.xls",
"otherData2": "yyy"
},
{
"computerName": "123",
"userName": "Antoine",
"otherData1": "test2.xls",
"otherData2": "yyy"
},
]
'#
# expected output with this dataset :
# 1st and 2nd are not in any listb and 3rd.otherData2 do not match with each other
[
{
"computerName": "456",
"userName": "Benoit",
"otherData1": "file1.txt",
"otherData2": "yyy"
},
{
"computerName": "789",
"userName": "Cyril",
"otherData1": "file1.pps",
"otherData2": "yyy"
},
{
"computerName": "321",
"userName": "Damien",
"otherData1": "doc.docx",
"otherData2": "yyy"
}
]
what i'm trying to do :
If (obj'w' in lista'x') match computerName and userName and otherData1 in (obj'y' in listb'z')
AND IF same object NOT match otherData2 -> return said object
ELSE IF (obj'w' in lista'x') NOT match any object in every listb[0..z] -> return said object
here is what i did so far and where i'm stuck.
I more or less managed to make it work with "single file" on both listA & listB
Problem1 is i'm clearly drowing in foreach loop and i fail to find a "clean way".
Problem2 is i fail to properly check if objA is present in listB resulting in not being able to return it. (or returning all objects)
$listAPath = 'D:\A'
$listBPath = 'D:\B'
# multiples files in each directory
$listAFiles = Get-ChildItem $listAPath
$listBFiles = Get-ChildItem $listBPath
$failList = foreach ($listAFile in $listAFiles) {
$listObjA = Get-Content $listAFile.fullname -raw | ConvertFrom-Json
foreach ($objA in $listObjA) {
foreach ($listBFile in $listBFiles) {
$listObjB = Get-Content $listBFile.fullname -raw | ConvertFrom-Json
foreach ($objB in $listObjB) {
if ($objA.computerName -eq $objB.computerName -and $objA.userName -eq $objB.userName -and $objA.otherData1 -eq $objB.otherData1) {
if (<# do something with OtherData2 #>) {
$ObjA
}
} <# else {return $objA} #>
}
}
}
}
$failList
Reworked this script while typing it, hope i did not forget anything while doing it.
I'd be glad if i could get help here
If you need some more information, do not hesitate to ask.
Ty in advance.
EDITED : trying to apply what was asked in comment / answer, did not modify first "version" of script provided even though it might no match anymore with given dataset and reworked informations... will edit if asked.
For thoose who are curious, otherData2 is a datetime data with the "real" script.
If I understand you correctly, if an object in A matches the 3 properties of an object in list b then you will check OtherData2 and return the object depending on that check. If it's not in list b then just return the object. I've created some sample data for this demonstration.
$lista = #'
[
{
"computerName": 123,
"userName": "john",
"otherData1": "data128",
"otherData2": "yyy"
},
{
"computerName": 456,
"userName": "joan",
"otherData1": "data821",
"otherData2": "nnn"
},
{
"computerName": 789,
"userName": "doug",
"otherData1": "data111",
"otherData2": "abc"
}
]
'#
$listb = #'
[
{
"computerName": 123,
"userName": "john",
"otherData1": "data128",
"otherData2": "yyy"
},
{
"computerName": 789,
"userName": "doug",
"otherData1": "data111",
"otherData2": "abc"
}
]
'#
$listObjA = $lista | ConvertFrom-Json
$listObjB = $listb | ConvertFrom-Json
foreach($objA in $listObjA)
{
if($listObjB | where {$_.computername -eq $objA.computername -and
$_.username -eq $objA.username -and
$_.otherData1 -eq $objA.otherdata1})
{
if($objA.otherdata2 -eq 'yyy')
{
$objA
}
}
else
{
$objA
}
}
The output will be 2 objects from list a, one that was in list b AND otherdata2 met the criteria and the other was not in list b.
computerName userName otherData1 otherData2
------------ -------- ---------- ----------
123 john data128 yyy
456 joan data821 nnn
Edit
Ok, first issue you're going to face is combining of the different JSON files that contain arrays. See the following test.
$listb1 = New-TemporaryFile
$listb2 = New-TemporaryFile
#'
[
{
"computerName": "789",
"userName": "Cyril",
"otherData1": "file2.pps",
"otherData2": "yyy"
},
{
"computerName": "321",
"userName": "Damien",
"otherData1": "doc.docx",
"otherData2": "bbb"
}
]
'# | Set-Content $listb1 -Encoding UTF8
#'
[
{
"computerName": "123",
"userName": "Antoine",
"otherData1": "test1.xls",
"otherData2": "yyy"
},
{
"computerName": "123",
"userName": "Antoine",
"otherData1": "test2.xls",
"otherData2": "yyy"
}
]
'# | Set-Content $listb2 -Encoding UTF8
$listObjB = $listb1,$listb2 | foreach {Get-content $_ -Raw | ConvertFrom-Json}
$listObjB.Count
2
You end up with 2 arrays of 2 objects, surely not what you were expecting. However, by simply putting a subexpression around the get-content | convert command, powershell will unroll those into individual items. See this excellent answer by the other commenter Maximillian Burszley for more info.
$listObjB = $listb1,$listb2 | foreach {(Get-content $_ -Raw | ConvertFrom-Json)}
$listObjB.Count
4
That's what we were expecting. We will need to apply the same to the obja list
$lista1 = New-TemporaryFile
$lista2 = New-TemporaryFile
#'
[
{
"computerName": "123",
"userName": "Antoine",
"otherData1": "test1.xls",
"otherData2": "yyy"
},
{
"computerName": "123",
"userName": "Antoine",
"otherData1": "test2.xls",
"otherData2": "yyy"
},
{
"computerName": "456",
"userName": "Benoit",
"otherData1": "file1.txt",
"otherData2": "yyy"
}
]
'# | Set-Content $lista1 -Encoding UTF8
#'
[
{
"computerName": "789",
"userName": "Cyril",
"otherData1": "file1.pps",
"otherData2": "yyy"
},
{
"computerName": "789",
"userName": "Cyril",
"otherData1": "file2.pps",
"otherData2": "yyy"
},
{
"computerName": "321",
"userName": "Damien",
"otherData1": "doc.docx",
"otherData2": "yyy"
}
]
'# | Set-Content $lista2 -Encoding UTF8
$listobjA = $lista1,$lista2 | foreach {(Get-content $_ -Raw | ConvertFrom-Json)}
Then make a slight change to my suggested solution
foreach($objA in $listobjA)
{
if($matchedobj = $listObjB | where {$_.computername -eq $objA.computername -and
$_.username -eq $objA.username -and
$_.otherData1 -eq $objA.otherdata1})
{
if($objA.otherdata2 -ne $matchedobj.otherData2)
{
$objA
}
}
else
{
$objA
}
}
And we will end up with the expected results.
computerName userName otherData1 otherData2
------------ -------- ---------- ----------
456 Benoit file1.txt yyy
789 Cyril file1.pps yyy
321 Damien doc.docx yyy
You could also do the a conversion as part of the loop like this.
foreach($objA in $lista1,$lista2 | foreach {(Get-content $_ -Raw | ConvertFrom-Json)})
{
if($matchedobj = $listObjB | where {$_.computername -eq $objA.computername -and
$_.username -eq $objA.username -and
$_.otherData1 -eq $objA.otherdata1})
{
if($objA.otherdata2 -ne $matchedobj.otherData2)
{
$objA
}
}
else
{
$objA
}
}
But I personally feel the former was easier to read and understand.

Filter JSON in Powershell

I have the following JSON in a Powershell variable:
{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
I want to get JSON where the value of criteria is 1. The result therefore should look as follows:
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
I made an attempt with the following code:
$json | Get-ObjectMembers | Select-Object | where { $_.value.criteria -eq 1 };
While this basically goes into the right direction, it is not what I want exactly, because the result looks like this:
{
"name": "asdf2",
"criteria": 1
}
See that the Object2 information is lost and one depth-level is lost.
How can I achieve the desired result as shown above?
In essence, you're looking to only retain properties of interest from your single input object or, to put it differently, to remove properties you're not interested in.
Here's a PSv4+ solution:
$json = #'
{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
'#
($json | ConvertFrom-Json).psobject.Properties.
Where({ $_.Value.criteria -eq 1 }).
ForEach({ [pscustomobject] #{ $_.Name = $_.Value } }) |
ConvertTo-Json
The above yields:
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}
The jq solution. with_entries temporarily turns a list of properties into key and value pairs. https://stedolan.github.io/jq/manual/#to_entries,from_entries,with_entries (section id's in html come in handy) (Json creators don't believe in arrays?)
$json = '{
"Object1": {
"name": "asdf1",
"criteria": 2
},
"Object2": {
"name": "asdf2",
"criteria": 1
}
}'
$json | jq 'with_entries(select(.value.criteria == 1))'
{
"Object2": {
"name": "asdf2",
"criteria": 1
}
}

Create a json document based on existing document with powershell 5.1

I'm trying to build a json document which takes it's structure, column names and some values from an already existing base json document, and sets other values to variables assigned via powershell 5.1 in a foreach-object loop.
I tried editing a copy of the base json and creating a new one entirely and I can get close to what I want but not exactly.
A cut down example json document is:
[
{
"name": "Application1",
"tags": ["monitor"],
"description": "Get information about application 1.",
"query":
[
{
"target": {
"version": ["v1","v2","v3","v4","v5"],
"platform": ["Windows","Linux"]
},
"implementation": {
"action": "do something"
}
}
]
},
{
"name": "Application2",
"tags": ["monitor"],
"description": "Get information about application 2",
"query":
[
{
"target": {
"version": ["v1","v2","v3"],
"platform": ["Windows"]
},
"implementation": {
"action": "do something for v1, v2 and v3"
}
},
{
"target": {
"version": ["v4","v5"],
"platform": ["Windows"]
},
"implementation": {
"action": "do something for v4 and v5"
}
}
]
}
]
My script currently looks like:
$BaseJson = (Get-Content "$SourcePath\probes.json" | ConvertFrom-Json)
$ClientConfig = #"
{
"targetHost": "$DigitalSymphonyHost",
"targetPort": "$DigitalSymphonyPort",
"source": "$TargetSqlInstance",
"sensor": {}
}
"#
$ClientJson = ConvertFrom-Json -InputObject $ClientConfig
$BaseJson | Where-Object {$_.tags -contains "monitor"} | ForEach-Object {
# For each sensor in the base-sensors json document
$SensorName = $_.name
$Severity = 2
$Version = $_.query.target.version
$Platform = $_.query.target.platform
$Query = $_.query.implementation.action
$Sensor =#"
{
"severity": "$Severity",
"sensorId": "$SensorId",
"type": "$Type",
"target": {
"version": "$Version",
"platform": "$Platform",
"engineEdition": "$Edition"
},
"implementation": {
"query": "$Query"
}
}
"#
$ClientJson.sensor | Add-Member -Name $SensorName -Value (ConvertFrom-Json $Sensor) -MemberType NoteProperty
}
$ClientJson | ConvertTo-Json | Out-File "$DestinationPath\client-sensors.json"
My desired result is to add the $SensorId as per:
[
{
"targetHost": "servername",
"targetPort": "port",
"source": "servername",
"sensor": [{
"name": "Application1",
"sensorId": "<value_from_$SensorId>",
"tags": [],
"description": "Get information about application 1.",
"query":
[
{
"target": {
"version": ["v1","v2","v3","v4","v5"],
"platform": ["Windows","Linux"]
},
"implementation": {
"action": "do something"
}
}
]
},
{
"name": "Application2",
"sensorId": "<value_from_$SensorId>",
"tags": [],
"description": "Get information about application 2",
"query":
[
{
"target": {
"version": ["v1","v2","v3"],
"platform": ["Windows"]
},
"implementation": {
"action": "do something for v1, v2 and v3"
}
},
{
"target": {
"version": ["v4","v5"],
"platform": ["Windows"]
},
"implementation": {
"action": "do something for v4 and v5"
}
}
]
}]
}
]
My current result is
{
"targetHost": "servername",
"targetPort": "port",
"source": "server",
"sensor": {
"applicationname1": {
"tags": ["monitor],
"description": "Get information about application 2",
"sensorId": "420",
"target": "#{version=v1,v2,v3,v4,v5; platform=Windows Windows;}",
"action": "do something for v4 and v5"
}
}
}
I'm not sure where you're getting sensorID, but I'd do this more like the below:
param(
$SourcePath = $PSScriptRoot,
$DestinationPath = $PSScriptRoot,
$DigitalSymphonyHost = 'SomeHost',
$DigitalSymphonyPort = '8765',
$TargetSqlInstance
)
$BaseObj = (Get-Content "$SourcePath\probes.json" | ConvertFrom-Json)
$Client = [PSCustomObject] #{
targetHost = $DigitalSymphonyHost
targetPort = $DigitalSymphonyPort
source = $TargetSqlInstance
sensor = #()
}
$BaseObj | Where-Object {$_.tags -contains "monitor"} | ForEach-Object {
$sensor = $_
# TODO: How are you getting sensorId?
$sensor | Add-Member -Name 'sensorId' -Value 777 -MemberType NoteProperty
$Client.sensor += $sensor
}
ConvertTo-Json #($Client) -Depth 10 | Out-File "$DestinationPath\client-sensors.json" -Force
That will get you the JSON you're looking for.

Need to convert nested Json response to csv in powershell

I have below sample nested json response and I need to convert this response with specific values into CSV file. Below is the sample nested Json response:
{
"transaction": {
"id": "TestTransID",
"testCode": "NEW",
"TestStatus": "SUCCESS",
"client": {
"TestNumber": "112112111"
},
"subject": {
"individual": {
"additionalAttributes": {
"extraid": "787877878"
},
"addressList": [
{
"city": "New York",
"country": {
"name": "United States"
},
"postalCode": "123456789",
"stateOrProvince": {
"codeValue": "NY"
}
}
],
"gender": "F",
"identificationDocumentList": [
{
"number": "1214558520",
"type": "TestId"
}
],
"name": {
"firstName": "Qusay TestFull",
"lastName": "TestLast",
"middleName": "Middle 3"
}
}
},
"PROCESSConfiguration": {
"id": 1
},
"testProductList": [
{
"product": {
"id": 00,
"name": "Test PROCESS",
"productCode": "EFG",
"disclaimer": "TestDisclaimer"
},
"testSourceResponseList": [
{
"testSource": {
"id": 1,
"name": "TEST"
},
"testSourceRecordList": [
{
"type": "TestRecord",
"alertReasonCode": "TESTS",
"alertReasonDescription": "ACTION LIST HIT - TEST",
"testSource": "TEST",
"varListNameFull": "TEST FULL NAME",
"varListNameShort": "TEST SHORT",
"varProgList": [
"SHORT"
],
"varListId": "3421",
"subject": {
"individual": {
"TestScore": {
"TestScore": 100,
"triggeredRule": "TestRule"
},
"aNameList": [
{
"fullName": " TestNameA",
"lastName": "TestNameA"
},
{
"firstName": "TestFirst",
"fullName": "TestFirst HUSAYN",
"lastName": "TestLast"
},
{
"firstName": "TestFirst",
"fullName": "TestFull",
"lastName": "TestLast"
},
{
"firstName": "TestFirst",
"fullName": "TestFull",
"lastName": "TestLast"
}
],
"birthList": [
{
"dateOfBirth": "12 Apr 1910",
"dateOfBirthVerified": "true"
}
],
"name": {
"firstName": "TestFirst",
"fullName": "TestFull",
"lastName": "TestLast"
},
"varNationality": [
{
"verified": "true"
}
],
"remarks": "remark1"
}
}
},
{
"testSource": "TEST",
"varListNameFull": "TEST FULL",
"varListNameShort": "TEST SHORT",
"varProgList": [
"XYZ"
],
"varListId": "1234",
"subject": {
"individual": {
"overallScore": {
"TestScore": 100,
"triggeredRule": "Testing"
},
"birthList": [
{
"dateOfBirth": "1965",
},
{
"dateOfBirth": "1966",
}
],
"name": {
"firstName": "TestFirst",
"fullName": "TestFull",
"lastName": "TestLast",
},
"varNationality": [
{
"verified": "true"
}
],
"remarks": "REMARK2"
}
}
}
]
}
]
}
],
}
}
I need to take response from ""PROCESSConfiguration": {
"id": 1"
from row # 40. If u'll take above code in notepad ++.
Also I need the response with respect to var value like first name, last name full name, DOB etc.
I am still unsure what is being asked for. Let me assume that you want a fully qualified path for each of the elements in the JSON file.
I started by making some small adjustments to the JSON based on http://jsonlint.com/.
Based on that, I did a proof of concept that works for ONE OBJECT like the JSON you posted. It works for THIS CASE. I wrapped the logic to deal with multiple objects making assumptions about when one object started and the next began. Also, logic would should be added to deal with nested series/array containing multiple properties (i.e. Threads in Get-Process) to handle a generic case. A single element with an array of values (like .transaction.varProgList in this case) is handled by putting a ‘;’ between them.
CSV files assume that all the objects are symmetric (have the same properties). There is no check to see that the properties for each object align with the properties of the other objects. Note the handling of nested series is related to this. You may see an example of this by uncommenting the [System.Collections.ICollection] section and trying something like (Get-Process r*) | Select-Object Name,Threads | Expand-NestedProperty | Out-File .\t.csv -Width 100000
The repro goes as follows where $a is the adjusted JSON content and the function is saved as Expand-NestedProperty.ps1.
# Load the function
. .\Expand-NestedProperty.ps1
# Create PowerShell object(s) based on the JSON
$b = $a | ConvertFrom-Json
# Create a file with the CSV contents.
$b | Expand-NestedProperty | Out-File -FilePath .\my.csv -Width 100000
Save this as Expand-NestedProperty.ps1
function Expand-NestedProperty {
[CmdletBinding()]
param (
[Parameter( Position=0,
Mandatory=$true,
ValueFromPipeline=$true,
ValueFromPipelineByPropertyName=$true,
ValueFromRemainingArguments=$false,
HelpMessage='Object required...' )]
$InputObject
)
begin {
function ExpandNestedProperty {
[CmdletBinding()]
param (
[Parameter( Position=0,
Mandatory=$true,
ValueFromPipeline=$true,
ValueFromPipelineByPropertyName=$true,
ValueFromRemainingArguments=$false,
HelpMessage='Object required...' )]
$InputObject,
[Parameter( Position=1,
Mandatory=$false,
ValueFromPipeline=$false,
ValueFromPipelineByPropertyName=$true,
ValueFromRemainingArguments=$false,
HelpMessage='String required...' )]
[string]
$FullyQualifiedName = ""
)
begin {
$localResults =#()
$FQN = $FullyQualifiedName
$nestedProperties = $null
}
process {
foreach ($obj in $InputObject.psobject.Properties) {
if ($(try {$obj.Value[0] -is [PSCustomObject]} catch {$false})) { # Catch 'Cannot index into a null array' for null values
# Nested properties
$FQN = "$($FullyQualifiedName).$($obj.Name)"
$nestedProperties = $obj.value | ExpandNestedProperty -FullyQualifiedName $FQN
}
elseif ($obj.Value -is [array]) {
# Array property
$FQN = "$($FullyQualifiedName).$($obj.Name)"
[psobject]$nestedProperties = #{
$FQN = ($obj.Value -join ';')
}
}
# Example of how to deal with generic case.
# This needed for the Get-Process values ([System.Collections.ReadOnlyCollectionBase] and [System.Diagnostics.FileVersionInfo]) that are not [array] collection type.
<#
elseif ($obj.Value -is [System.Collections.ICollection]) {
# Nested properties
$FQN = "$($FullyQualifiedName).$($obj.Name)"
$nestedProperties = $obj.value | ExpandNestedProperty -FullyQualifiedName $FQN
}
#>
else { # ($obj -is [PSNoteProperty]) for this case, but could be any type
$FQN = "$($FullyQualifiedName).$($obj.Name)"
[psobject]$nestedProperties = #{
$FQN = $obj.Value
}
}
$localResults += $nestedProperties
} #foreach $obj
}
end {
[pscustomobject]$localResults
}
} # function ExpandNestedProperty
$objectNumber = 0
$firstObject = #()
$otherObjects = #()
}
process {
if ($objectNumber -eq 0) {
$objectNumber++
$firstObject = $InputObject[0] | ExpandNestedProperty
}
else {
if ($InputObject -is [array]) {
foreach ($nextInputObject in $InputObject[1..-1]) {
$objectNumber++
$otherObjects += ,($nextInputObject | ExpandNestedProperty)
}
}
else {
$objectNumber++
$otherObjects += ,($InputObject | ExpandNestedProperty)
}
}
}
end {
# Output CSV header and a line for each object which was the specific requirement here.
# Could create an array of objects using $firstObject + $otherObjects that is then piped to Export-CSV if we want a generic case.
Write-Output "`"$($firstObject.Keys -join '","')`""
Write-Output "`"$($firstObject.Values -join '","')`""
foreach ($otherObject in $otherObjects) {
Write-Output "`"$($otherObject.Values -join '","')`""
}
}
} # function Expand-NestedProperty