Merge two json objects - json

I have the following inputs - 2 json files one is the base one and the second contains the same properties but the different values, I'd like to merge that objects.
For example:
{
a:{
b:"asda"
}
c: "asdasd"
}
And the second file:
{
a:{
b:"d"
}
}
And the result should be like this:
{a:{b:"d"},c:"asdasd"}
Is that is possible with powershell?

Join (Join-Object) is not a built-in CmdLet
This is an extension of #Mark's answer which also recurses through child objects.
function merge ($target, $source) {
$source.psobject.Properties | % {
if ($_.TypeNameOfValue -eq 'System.Management.Automation.PSCustomObject' -and $target."$($_.Name)" ) {
merge $target."$($_.Name)" $_.Value
}
else {
$target | Add-Member -MemberType $_.MemberType -Name $_.Name -Value $_.Value -Force
}
}
}
merge $Json1 $Json2
$Json1

$Json1 | Join $Json2 -Merge {$Right.$_} | ConvertTo-Json (see update below)
Install-Module -Name JoinModule
($Json1 ConvertFrom-Json) | Merge ($Json2 ConvertFrom-Json) | ConvertTo-Json
Result:
{
"c": "asdasd",
"a": {
"b": "d"
}
}
You might consider not to overwrite the left value:
($Json1 ConvertFrom-Json) | Join ($Json2 ConvertFrom-Json) | ConvertTo-Json
In that case the result will be:
{
"c": "asdasd",
"a": [
{
"b": "asda"
},
{
"b": "d"
}
]
}
For details see: https://stackoverflow.com/a/45483110/1701026
Update 2019-11-16
The -Merge parameter has been depleted and divided over the -Discern and -Property parameters (sorry for the breaking change). The good news is that the default parameter settings for an object merge are accommodated in a proxy command named Merge-Object (alias Merge) which simplifies the concerned syntax to just: $Object1 | Merge $Object2. For details, see readme or embedded help.

If you know the names of the elements (per your example above), you could do it explicitly like this:
$Json1 ='{
a: {
b:"asda"
},
c: "asdasd"
}
' | ConvertFrom-Json
$Json2 = '{
a:{
b:"d"
}
}
' | ConvertFrom-Json
$Json1.a = $Json2.a
Result:
$Json1 | ConvertTo-Json
{
"a": {
"b": "d"
},
"c": "asdasd"
}
If you're looking for something that will merge the two without knowing the explicit key name, you could do something like the following. This will essentially overwrite any properties in the first Json with those from the second Json, where they are duplicated at the first level (it won't seek matches in the nested properties and again this is an overwrite not a merge):
$Json2.psobject.Properties | ForEach-Object {
$Json1 | Add-Member -MemberType $_.MemberType -Name $_.Name -Value $_.Value -Force
}

Related

Populate collection of objects from one JSON file to the collection of another one with PowerShell

I have two JSON files and want to transfer collection of objects from one file to another. Suppose, the from.json file contains property which represents collection of clients:
"Clients":
[
{
"Name": "Name1",
"Age": "12"
},
{
"Name": "Name2",
"Age": "14"
}
]
to.json file contains an empty collection, "Objects: []" ,which must be filled with objects from from.json. Each objects in toJson variable must contain additional property - Id, so eventually, my "to.json" file should look like this:
"Objects":
[
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
},
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
}
]
I've converted two files into variables:
$fromJson = (Get-Content -Raw -Path {fromPath}) | ConvertFrom-Json
$toJson = (Get-Content -Raw -Path {toPath}) | ConvertFrom-Json
I know that objects from fromJson to toJson can be transferred in the following manner:
toJson.Objects += fromJson.Clients, but that's not enough in my case. I think that it could be done by iterating through fromJson.Clients array but have no idea how to create an object and add it into toJson.Objects collection.
Here's a more efficient solution, based on:
Use of a calculated property with Select-Object, which allows you to place the new property first in the output objects.
Instead of building the array one by one with += (which is inefficient, because a new array must technically be created behind the scenes in every iteration), the solution below lets PowerShell collect the output objects of the Select-Object call in an array automatically (the [array] type constraint is needed to ensure that an array is created even if only one object happens to be output.)
# Sample input.
$fromJson = ConvertFrom-Json '{"Clients":[{"Name":"Name1","Age":"12"},{"Name":"Name2","Age":"14"}]}'
$toJson = ConvertFrom-Json '{ "Objects": [] }'
[array] $toJson.Objects =
$fromJson.Clients |
Select-Object #{ Name='Id'; Expression = { [string] (New-Guid) } }, *
$toJson | ConvertTo-Json -Depth 3 # append | Set-Content as needed.
Kind of new to the PowerShell, but after a bit of investigation came up with the following solution:
fromJson.Clients | ForEach-Object {
$_ | Add-Member -MemberType NoteProperty -Name 'Id' -Value ([guid]::NewGuid().Guid.ToString())
$toJson += $_
}
...
$toJson | ConvertTo-Json | Out-File {to.json_path}
Frankly, don't know if that is a 'proper' way to do that, but generally it works for that particular case. For now, see no other solution.

Extracting fields to CSV from a JSON file with Powershell

i have a json extracted from a API exact like this:
{
"LPEONASVVAP0": {
"LPEONASVVAP0": {
"id": "urn:vcloud:vm:f526d27d-e0f9-4d4f-ae81-4824e397c027",
"name": "LPEONASVVAP0",
"description": "_vm_desc_",
"dateCreated": "2021-04-06T14:56:09.640+0000"
}
},
"WDEONDSVDIS6": {
"WDEONDSVDIS6": {
"id": "urn:vcloud:vm:7ed43492-a7ce-4963-b5bb-5ec2ca89477c",
"name": "WDEONDSVDIS6",
"description": "",
"dateCreated": "2021-04-13T13:44:29.973+0000"
}
},
"WDEONASVSTR0": {
"WDEONASVSTR0": {
"id": "urn:vcloud:vm:7afa34fe-b239-4abe-90df-3f270b44db1f",
"name": "WDEONASVSTR0",
"description": "",
"dateCreated": "2021-03-10T16:17:50.947+0000"
}
},
}
I need extract only fields id, name and description to create a csv with them. I test this but the output file is in blank:
$pathToJsonFile = x
$pathToOutputFile = x
$obj = Get-Content $pathToJsonFile -Raw | ConvertFrom-Json
print $obj
$obj | select id, name, description | Convertto-csv > $pathToOutputFile
You'll need to "discover" the parent property names (eg. 'LPEONASVVAP0') via the psobject hidden memberset. Since the outer and inner properties are named the same, we can re-use the name to get the inner property value:
$obj.psobject.Properties |ForEach-Object {
$_.Value.$($_.Name)
} |Select id,name,description |Export-Csv -NoTypeInformation -Path $pathToOutputFile
Edit: Mathias R. Jessens answer is better written than this, i would do it that way instead of how i posted.
Okay so i copied the json you posted, imported it. Since each array of information is stored like this
"WDEONDSVDIS6": {
"WDEONDSVDIS6": {
i used get-member to iterate each of the arrays and then select the info from that.
Also, you dont need to use Convertto-csv > $pathToOutputFile, use the export-csv command instead.
Below is my code how i would have done it, there is probably a better way but this works :)
$pathToOutputFile = x.csv
$obj = Get-Content example.json -Raw| ConvertFrom-Json
$obj2 = ($obj | Get-Member -MemberType noteproperty).Name
$result = foreach($item in $obj2){
$obj.$item.$item | select id,name,description
}
$result | Export-Csv -Path $pathToOutputFile -Encoding utf8 -NoTypeInformation

How to Get the values from json using powershell

Guys this is my JSON file and I want to create a PowerShell script which will give me result like
I have used method like Get-Content and other but there are some issues with the JSON parsing. Please find what is my requirement I have explained in details below.
MyLocalMachineHome
LocalMachine = Sahil_LocalMachine
Second_MyLocalMachine = Sahil_MylocalMachine
Second_MyLocalMachine = ""
Staging
Second_Staging = Sahil;_Secconf
Staging = Sahil_Staging
third_staging = stsajiii
There is also one functionality which I would like to have if I want to get only variables of "staging".
I was using this function Get-Content -Raw -Path E:\shell\Powershell\1ReleasePipelines.json | ConvertFrom-Json | select -ExpandProperty variables on my original JSON file but somehow there is some kind of limit in storing string which I was getting from this method.
{
"environments": [
{
"id": 3,
"name": "MyLocalMachineHome",
"variableGroups": [],
"variables": {
"LocalMachine": {
"value": "Sahil_LocalMachine"
},
"Second_MyLocalMachine": {
"value": "Sahil_MylocalMachine"
},
"thirf_mylocal": {
"value": ""
}
}
},
{
"id": 7,
"name": "Staging",
"variableGroups": [],
"variables": {
"Second_Staging": {
"value": "Sahil;_Secconf"
},
"Staging": {
"value": "Sahil_Staging"
},
"third_staging": {
"value": "stsajiii"
}
}
}
]
}
If we assume that $json contains your JSON content, you can do the following ugly code:
$environment = 'staging'
$j = $json | ConvertFrom-Json
($j.environments | where name -eq $environment).variables | Foreach-Object {
$CurrentObject = $_
$CurrentObject | Get-Member -MemberType NoteProperty |
Select-Object -Expand Name | Foreach-Object {
$CurrentObject.$_.value
}
}
It appears your issue is that you don't know what variables are going to be contained within your JSON. So you can't easily use Select-Object variable or $object.variable. You need a dynamic approach.
If you know your variables ahead of time, things become simpler. You can store your variable names in an array and loop over them.
$variables = 'Second_Staging','Staging','third_staging'
$environment = 'staging'
$j = $json | ConvertFrom-Json
$jsonVars = ($j.environments | where name -eq $environment).variables
$variables | Foreach-Object {
$jsonVars.$_.value
}
View all the sub-properties of variables with format-list instead of format-table. Since the properties vary, format-table won't show all of them. There's a lot of sloppy object construction in json.
$a = get-content file.json
$a.environments.variables | format-table
LocalMachine Second_MyLocalMachine thirf_mylocal
------------ --------------------- -------------
#{value=Sahil_LocalMachine} #{value=Sahil_MylocalMachine} #{value=}
$a.environments.variables | format-list
LocalMachine : #{value=Sahil_LocalMachine}
Second_MyLocalMachine : #{value=Sahil_MylocalMachine}
thirf_mylocal : #{value=}
Second_Staging : #{value=Sahil;_Secconf}
Staging : #{value=Sahil_Staging}
third_staging : #{value=stsajiii}
Get the staging variables?
$a.environments | where name -eq staging | foreach variables
Second_Staging Staging third_staging
-------------- ------- -------------
#{value=Sahil;_Secconf} #{value=Sahil_Staging} #{value=stsajiii}
cls
start-transcript -path 'C:\E\Devops\PowerShell_Chapters\ABC.txt'
write-output "**********Variables of Release************"
get-content -raw -path 'C:\E\Devops\PowerShell_Chapters\Release.json'| Convertfrom-Json | Select -ExpandProperty variables
$json = get-content -raw -path 'C:\E\Devops\PowerShell_Chapters\Release.json'| Convertfrom-Json | Select -ExpandProperty environments
$EnvirnomentsVariables = get-content -raw -path 'C:\E\Devops\PowerShell_Chapters\Release.json'| Convertfrom-Json | Select -ExpandProperty environments |Select -ExpandProperty name
$ReleaseVariable = get-content -raw -path 'C:\E\Devops\PowerShell_Chapters\Release.json'| Convertfrom-Json | Select -ExpandProperty environments |Select -ExpandProperty variables
$i = 0
foreach($a in $EnvirnomentsVariables)
{
$ABC_Staging = $EnvirnomentsVariables[$i]
#write-output $ABC_Staging
if( $ABC_Staging -match "ABC Staging")
{
write-output "****************Variables of " $EnvirnomentsVariables[$i]*************"
#add-content 'C:\E\Devops\PowerShell_Chapters\ABC.txt' $EnvirnomentsVariables[$i]
# Set-content -path 'C:\E\Devops\PowerShell_Chapters\Sahil.json'| ConvertTo-Json | select $EnvirnomentsVariables[$i]
write-output $ReleaseVariable[$i]
# add-content 'C:\E\Devops\PowerShell_Chapters\ABC.txt' $ReleaseVariable[$i]
# Set-content -path 'C:\E\Devops\PowerShell_Chapters\Sahil.json'| ConvertTo-Json | select $ReleaseVariable[$i]
}
$i = $i + 1
}
stop-transcript

Powershell JSON pipeline expand multiple values into one column csv

I'm trying to automate some data pipelines with Powershell, but I'm kinda stuck with converting a JSON list to a single cell per row in a CSV file. Hope some of you can help me out.
The JSON I get looks like the following:
{"result": [
{
"uid": "1",
"EducationHistory": []
},
{
"uid": "2",
"EducationHistory": []
},
{
"uid": "3",
"EducationHistory": []
},
{
"uid": "4",
"EducationHistory": {
"10466632": {
"euid": 10466632,
"degree": "Highschool",
"educationLevel": null
},
"10466634": {
"euid": 10466634,
"degree": "Law",
"educationLevel": "batchelor"
},
"10466635": {
"euid": 10466635,
"degree": "Law",
"educationLevel": "master"
}
}
},
{
"uid": "5",
"EducationHistory": {
"10482462": {
"euid": 10482462,
"degree": "IT",
"educationLevel": "master"
}
}
}
]
}
What I want to do is collect the educationLevels per uid in one column. So something like this:
uid | educationLevel
----+------------------
1 |
2 |
3 |
4 | barchelor, master
5 | master
Normally I would like Expandproperty to get down to a lower level, but this doesn't work for this case because every EducationHistory entry is behind a euid for that specific entry. Expanding every single one of them like in the example below isn't workable because of the number of records.
So I think I need something of a loop, but I don't know how. Hope you can help me. First post here and a Powershell newbie, so I hope my question is clear. Please let me know if you need more info.
The code for one entry, as example:
$json = Get-content -raw -path C:\TEMP\File.json
(ConvertFrom-Json -InputObject $json).result |
Select-Object uid,
#Expand one of the entries:
#{Name = "Edu.Level";E={$_.EducationHistory | Select-Object -
expandproperty 10466632 |Select-Object -expandpropert degree }} |
Format-Table
$content = Get-Content .\test.json
$result = ($content | ConvertFrom-Json).result
$totalResult = #()
foreach($res in $result) {
$tempArray = #()
if($res.EducationHistory -ne $null) {
$properties = $res.EducationHistory | Get-Member -MemberType NoteProperty
foreach($property in $properties) {
$eduLevel = $res.EducationHistory.$($property.Name).educationLevel
if(![String]::IsNullOrEmpty($eduLevel)) {
$tempArray += $eduLevel
}
}
}
$totalResult += [PSCustomObject]#{
uid = $res.uid
educationLevel = $tempArray -join ", "
}
}
$totalResult
This will output desired result for the input you have provided.
The trickiest part is the value of EducationHistory property. You have to use Get-Member cmdlet (see Get-Help Get-Member) to get the properties of the current object in loop. Then using the name of the property to access the educationLevel.
Your first question, my first answer I believe :) Similar to the last answer. You need to jump through the hoop of finding the object names in EducationalHistory to reference them.
$json = (Get-content C:\TEMP\File.json | ConvertFrom-Json).result
$results = #()
foreach ( $u in $json)
{
foreach ( $h in $u.EducationHistory)
{
$results += $h.PSObject.properties.Name | ForEach-Object{new-object PSObject -property #{ uid=$u.uid; degree=$h.$_.degree}}
}
}
$results | ConvertTo-Csv | select -skip 1

Prettify json in powershell 3

Given a standard json string value:
$jsonString = '{ "baz": "quuz", "cow": [ "moo", "cud" ], "foo": "bar" }'
How can I get this to be all pretty with newlines, preferably without brute-force regex?
Simplest method I've found so far is:
$jsonString | ConvertFrom-Json | ConvertTo-Json
However, that seems kinda silly.
Works for me. Parentheses make sure get-content is done before piping. Default depth of convertto-json is 2, which is often too low.
function pjson ($jsonfile) {
(get-content $jsonfile) | convertfrom-json | convertto-json -depth 100 |
set-content $jsonfile
}
If you really don't want to go down the simplest route, which is to use inbuilt PowerShell functions | ConvertFrom-Json | ConvertTo-Json, here is another method, using JSON.net
# http://james.newtonking.com/projects/json-net.aspx
Add-Type -Path "DRIVE:\path\to\Newtonsoft.Json.dll"
$jsonString = '{ "baz": "quuz", "cow": [ "moo", "cud" ], "foo": "bar" }'
[Newtonsoft.Json.Linq.JObject]::Parse($jsonString).ToString()
I put this in my profile
function PrettyPrintJson {
param(
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
$json
)
$json | ConvertFrom-Json | ConvertTo-Json -Depth 100
}
Which works with pipes, and can be auto-completed, so it's at least somewhat less typing:
cat .\file.json | PrettyPrintJson
curl https://api.twitter.com/1.1/statuses/user_timeline.json | PrettyPrintJson
Adding to #JS2010's answer I added logic to escape out certain characters and clean up my output even further. The parenthesis seems key and -depth is a big one since you can lose details without it, from what I've seen, on depth that goes beyond the default of 5, I believe it is.
function Format-Json ($JSON)
{
$PrettifiedJSON = ($JSON) | convertfrom-json | convertto-json -depth 100 | ForEach-Object { [System.Text.RegularExpressions.Regex]::Unescape($_) }
$PrettifiedJSON
}
I think what you are looking for is this:
$jsonString = #{
'baz' = 'quuz'
'cow'= "moo, cud"
'foo'= "bar"
}
$jsonString|ConvertTo-Json
it produces this output
{
"baz": "quuz",
"cow": "moo, cud",
"foo": "bar"
}
Added note
You could also array your cow values to "prettify" it a bit more:
$jsonString = #{
'baz' = 'quuz'
'cow'= #("moo"; "cud")
'foo'= "bar"
}
output:
{
"baz": "quuz",
"cow": [
"moo",
"cud"
],
"foo": "bar"
}