Replace a value in a JSON file using Powershell - json

How can I replace a value in a JSON file with another value taken from a config file (JSON)?
This is the code for the JSON files:
Config.json
{
"ABlob_AAD_Snapshot": [
{
"name": "properties.availability.frequency",
"value": "$Frequency$"
}]
}
ABlob_AAD_Snapshot.json
{
"name": "AzureBlobStore",
"properties": {
"type": "AzureStorage",
"availability": {
"frequency": "Value from Config.json file"
}
}
}
What I'm trying to do is the following:
Go through the Config.json file and store the values for "name" and "value" segments in variables. The "name" segment value is a path in ABlob_AAD_Snapshot.json file.
Follow the path in the ABlob_AAD_Snapshot.json file and replace the segment "frequency" with the value of segment "value" ($frequency$)
After this process, the ABlob_AAD_Snapshot.json should look like this:
{
"name": "AzureBlobStore",
"properties": {
"type": "AzureStorage",
"availability": {
"frequency": "$frequency$"
}
}
}
The problem here is that my original config.json file has more than one array (which represents file names) so I will parse more than one file and the value for "name" segment will not always be the same I mean, in this case, the value (or path) is properties.availability.frequency but it could be properties.activities.scheduler.interval or properties.activities.typeProperties.extendedProperties.WebAppClientID.
So as you can see the name and the quantity of "nodes" could change.
This is my PowerShell script:
$ScriptPath = split-path -parent $MyInvocation.MyCommand.Definition
#path to config file
$ConfigFile = "$ScriptPath\ParameterConfigOriginal.json"
#Convert the json file to PSObject
$json = Get-Content $ConfigFile | Out-String | ConvertFrom-Json
#Get all the arrays (files) in Conifg.json
$files = $json | Get-Member -MemberType Properties | Select-Object -ExpandProperty Name
#Go through all the arrays (files)
Foreach($file in $files)
{
if( $file -eq '$schema') {continue}
#store the path of the file to be modified
$FileName = $file + ".json"
$FilePath = "$ScriptPath\LinkedServices\" + $FileName"
#Go through all the elements of the arrray
Foreach($item in $json.$file)
{
#Store the path
$name = $item.name
#Store the value
$value = $item.value
#Convert the file to be modified to PSObject
$file = Get-Content $FilePath | Out-String | ConvertFrom-Json
#======STUCK IN HERE=============
# How can dynamically navigate through the file nodes like this?
$file.properties.availability.frequency
#and set the corresponding value
$file.properties.availability.frequency = $value
}
}
I'm new in the PowerShell world and I don't know if there is a cmdlet that helps me to do what I need.
Any suggestion will be appreciated.
EDIT
Simple Path
$snapshot.properties.availability.frequency
Path with arrays
$snapshot.properties.activities[0].scheduler.frequency
this is an example of the JSON file with arrays
Destination file
and this is the result
Destination file updated
Any idea on what could be happening?

Invoke-Expression will help you.
#Go through all the arrays (files)
Foreach($file in $files)
{
$snapshot = (Get-Content ("./" + $file + ".json") | ConvertFrom-Json)
# get config corresponds to the $file
$config = Invoke-Expression ('$json.' + $file)
# set value according to the config
Invoke-Expression ('$snapshot.' + $config.name + "='" + $config.value + "'")
# $snapshot.properties.availability.frequency
# -> $Frequency$
}
Edit
You have to use ConvertTo-Json -Depth 100 to write the result to JSON file properly (specify the appropriate depth according to your JSON files.).
Without -Depth option, you will get the result like "#{type=Copy, typeProperties=;, ...}".

Related

Variable substitution: How to update values in a json file and then save the output of the now substituted file

I have a script that I want to do variable substitution on a json file.
It takes json file A and json file B and converts it into a dynamic object.
So you end up with something like this:
For Json File A:
Json file A key
Json file A value
Age
ENTER VALUE
ADDRESS
ENTER VALUE
For Json File B:
Json file B key
Json file B value
Age
38
ADDRESS
10 example lane
If the keys match then the value from Json File B will be subbed into Json File A where the keys match.
So Json File A should end up looking like:
Json file A key
Json file A value
Age
38
ADDRESS
10 example lane
I am able to loop through and find matching keys and set the value of Json File A to the Value from Json File B for that Key.
However, I don't know how to get the new value into the actual json file A
Here are the relevant snippets of code:
$appSettings = Get-Content $JsonFileA -Raw | ConvertFrom-Json | Get-LeafProperty
if ($keyAA -eq $keyBB ) {
write-host "$keyAA and $keyBB match"
$valueAA=$valueBB
write-host "new appsettings value is now $valueAA"
# Save newly populated jsonfile to a new file
$appSettings | ConvertTo-Json -Depth 4 | Set-Content "path/to/output/file"
}
The write-host "new appsettings value is now $appvalue" line shows that it has done what i want and updated the value for Json File A to the value from Json File B for the matching key
But it is not saving the value to the output file. The output file is still just what the original Json File A looks like
FULL CODE for context:
function Get-LeafProperty {
param([Parameter(ValueFromPipeline)] [object] $InputObject, [string] $NamePath)
process {
if ($null -eq $InputObject -or $InputObject -is [DbNull] -or $InputObject.GetType().IsPrimitive -or $InputObject.GetType() -in [string], [datetime], [datetimeoffset], [decimal], [bigint]) {
[pscustomobject] #{ NamePath = $NamePath; Value = $InputObject }
}
elseif ($InputObject -is [System.Collections.IEnumerable] -and $InputObject -isnot [System.Collections.IDictionary]) {
$i = 0
foreach ($o in $InputObject) { Get-LeafProperty $o ($NamePath + '[' + $i++ + ']') }
}
else {
$props = if ($InputObject -is [System.Collections.IDictionary] -or $InputObject -is [System.Collections.IEnumerable]) { $InputObject.GetEnumerator() } else { $InputObject.psobject.properties }
$sep = '.' * ($NamePath -ne '')
foreach ($p in $props) {
Get-LeafProperty $p.Value ($NamePath + $sep + $p.Name)
}
}
}
}
$appSettings = Get-Content $jsonfileA -Raw | ConvertFrom-Json | Get-LeafProperty
$secrets = Get-Content $jsonfileB -Raw | ConvertFrom-Json | Get-LeafProperty
$leafProperties | ForEach-Object {
$keyAA = $_.NamePath
$valueAA = $_.Value
$secrets | ForEach-Object {
$keyBB = $_.NamePath
$keyBB = $_.Value
if ($keyAA -eq $keyBB ) {
write-host "$keyAA and $keyBB match"
$valueAA=$valueBB
write-host "new appsettings value is now $valueAA"
$appSettings | ConvertTo-Json -Depth 4 | Set-Content "path/to/output/file"
}
}
}

Update value (if it exists) in JSON from Powershell

I have an appSettings.json file in the following format
{
"ConnectionStrings": {
...
},
"appSettings": {
....
"LocalVersion": "1.0.0"
},
"Logging": {
....
},
"AllowedHosts":
....
}
}
I'm trying to write a Powershell script that will update this LocalVersion property. I need to script to 1. check if appSettings object exists, 2. check if LocalVersion property is there and 2. Update the LocalVersion to a new value and save the file.
My script seems to be mostly working, however the writing to the file does not work. The value is getting changed in the Powershell script but not being saved to file.
$appSettingsPath = "C:\git\Source\src\IMStack\appsettings.json"
Write-Host "Reading JSON from " $appSettingsPath
$jsonContent = (Get-Content $appSettingsPath -Raw) | ConvertFrom-Json
Write-Host $jsonContent
if ([bool]($jsonContent.PSobject.Properties.name -match "appSettings")){
$appSettings = $jsonContent.appSettings
Write-Host "Found appSettings" $appSettings
if ([bool]($appSettings.PSobject.Properties.name -match "LocalVersion")){
$version = $appSettings.LocalVersion
Write-Host $version
$jsonContent.appSettings.LocalVersion= "1.1.1"
Write-Host "Writing " $jsonContent " with " $jsonContent.appSettings " to " $appSettingsPath
$jsonContent | ConvertTo-Json -depth 100 | Set-Content "appSettingsPath"
}
}
How can I save this JSON to the file?

Loop json object using for loop or while loop in Powershell

As I am new to Powershell, can someone please support on the looping part?
Below is the json format from Test.json file:
{
"Pre-Production_AFM": {
"allowedapps": ["app1", "app2"]
},
"Production_AFM": {
"allowedapps": ["app1", "app2"]
}
}
I am reading the json file as below
$json = (Get-Content "Test.json" -Raw) | ConvertFrom-Json
I need to loop and get the 1st and 2nd objects - "Pre-Production_AFM" and "Production_AFM" one after another dynamically.
right now I have written the code as below :
foreach($i in $json){
if($i -contains "AFM"){
Write host "execute some code"
}
}
My dout is - Will $i holds the object "Pre-Production_AFM" dynamically?
If not please suggest the way to get the objects one after one dynamically for further execution.
# read the json text
$json = #"
{
"Pre-Production_AFM": {
"allowedapps": ["app1", "app2"]
},
"Production_AFM": {
"allowedapps": ["app1", "app2"]
}
}
"#
# convert to a PSCustomObject
$data = $json | ConvertFrom-Json
# just to prove it's a PSCustomObject...
$data.GetType().FullName
# System.Management.Automation.PSCustomObject
# now we can filter the properties by name like this:
$afmProperties = $data.psobject.Properties | where-object { $_.Name -like "*_AFM" };
# and loop through all the "*_AFM" properties
foreach( $afmProperty in $afmProperties )
{
$allowedApps = $afmProperty.Value.allowedApps
# do stuff
}

Convert and format XML to JSON on Azure Storage Account in Powershell

So i'm trying to convert XML files on an Azure Storage Container to JSON in the same container.
This way I'm able to read the information into an Azure SQL Database via an Azure Datafactory.
I'd like to stay clear from using Logic apps if able.
The JSON files need to be formatted.
And all this through the use of PowerShell scripting.
What i've got so far after some searching on the interwebs and shamelessly copying and pasting powershell code:
#Connect-AzAccount
# Helper function that converts a *simple* XML document to a nested hashtable
# with ordered keys.
function ConvertFrom-Xml {
param([parameter(Mandatory, ValueFromPipeline)] [System.Xml.XmlNode] $node)
process {
if ($node.DocumentElement) { $node = $node.DocumentElement }
$oht = [ordered] #{}
$name = $node.Name
if ($node.FirstChild -is [system.xml.xmltext]) {
$oht.$name = $node.FirstChild.InnerText
} else {
$oht.$name = New-Object System.Collections.ArrayList
foreach ($child in $node.ChildNodes) {
$null = $oht.$name.Add((ConvertFrom-Xml $child))
}
}
$oht
}
}
function Format-Json
{
<#
.SYNOPSIS
Prettifies JSON output.
.DESCRIPTION
Reformats a JSON string so the output looks better than what ConvertTo-Json outputs.
.PARAMETER Json
Required: [string] The JSON text to prettify.
.PARAMETER Minify
Optional: Returns the json string compressed.
.PARAMETER Indentation
Optional: The number of spaces (1..1024) to use for indentation. Defaults to 4.
.PARAMETER AsArray
Optional: If set, the output will be in the form of a string array, otherwise a single string is output.
.EXAMPLE
$json | ConvertTo-Json | Format-Json -Indentation 2
#>
[CmdletBinding(DefaultParameterSetName = 'Prettify')]
Param(
[Parameter(Mandatory = $true, Position = 0, ValueFromPipeline = $true)]
[string]$Json,
[Parameter(ParameterSetName = 'Minify')]
[switch]$Minify,
[Parameter(ParameterSetName = 'Prettify')]
[ValidateRange(1, 1024)]
[int]$Indentation = 4,
[Parameter(ParameterSetName = 'Prettify')]
[switch]$AsArray
)
if ($PSCmdlet.ParameterSetName -eq 'Minify')
{
return ($Json | ConvertFrom-Json) | ConvertTo-Json -Depth 100 -Compress
}
# If the input JSON text has been created with ConvertTo-Json -Compress
# then we first need to reconvert it without compression
if ($Json -notmatch '\r?\n')
{
$Json = ($Json | ConvertFrom-Json) | ConvertTo-Json -Depth 100
}
$indent = 0
$regexUnlessQuoted = '(?=([^"]*"[^"]*")*[^"]*$)'
$result = $Json -split '\r?\n' |
ForEach-Object {
# If the line contains a ] or } character,
# we need to decrement the indentation level unless it is inside quotes.
if ($_ -match "[}\]]$regexUnlessQuoted")
{
$indent = [Math]::Max($indent - $Indentation, 0)
}
# Replace all colon-space combinations by ": " unless it is inside quotes.
$line = (' ' * $indent) + ($_.TrimStart() -replace ":\s+$regexUnlessQuoted", ': ')
# If the line contains a [ or { character,
# we need to increment the indentation level unless it is inside quotes.
if ($_ -match "[\{\[]$regexUnlessQuoted")
{
$indent += $Indentation
}
$line
}
if ($AsArray) { return $result }
return $result -Join [Environment]::NewLine
}
# Storage account details
$resourceGroup = "insert resource group here"
$storageAccountName = "insert storage account name here"
$container = "insert container here"
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $resourceGroup -Name $storageAccountName).Value[0]
$storageAccount = Get-AzStorageAccount -ResourceGroupName $resourceGroup -Name $storageAccountName
# Creating Storage context for Source, destination and log storage accounts
#$context = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
$context = New-AzStorageContext -ConnectionString "insert connection string here"
$blob_list = Get-AzStorageBlob -Container $container -Context $context
foreach($blob_iterator in $blob_list){
[XML](Get-AzStorageBlobContent $blob_iterator.name -Container $container -Context $context) | ConvertFrom-Xml | ConvertTo-Json -Depth 11 | Format-Json | Set-Content ($blob_iterator.name + '.json')
}
Output =
Cannot convert value "Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageBlob" to type "System.Xml.XmlDocument". Error: "The specified node cannot
be inserted as the valid child of this node, because the specified node is the wrong type."
At C:\Users\.....\Convert XML to JSON.ps1:116 char:6
+ [XML](Get-AzStorageBlobContent $blob_iterator.name -Container $c ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvalidCastToXmlDocument
When I run the code the script asks me if I want to download the xml file to a local folder on my laptop.
This is not what I want, I want the conversion to be done in Azure on the Storage container.
And I think that I'm adding ".json" to the .xml file name.
So the output would become something like filename.xml.json instead of just filename.json
What's going wrong here?
And how can it be fixed?
Thank you in advance for your help.

Find and Replace Nested JSON Values with Powershell

I have an appsettings.json file that I would like to transform with a PowerShell script in a VSTS release pipeline PowerShell task. (BTW I'm deploying a netstandard 2 Api to IIS). The JSON is structured like the following:
{
"Foo": {
"BaseUrl": "http://foo.url.com",
"UrlKey": "12345"
},
"Bar": {
"BaseUrl": "http://bar.url.com"
},
"Blee": {
"BaseUrl": "http://blee.url.com"
}
}
I want to replace BaseUrl and, if it exists, the UrlKey values in each section which are Foo, Bar and Blee. (Foo:BaseUrl, Foo:UrlKey, Bar:BaseUrl, etc.)
I'm using the following JSON structure to hold the new values:
{
"##{FooUrl}":"$(FooUrl)",
"##{FooUrlKey}":"$(FooUrlKey)",
"##{BarUrl}":"$(BarUrl)",
"##{BleeUrl}":"$(BleeUrl)"
}
So far I have the following script:
# Get file path
$filePath = "C:\mywebsite\appsettings.json"
# Parse JSON object from string
$jsonString = "$(MyReplacementVariablesJson)"
$jsonObject = ConvertFrom-Json $jsonString
# Convert JSON replacement variables object to HashTable
$hashTable = #{}
foreach ($property in $jsonObject.PSObject.Properties) {
$hashTable[$property.Name] = $property.Value
}
# Here's where I need some help
# Perform variable replacements
foreach ($key in $hashTable.Keys) {
$sourceFile = Get-Content $filePath
$sourceFile -replace $key, $hashTable[$key] | Set-Content $filePath
Write-Host 'Replaced key' $key 'with value' $hashTable[$key] 'in' $filePath
}
Why are you defining your replacement values as a JSON string? That's just going to make your life more miserable. If you're defining the values in your script anyway just define them as hashtables right away:
$newUrls = #{
'Foo' = 'http://newfoo.example.com'
'Bar' = 'http://newbaz.example.com'
'Blee' = 'http://newblee.example.com'
}
$newKeys = #{
'Foo' = '67890'
}
Even if you wanted to read them from a file you could make that file a PowerShell script containing those hashtables and dot-source it. Or at least define the values as lists of key=value lines in text files, which can easily be turned into hashtables:
$newUrls = Get-Content 'new_urls.txt' | Out-String | ConvertFrom-StringData
$newKeys = Get-Content 'new_keys.txt' | Out-String | ConvertFrom-StringData
Then iterate over the top-level properties of your input JSON data and replace the nested properties with the new values:
$json = Get-Content $filePath | Out-String | ConvertFrom-Json
foreach ($name in $json.PSObject.Properties) {
$json.$name.BaseUrl = $newUrls[$name]
if ($newKeys.ContainsKey($name)) {
$json.$name.UrlKey = $newKeys[$name]
}
}
$json | ConvertTo-Json | Set-Content $filePath
Note that if your actual JSON data has more than 2 levels of hierarchy you'll need to tell ConvertTo-Json via the parameter -Depth how many levels it's supposed to convert.
Side note: piping the Get-Content output through Out-String is required because ConvertFrom-Json expects JSON input as a single string, and using Out-String makes the code work with all PowerShell versions. If you have PowerShell v3 or newer you can simplify the code a little by replacing Get-Content | Out-String with Get-Content -Raw.
Thank you, Ansgar for your detailed answer, which helped me a great deal. Ultimately, after having no luck iterating over the top-level properties of my input JSON data, I settled on the following code:
$json = (Get-Content -Path $filePath) | ConvertFrom-Json
$json.Foo.BaseUrl = $newUrls["Foo"]
$json.Bar.BaseUrl = $newUrls["Bar"]
$json.Blee.BaseUrl = $newUrls["Blee"]
$json.Foo.Key = $newKeys["Foo"]
$json | ConvertTo-Json | Set-Content $filePath
I hope this can help someone else.
To update values of keys at varying depth in the json/config file, you can pass in the key name using "." between the levels, e.g. AppSettings.Setting.Third to represent:
{
AppSettings = {
Setting = {
Third = "value I want to update"
}
}
}
To set the value for multiple settings, you can do something like this:
$file = "c:\temp\appSettings.json"
# define keys and values in hash table
$settings = #{
"AppSettings.SettingOne" = "1st value"
"AppSettings.SettingTwo" = "2nd value"
"AppSettings.SettingThree" = "3rd value"
"AppSettings.SettingThree.A" = "A under 3rd"
"AppSettings.SettingThree.B" = "B under 3rd"
"AppSettings.SettingThree.B.X" = "Z under B under 3rd"
"AppSettings.SettingThree.B.Y" = "Y under B under 3rd"
}
# read config file
$data = Get-Content $file -Raw | ConvertFrom-Json
# loop through settings
$settings.GetEnumerator() | ForEach-Object {
$key = $_.Key
$value = $_.Value
$command = "`$data.$key = $value"
Write-Verbose $command
# update value of object property
Invoke-Expression -Command $command
}
$data | ConvertTo-Json -Depth 10 | Out-File $file -Encoding "UTF8"