I am taking the parameters from TemplateJSON file and reading data from CSV file and the converting it to JSON again.
TemplateJSON:
{
"adhocUARs":[
""
],
"rolefullpath": ""
}
CSV File:
RoleFullPath,ResourceName
FolderName\ABCD,ABCD User Account Resource
I am getting the output in this format (without the square brackets):
NEWJSON:
{
"adhocUARs":"ABCD User Account Resource",
"rolefullpath": "xyz"
}
But I expect the output to be in the following format(with the square brackets):
NEWJSON:
{
"adhocUARs":["ABCD User Account Resource"]
"rolefullpath": "xyz"
}
Code used:
$TemplateJSON = Convertfrom-json ([IO.File]::ReadAllText("TemplateJSON.json"))
$RoleFile = Import-CSV "CSVFile.csv" -Delimiter ','
[int]$Row = 0
Foreach ($Line in $RoleFile)
{
$Row = $Row + 1
$NewJSON = $TemplateJSON
$NewJSON.adhocUARs = $Line.ResourceName
$NewJSON.roleFullPath= $Line.RoleFullPath
$RolePath = "D:\\DummyFolder\"
$JSONPath = $RolePath + "patch.json"
Convertto-JSON $NewJSON | Out-file -Encoding "UTF8" $JSONPath
}
What am I missing?gr
so in JSON the [ ] is an array.
Currently you have $NewJSON.adhocUARs as a string single value.
A simple solution would be :
$NewJSON.adhocUARs = #($Line.ResourceName)
Related
This is what I am doing:
$appParametersXml = [Xml] (Get-Content "$appParameterFilePath\$appParameterFile")
$parameterJsonFile = "$appParameterFilePath\$applicationName"+ "." + $jsonFileName
# Transform the "Parameter" elements into a nested hashtable.
# Convert any values that can be interpreted as [int] to [int] and strip out any comments in the xml file.
$hash = [ordered] #{}
$appParametersXml.Application.Parameters.ChildNodes | Where-Object {$_.NodeType -ne 'Comment'} | % {
$hash[$_.Name] = #{ value = if ($num = $_.Value -as [int]) { $num } else { $_.Value }
}
}
# Wrap the hashtable in a top-level hashtable and convert to JSON.
[ordered] #{
'$schema' = 'https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#'
contentVersion ='1.0.0.0'
parameters = $hash
} | ConvertTo-Json |Out-File $parameterJsonFile
Write-Host "The JSON File is: " $parameterJsonFile
After I build the hash table with existing information from the XML file, I need to add additional parameter values like this Before converting to JSON
"parameters": {
"applicationName": {
"value": "somevalue"
},
"applicationTypeName": {
"value": "somevalue"
},
"applicationTypeVersion": {
"value": "somevalue"
},
Everything that I have tried so far has given me this as additional values. The regular XML values are being converted the correct way but the additional items that I am adding before converting are coming up like this!
"applicationName": "somevalue"
How can i seperate that out on different lines?
So, assuming your input xml file looks something like this ...
<application>
<parameters>
<applicationName>My Awesome App</applicationName>
<!--my awesome comment-->
<applicationTypeName>Mobile App</applicationTypeName>
<applicationTypeVersion>299</applicationTypeVersion>
<!--my other awesome comment-->
</parameters>
</application>
Here is my revised PowerShell ... you can't use if ($num = $_.Value -as [int]) casting as it won't work for 0, as it would be interpreted as false. I prefer to break the steps down and test and check each. Also I've used InnerText for the node value instead of Value as typically Value is evaluated as $null and I'm not sure what your xml looks like.
$fileXml = "./config.xml"
$fileJson = "./config.json"
$xmlContent = [Xml](Get-Content $fileXml)
# Transform the "Parameter" elements into a nested hashtable.
# Set any string values which are integers as [int] and strip out any comments in the xml file.
$parameters = [ordered]#{}
$nodes = $xmlContent.application.parameters.ChildNodes.Where{ $_.NodeType -ne 'Comment' }
foreach ($node in $nodes) {
$parameter = $node.Name
$value = $node.InnerText
if ($value -match "^\d+$") { $value = [int] $value }
$parameters.Add($parameter, #{ value = $value })
}
# if you need to add additional attributes, it's as simple as:
$parameters.Add("newParameter1", #{ value = "newValue1" })
$parameters.Add("newParameter2", #{ value = "newValue2" })
# Wrap the hashtable in a top-level hashtable and convert to JSON.
[ordered]#{
'$schema' = 'https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#'
contentVersion = '1.0.0.0'
parameters = $parameters
} | ConvertTo-Json | Out-File $fileJson
And here is the output saved to the json file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"applicationName": {
"value": "My Awesome App"
},
"applicationTypeName": {
"value": "Mobile App"
},
"applicationTypeVersion": {
"value": 299
},
"newParameter1": {
"value": "newValue1"
},
"newParameter2": {
"value": "newValue2"
}
}
}
In case anyone else runs into this, it was as simple as doing this after the hash table gets created with the existing XML file
$appParametersFileName = "$appParameterFilePath\$appParameterFile"
$appParametersXml = [Xml] (Get-Content "$appParametersFileName")
$parameterJsonFile = "$appParameterFilePath\$applicationName"+ "." + $jsonFileName
# Transform the "Parameter" elements into a nested hashtable.
# Convert any values that can be interpreted as [int] to [int] and strip out any comments in the xml file.
$hash = [ordered] #{}
$appParametersXml.Application.Parameters.ChildNodes | Where-Object {$_.NodeType -ne 'Comment'} | % {
$hash[$_.Name] = #{ value = if ($num = $_.Value -as [int]) { $num } else { $_.Value }
}
}
$hash["newvalue1"]=#{ value="value1"}
$hash["newvalue2"]=#{ value="value2"}
# Wrap the hashtable in a top-level hashtable and convert to JSON.
[ordered] #{
'$schema' = 'https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#'
contentVersion ='1.0.0.0'
parameters = $hash
} | ConvertTo-Json |Out-File $parameterJsonFile
Write-Host "The JSON File is: " $parameterJsonFile
How do I modify a Key Value Pair in a JSON File with powershell?
We are trying to modify Database Connection, sometimes it can be two levels nested deep, sometimes it can be three levels deep.
Trying to utilize this answer,
Currently we are switching servers in multiple json files, so we can test in different server environments.
Add new key value pair to JSON file in powershell.
"JWTToken": {
"SecretKey": "Security Key For Generate Token",
"Issuer": "ABC Company"
},
"AllowedHosts": "*",
"ModulesConfiguration": {
"AppModules": [ "ABC Modules" ]
},
"ConnectionStrings": {
"DatabaseConnection": "Server=testserver,1433;Database=TestDatabase;User Id=code-developer;password=xyz;Trusted_Connection=False;MultipleActiveResultSets=true;",
"TableStorageConnection": "etc",
"BlobStorageConnection": "etc"
},
Once you convert JSON string to an object with PowerShell, it's not really a problem to then change the properties. The main issue you are going to face here is that your string is currently invalid JSON for .Net or at least it won't be expecting it in the current format. We can fix that though.
Here is your current JSON.
"JWTToken": {
"SecretKey": "Security Key For Generate Token",
"Issuer": "ABC Company"
},
"AllowedHosts": "*",
"ModulesConfiguration": {
"AppModules": [ "ABC Modules" ]
},
"ConnectionStrings": {
"DatabaseConnection": "Server=testserver,1433;Database=TestDatabase;User Id=code-developer;password=xyz;Trusted_Connection=False;MultipleActiveResultSets=true;",
"TableStorageConnection": "etc",
"BlobStorageConnection": "etc"
},
There may be other issues, for PowerShell JSON, in your application.config file, but these two are immediately noticeable to me.
Unnecessary trailing commas
No definitive opening { and closing }
How Can We Fix This?
We can use simple string concatenation to add { and } where necessary.
$RawText = Get-Content -Path .\path_to\application.config -Raw
$RawText = "{ " + $RawText + " }"
To remove any unnecessary parsing issues with trailing commas when parsing the JSON with ConvertFrom-Json we need to remove them via regex. My proposed approach would be to identify them by whether the current array } or ] closes after them, it might be that these closing brackets have a number of spaces or \s before they appear. So we would have a regex that looks like this:
"\,(?=\s*?[\}\]])".
We could then use that with -replace in PowerShell. Of course we will replace them with an empty string.
$FormattedText = $RawText -replace "\,(?=\s*?[\}\]])",""
From here we convert to JSON.
$JsonObj = $FormattedText | ConvertFrom-Json
We can now change your database string by setting a property.
$JsonObj.ConnectionStrings.DatabaseConnection = "your new string"
We use ConvertTo-Json to convert the array back to a Json string.
$JsonString = $JsonObj | ConvertTo-Json
It's not important to return the trailing commas, they aren't valid JSON, but your file needs the first { and last } removing before we commit it back to file with Set-Content.
# Remove the first { and trim white space. Second TrimStart() clears the space.
$JsonString = $JsonString.TrimStart("{").TrimStart()
# Repeat this but for the final } and use TrimEnd().
$JsonString = $JsonString.TrimEnd("}").TrimEnd()
# Write back to file.
$JsonString | Set-Content -Path .\path_to\application.config -Force
Your config file should be written back more or less as you found it. I will try and think of a regex to fix the appearance of the formatting, it shouldn't error, it just doesn't look great. Hope that helps.
EDIT
Here is a function to fix the unsightly appearance of the text in the file.
function Restore-Formatting {
Param (
[parameter(Mandatory=$true,ValueFromPipeline=$true)][string]$InputObject
)
$JsonArray = $InputObject -split "\n"
$Tab = 0
$Output = #()
foreach ($Line in $JsonArray) {
if ($Line -match "{" -or $Line -match "\[") {
$Output += (" " * $Tab) + $Line.TrimStart()
$Tab += 4
}
elseif ($Line -match "^\s+}" -or $Line -match "^\s+\]") {
$Tab -= 4
$Output += (" " * $Tab) + $Line.TrimStart()
}
else {
$Output += (" " * $Tab) + $Line.TrimStart()
}
}
$Output
}
TL;DR Script:
$RawText = Get-Content -Path .\path_to\application.config -Raw
$RawText = "{ " + $RawText + " }"
$FormattedText = $RawText -replace "\,(?=\s*?[\}\]])",""
$JsonObj = $FormattedText | ConvertFrom-Json
$JsonObj.ConnectionStrings.DatabaseConnection = "your new string"
$JsonString = $JsonObj | ConvertTo-Json
$JsonString = $JsonString.TrimStart("{").TrimStart()
$JsonString = $JsonString.TrimEnd("}").TrimEnd()
$JsonString | Restore-Formatting | Set-Content -Path .\path_to\application.config -NoNewLine -Force
So i'm trying to convert XML files on an Azure Storage Container to JSON in the same container.
This way I'm able to read the information into an Azure SQL Database via an Azure Datafactory.
I'd like to stay clear from using Logic apps if able.
The JSON files need to be formatted.
And all this through the use of PowerShell scripting.
What i've got so far after some searching on the interwebs and shamelessly copying and pasting powershell code:
#Connect-AzAccount
# Helper function that converts a *simple* XML document to a nested hashtable
# with ordered keys.
function ConvertFrom-Xml {
param([parameter(Mandatory, ValueFromPipeline)] [System.Xml.XmlNode] $node)
process {
if ($node.DocumentElement) { $node = $node.DocumentElement }
$oht = [ordered] #{}
$name = $node.Name
if ($node.FirstChild -is [system.xml.xmltext]) {
$oht.$name = $node.FirstChild.InnerText
} else {
$oht.$name = New-Object System.Collections.ArrayList
foreach ($child in $node.ChildNodes) {
$null = $oht.$name.Add((ConvertFrom-Xml $child))
}
}
$oht
}
}
function Format-Json
{
<#
.SYNOPSIS
Prettifies JSON output.
.DESCRIPTION
Reformats a JSON string so the output looks better than what ConvertTo-Json outputs.
.PARAMETER Json
Required: [string] The JSON text to prettify.
.PARAMETER Minify
Optional: Returns the json string compressed.
.PARAMETER Indentation
Optional: The number of spaces (1..1024) to use for indentation. Defaults to 4.
.PARAMETER AsArray
Optional: If set, the output will be in the form of a string array, otherwise a single string is output.
.EXAMPLE
$json | ConvertTo-Json | Format-Json -Indentation 2
#>
[CmdletBinding(DefaultParameterSetName = 'Prettify')]
Param(
[Parameter(Mandatory = $true, Position = 0, ValueFromPipeline = $true)]
[string]$Json,
[Parameter(ParameterSetName = 'Minify')]
[switch]$Minify,
[Parameter(ParameterSetName = 'Prettify')]
[ValidateRange(1, 1024)]
[int]$Indentation = 4,
[Parameter(ParameterSetName = 'Prettify')]
[switch]$AsArray
)
if ($PSCmdlet.ParameterSetName -eq 'Minify')
{
return ($Json | ConvertFrom-Json) | ConvertTo-Json -Depth 100 -Compress
}
# If the input JSON text has been created with ConvertTo-Json -Compress
# then we first need to reconvert it without compression
if ($Json -notmatch '\r?\n')
{
$Json = ($Json | ConvertFrom-Json) | ConvertTo-Json -Depth 100
}
$indent = 0
$regexUnlessQuoted = '(?=([^"]*"[^"]*")*[^"]*$)'
$result = $Json -split '\r?\n' |
ForEach-Object {
# If the line contains a ] or } character,
# we need to decrement the indentation level unless it is inside quotes.
if ($_ -match "[}\]]$regexUnlessQuoted")
{
$indent = [Math]::Max($indent - $Indentation, 0)
}
# Replace all colon-space combinations by ": " unless it is inside quotes.
$line = (' ' * $indent) + ($_.TrimStart() -replace ":\s+$regexUnlessQuoted", ': ')
# If the line contains a [ or { character,
# we need to increment the indentation level unless it is inside quotes.
if ($_ -match "[\{\[]$regexUnlessQuoted")
{
$indent += $Indentation
}
$line
}
if ($AsArray) { return $result }
return $result -Join [Environment]::NewLine
}
# Storage account details
$resourceGroup = "insert resource group here"
$storageAccountName = "insert storage account name here"
$container = "insert container here"
$storageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $resourceGroup -Name $storageAccountName).Value[0]
$storageAccount = Get-AzStorageAccount -ResourceGroupName $resourceGroup -Name $storageAccountName
# Creating Storage context for Source, destination and log storage accounts
#$context = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
$context = New-AzStorageContext -ConnectionString "insert connection string here"
$blob_list = Get-AzStorageBlob -Container $container -Context $context
foreach($blob_iterator in $blob_list){
[XML](Get-AzStorageBlobContent $blob_iterator.name -Container $container -Context $context) | ConvertFrom-Xml | ConvertTo-Json -Depth 11 | Format-Json | Set-Content ($blob_iterator.name + '.json')
}
Output =
Cannot convert value "Microsoft.WindowsAzure.Commands.Common.Storage.ResourceModel.AzureStorageBlob" to type "System.Xml.XmlDocument". Error: "The specified node cannot
be inserted as the valid child of this node, because the specified node is the wrong type."
At C:\Users\.....\Convert XML to JSON.ps1:116 char:6
+ [XML](Get-AzStorageBlobContent $blob_iterator.name -Container $c ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvalidCastToXmlDocument
When I run the code the script asks me if I want to download the xml file to a local folder on my laptop.
This is not what I want, I want the conversion to be done in Azure on the Storage container.
And I think that I'm adding ".json" to the .xml file name.
So the output would become something like filename.xml.json instead of just filename.json
What's going wrong here?
And how can it be fixed?
Thank you in advance for your help.
JSON:
[
{
"Category-1": [
"Value1"
]
},
{
"Category-2": [
"Value1"
]
},
{
"Category-3": [
"Value1",
"Value2"
]
}
]
PowerShell Script:
$jsonToParse = (Get-Content -Path $jsonPath) -join "`n" | ConvertFrom-Json
foreach ($entry in $jsonToParse) {
log -Message ($entry) #Log function spits output to file
}
Output:
[10:39:03]#{Category-1=System.Object[]}
[10:39:03]#{Category-2=System.Object[]}
[10:39:03]#{Category-3-Med=System.Object[]}
How can I parse this? I have square brackets mixed with curly brackets, and I'm having a hard time finding a foothold with which to really get at the data.
What can I do to get the "Category" names? What can I do to get the "Values" for each category name? The fact that these aren't all key/value pairs is what's causing me trouble, I think.
I think what you are after is something like this:
# use the -Raw switch to get the file content as one single string
$jsonToParse = Get-Content -Path $jsonPath -Raw | ConvertFrom-Json
foreach ($entry in $jsonToParse) {
# format a string for the log file using the object ($entry) Name followed by the Value
# This Value can be an array of more than one items, so join these with a comma
$msg = '{0} = {1}' -f $entry.PSObject.Properties.Name, ($entry.PSObject.Properties.Value -join ', ')
log -Message $msg #Log function spits output to file
}
output:
[10:39:03]Category-1 = Value1
[10:39:03]Category-2 = Value1
[10:39:03]Category-3 = Value1, Value2
You have an array which contains values ("Category-1" etc). The values contain arrays with values ("Value1" etc).
$jsonToParse[0] = Category-1
$jsonToParse[1] = Category-2
$jsonToParse[2] = Category-3
$jsonToParse[1].'Category-2' = Value1
If you fill in the above examples, the output is like this!
How can I replace a value in a JSON file with another value taken from a config file (JSON)?
This is the code for the JSON files:
Config.json
{
"ABlob_AAD_Snapshot": [
{
"name": "properties.availability.frequency",
"value": "$Frequency$"
}]
}
ABlob_AAD_Snapshot.json
{
"name": "AzureBlobStore",
"properties": {
"type": "AzureStorage",
"availability": {
"frequency": "Value from Config.json file"
}
}
}
What I'm trying to do is the following:
Go through the Config.json file and store the values for "name" and "value" segments in variables. The "name" segment value is a path in ABlob_AAD_Snapshot.json file.
Follow the path in the ABlob_AAD_Snapshot.json file and replace the segment "frequency" with the value of segment "value" ($frequency$)
After this process, the ABlob_AAD_Snapshot.json should look like this:
{
"name": "AzureBlobStore",
"properties": {
"type": "AzureStorage",
"availability": {
"frequency": "$frequency$"
}
}
}
The problem here is that my original config.json file has more than one array (which represents file names) so I will parse more than one file and the value for "name" segment will not always be the same I mean, in this case, the value (or path) is properties.availability.frequency but it could be properties.activities.scheduler.interval or properties.activities.typeProperties.extendedProperties.WebAppClientID.
So as you can see the name and the quantity of "nodes" could change.
This is my PowerShell script:
$ScriptPath = split-path -parent $MyInvocation.MyCommand.Definition
#path to config file
$ConfigFile = "$ScriptPath\ParameterConfigOriginal.json"
#Convert the json file to PSObject
$json = Get-Content $ConfigFile | Out-String | ConvertFrom-Json
#Get all the arrays (files) in Conifg.json
$files = $json | Get-Member -MemberType Properties | Select-Object -ExpandProperty Name
#Go through all the arrays (files)
Foreach($file in $files)
{
if( $file -eq '$schema') {continue}
#store the path of the file to be modified
$FileName = $file + ".json"
$FilePath = "$ScriptPath\LinkedServices\" + $FileName"
#Go through all the elements of the arrray
Foreach($item in $json.$file)
{
#Store the path
$name = $item.name
#Store the value
$value = $item.value
#Convert the file to be modified to PSObject
$file = Get-Content $FilePath | Out-String | ConvertFrom-Json
#======STUCK IN HERE=============
# How can dynamically navigate through the file nodes like this?
$file.properties.availability.frequency
#and set the corresponding value
$file.properties.availability.frequency = $value
}
}
I'm new in the PowerShell world and I don't know if there is a cmdlet that helps me to do what I need.
Any suggestion will be appreciated.
EDIT
Simple Path
$snapshot.properties.availability.frequency
Path with arrays
$snapshot.properties.activities[0].scheduler.frequency
this is an example of the JSON file with arrays
Destination file
and this is the result
Destination file updated
Any idea on what could be happening?
Invoke-Expression will help you.
#Go through all the arrays (files)
Foreach($file in $files)
{
$snapshot = (Get-Content ("./" + $file + ".json") | ConvertFrom-Json)
# get config corresponds to the $file
$config = Invoke-Expression ('$json.' + $file)
# set value according to the config
Invoke-Expression ('$snapshot.' + $config.name + "='" + $config.value + "'")
# $snapshot.properties.availability.frequency
# -> $Frequency$
}
Edit
You have to use ConvertTo-Json -Depth 100 to write the result to JSON file properly (specify the appropriate depth according to your JSON files.).
Without -Depth option, you will get the result like "#{type=Copy, typeProperties=;, ...}".