Unable to convert Yaml to Powershell Object - json

Above is variables.stg.yml
I am trying to read it in my Powershell code. I used https://www.powershellgallery.com/packages/powershell-yaml/0.4.2 for this
$os_list = (Get-Content -Raw -Path ..\variables.stg.yml| ConvertFrom-Yaml)
$json = $os_list | ConvertTo-Json
Write-Host $json
#Convert JSON file to an object
$JsonParameters = ConvertFrom-Json -InputObject $json
Write-Host $JsonParameters
#Create new PSObject with no properties
$oData = New-Object PSObject
#Loop through properties of the $JsonParameters.parameters object, and add them to the new blank object
$JsonParameters.parameters.psobject.Properties.Name |
ForEach{
Add-Member -InputObject $oData -NotePropertyName $_ -NotePropertyValue
$JsonParameters.parameters.$_.Value
}
Write-Host $oData
However what i see is :

The immediate problem with your code is that you're referencing $JsonParameter.parameters when you really want $JsonParameters.variables - the property name in the yaml file is variables, not parameters.
A less cumbersome way to obtain an object with the ABC and Test entries from the yaml file as properties would be to simply cast the hashtable generated by ConvertTo-Yaml to a [PSCustomObject]:
$documentWithVariables = Get-Content -Path ..\variables.stg.yml -Raw |ConvertFrom-Yaml
$oData = [PSCustomObject]$documentWithVariables.variables
Much simpler :)

It seems like you're trying to use Convertto-YAML to converted to serialized JSON.
Accordding to their documentation you need to use the jsoncompatible flag to do this.
Converting from YAML to JSON
The awesome YamlDotNet assembly allows us to serialize an object in a
JSON compatible way. Unfortunately it does not support indentation.
Here is a simple example:
Import-Module powershell-yaml
PS C:\> $yaml = #"
anArray:
- 1
- 2
- 3
nested:
array:
- this
- is
- an
- array
hello: world
"#
PS C:\> $obj = ConvertFrom-Yaml $yaml
PS C:\> $obj
Name Value
---- -----
anArray {1, 2, 3}
nested {array}
hello world
PS C:\> ConvertTo-Yaml -JsonCompatible $obj
{"anArray": [1, 2, 3], "nested": {"array": ["this", "is", "an", "array"]}, "hello": "world"}
# Or you could do it in one line.
PS C:\> ConvertFrom-Yaml $yaml | ConvertTo-Yaml -JsonCompatible
{"anArray": [1, 2, 3], "nested": {"array": ["this", "is", "an", "array"]}, "hello": "world"}
Your array is also not formatted correctly to be imported as a nested. Below is correct syntax:
variables:
ABC:
XYz
Test:
preprod01
Finally:
[pscustomobject]$os_list = (ConvertFrom-Yaml -yaml (get-content -Raw C:\Powershell\TestCSVs\variables.stg.yml))
[pscustomobject]$os_list = ConvertTo-Yaml $os_list -JsonCompatible
$os_list
$oData = $os_list.variables
$oData
PS:> {"variables": {"ABC": "XYz", "Test": "preprod01"}}

Related

Assign Value To Variable After Extracting From Json File Using Powershell

I have the below json file
{
"name": "ca",
"version": "5.2.0"
}
I am extracting the value of version and trying to assign it to a variable ver
$jsonString = Get-Content -Path ./package.json
$jsonObj = $jsonString | ConvertFrom-Json
echo $jsonObj.version
Write-Host "##vso[task.setvariable variable=ver]$jsonObj.version"
echo $ver
Below is the output
5.2.0
I am expecting the value to be printed twice and also get it assigned to the variable ver but it is not getting assigned
If you are trying to use it in builds, you will not see a new value in the same step. Additionally, use $($jsonObj.version) in the logging command:
steps:
- powershell: |
$jsonString = Get-Content -Path ./package.json
$jsonObj = $jsonString | ConvertFrom-Json
$jsonObj.version
Write-Host "##vso[task.setvariable variable=my.ver]$($jsonObj.version)"
displayName: 'Set vars'
- powershell: |
$ver = '$(my.ver)'
$ver
displayName: 'Read vars'

how to format json string for aws with powershell

i want to send an event to aws via a cli command in a powershell script. Here is the Json i need to send to the eventbridge:
[
{
"Sensor":"{\"id\":\"880/2021-04-13\",\"attributes\":\"green\",\"Name\":\"SensorGreen\",\"state\":\"SUCCEEDED\"}",
"Source":"google.com"
}
]
Thats what a tried in powershell:
$json='[{"Sensor":"{\"id\":\"880/2021-04-13\",\"attributes\":\"green\",\"Name\":\"SensorGreen\",\"state\":\"SUCCEEDED\"}","Source":"google.com"}]'|ConvertTo-Json -Compress
aws events put-events --entries $json --region "eu-central-1"
That does not work. I even tried to write it to a json file and read it and send it from the file but it doesnt work. It somehow leads to too many "\" slashes or no slashes for my "Sensor" where it is necessary. I even tried to create a object and just convert the Sensor object to Json and then the whole object again to have the escaping, but it is not working.
EDIT:
i tried also this as mentioned in the answer:
$RequestObject = [pscustomobject] #(#{
Sensor = [pscustomobject] #{
id = "880/2021-04-13"
attribute = "green"
Name = "SensorGreen"
state = "SUCCEEDED"
}
Source = "google.com"
})
$RequestObject.Get(0).Sensor=$RequestObject.Get(0).Sensor | ConvertTo-Json -Compress
$Json = $RequestObject | ConvertTo-Json -Compress
aws events put-events --entries $Json --region "eu-central-1"
i included the #() to have an array and converted the sensor object twice to json to include the slashes. but the result is:
Error parsing parameter '--entries': Invalid JSON: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
if i use the command line interface only with the aws command and put the object directly into the command, then it works.. but the powershell does not.
You're passing a JSON string to an external program (aws)
Irrespective of how you constructed that string - directly as a string, or from an object / hashtable via ConvertTo-Json - as of PowerShell 7.1 - manual escaping of embedded " as \" is required, which, in turn requires escaping the preexisting \ preceding the embedded " as \\.
Note: None of this should be necessary, but due to a long-standing bug in PowerShell is - see this answer for details.
PowerShell Core 7.2.0-preview.5 now includes experimental feature PSNativeCommandArgumentPassing with an attempted fix, but, unfortunately, it looks like it will lack important accommodations for high-profile CLIs on Windows - see this summary from GitHub issue #15143. However, it would fix calls to aws.exe, the Amazon Web Services CLI.
# The JSON string to be passed as-is.
$json = #'
[
{
"Sensor":"{\"id\":\"880/2021-04-13\",\"attributes\":\"green\",\"Name\":\"SensorGreen\",\"state\":\"SUCCEEDED\"}",
"Source":"google.com"
}
]
'#
# Up to at least PowerShell 7.1:
# Manually perform the required escaping, to work around PowerShell's
# broken argument-passing to external programs.
# Note:
# -replace '\\', '\\' *looks* like a no-op, but replaces each '\' with '\\'
$jsonEscaped = $json -replace '\\', '\\' -replace '"', '\"'
# Now pass the *escaped* JSON string to the aws CLI:
aws events put-events --entries $jsonEscaped --region "eu-central-1"
ConvertTo-Json is for converting objects in PowerShell, not a string that you have tried to already write in Json. Your $Json variable produces this.
"[{\"Sensor\":\"{\\\"id\\\":\\\"880/2021-04-13\\\",\\\"attributes\\\":\\\"green\\\",\\\"Name\\\":\\\"SensorGreen\\\",\\\"state\\\":\\\"SUCCEEDED\\\"}\",\"Source\":\"google.com\"}]"
If you want to create the object in PowerShell and convert it to Json, then you can do this.
$RequestObject = [pscustomobject] #{
Sensor = [pscustomobject] #{
id = "880/2021-04-13"
attribute = "green"
Name = "SensorGreen"
state = "SUCCEEDED"
}
Source = "google.com"
}
$Json = $RequestObject | ConvertTo-Json -Compress
aws events put-events --entries $Json --region "eu-central-1"
Your Json will look like this if you print your variable out.
{"Sensor":{"id":"880/2021-04-13","attribute":"green","Name":"SensorGreen","state":"SUCCEEDED"},"Source":"google.com"}
Which I think is like the Json that the command is expecting. Not entirely sure why you need the strings escaping or the array. Here it is uncompressed.
{
"Sensor": {
"id": "880/2021-04-13",
"attribute": "green",
"Name": "SensorGreen",
"state": "SUCCEEDED"
},
"Source": "google.com"
}
Just noticed the powershell-2.0 tag. If you are using it, then you should do this instead to create your Json.
$Sensor = New-Object psobject -Property #{
id = "880/2021-04-13"
attribute = "green"
Name = "SensorGreen"
state = "SUCCEEDED"
}
$RequestObject = New-Object psobject -Property #{
Sensor = $Sensor
Source = "google.com"
}
$Json = $RequestObject | ConvertTo-Json -Compress
EDIT
If you absolutely must escape the strings in that way and have a single item array, then you should just pass the Json that you have written in your answer without any further conversion.
$json='[{"Sensor":"{\"id\":\"880/2021-04-13\",\"attributes\":\"green\",\"Name\":\"SensorGreen\",\"state\":\"SUCCEEDED\"}","Source":"google.com"}]'
If you want to make PowerShell do that for you then you would need to perform some string replacement on the Sensor object first.
PowerShell 2.0
$Sensor = New-Object psobject -Property #{
id = "880/2021-04-13"
attribute = "green"
Name = "SensorGreen"
state = "SUCCEEDED"
}
$SensorJson = $Sensor | ConvertTo-Json -Compress
$SensorJson.Replace("`"","\`"")
$RequestObject = New-Object psobject -Property #{
Sensor = $SensorJson
Source = "google.com"
}
$Json = $RequestObject | ConvertTo-Json -Compress
PowerShell 3.0+
$Sensor = [pscustomobject] #{
id = "880/2021-04-13"
attribute = "green"
Name = "SensorGreen"
state = "SUCCEEDED"
}
$SensorJson = $Sensor | ConvertTo-Json -Compress
$SensorJson.Replace("`"","\`"")
$RequestObject = [pscustomobject] #{
Sensor = $SensorJson
Source = "google.com"
}
$Json = $RequestObject | ConvertTo-Json -Compress
Then your AWS command
# Add the array around the compressed Json string.
aws events put-events --entries "[$Json]" --region "eu-central-1"
"[$Json]" prints
[{"Sensor":"{\"id\":\"880/2021-04-13\",\"attribute\":\"green\",\"Name\":\"SensorGreen\",\"state\":\"SUCCEEDED\"}","Source":"google.com"}]

Find and Replace Nested JSON Values with Powershell

I have an appsettings.json file that I would like to transform with a PowerShell script in a VSTS release pipeline PowerShell task. (BTW I'm deploying a netstandard 2 Api to IIS). The JSON is structured like the following:
{
"Foo": {
"BaseUrl": "http://foo.url.com",
"UrlKey": "12345"
},
"Bar": {
"BaseUrl": "http://bar.url.com"
},
"Blee": {
"BaseUrl": "http://blee.url.com"
}
}
I want to replace BaseUrl and, if it exists, the UrlKey values in each section which are Foo, Bar and Blee. (Foo:BaseUrl, Foo:UrlKey, Bar:BaseUrl, etc.)
I'm using the following JSON structure to hold the new values:
{
"##{FooUrl}":"$(FooUrl)",
"##{FooUrlKey}":"$(FooUrlKey)",
"##{BarUrl}":"$(BarUrl)",
"##{BleeUrl}":"$(BleeUrl)"
}
So far I have the following script:
# Get file path
$filePath = "C:\mywebsite\appsettings.json"
# Parse JSON object from string
$jsonString = "$(MyReplacementVariablesJson)"
$jsonObject = ConvertFrom-Json $jsonString
# Convert JSON replacement variables object to HashTable
$hashTable = #{}
foreach ($property in $jsonObject.PSObject.Properties) {
$hashTable[$property.Name] = $property.Value
}
# Here's where I need some help
# Perform variable replacements
foreach ($key in $hashTable.Keys) {
$sourceFile = Get-Content $filePath
$sourceFile -replace $key, $hashTable[$key] | Set-Content $filePath
Write-Host 'Replaced key' $key 'with value' $hashTable[$key] 'in' $filePath
}
Why are you defining your replacement values as a JSON string? That's just going to make your life more miserable. If you're defining the values in your script anyway just define them as hashtables right away:
$newUrls = #{
'Foo' = 'http://newfoo.example.com'
'Bar' = 'http://newbaz.example.com'
'Blee' = 'http://newblee.example.com'
}
$newKeys = #{
'Foo' = '67890'
}
Even if you wanted to read them from a file you could make that file a PowerShell script containing those hashtables and dot-source it. Or at least define the values as lists of key=value lines in text files, which can easily be turned into hashtables:
$newUrls = Get-Content 'new_urls.txt' | Out-String | ConvertFrom-StringData
$newKeys = Get-Content 'new_keys.txt' | Out-String | ConvertFrom-StringData
Then iterate over the top-level properties of your input JSON data and replace the nested properties with the new values:
$json = Get-Content $filePath | Out-String | ConvertFrom-Json
foreach ($name in $json.PSObject.Properties) {
$json.$name.BaseUrl = $newUrls[$name]
if ($newKeys.ContainsKey($name)) {
$json.$name.UrlKey = $newKeys[$name]
}
}
$json | ConvertTo-Json | Set-Content $filePath
Note that if your actual JSON data has more than 2 levels of hierarchy you'll need to tell ConvertTo-Json via the parameter -Depth how many levels it's supposed to convert.
Side note: piping the Get-Content output through Out-String is required because ConvertFrom-Json expects JSON input as a single string, and using Out-String makes the code work with all PowerShell versions. If you have PowerShell v3 or newer you can simplify the code a little by replacing Get-Content | Out-String with Get-Content -Raw.
Thank you, Ansgar for your detailed answer, which helped me a great deal. Ultimately, after having no luck iterating over the top-level properties of my input JSON data, I settled on the following code:
$json = (Get-Content -Path $filePath) | ConvertFrom-Json
$json.Foo.BaseUrl = $newUrls["Foo"]
$json.Bar.BaseUrl = $newUrls["Bar"]
$json.Blee.BaseUrl = $newUrls["Blee"]
$json.Foo.Key = $newKeys["Foo"]
$json | ConvertTo-Json | Set-Content $filePath
I hope this can help someone else.
To update values of keys at varying depth in the json/config file, you can pass in the key name using "." between the levels, e.g. AppSettings.Setting.Third to represent:
{
AppSettings = {
Setting = {
Third = "value I want to update"
}
}
}
To set the value for multiple settings, you can do something like this:
$file = "c:\temp\appSettings.json"
# define keys and values in hash table
$settings = #{
"AppSettings.SettingOne" = "1st value"
"AppSettings.SettingTwo" = "2nd value"
"AppSettings.SettingThree" = "3rd value"
"AppSettings.SettingThree.A" = "A under 3rd"
"AppSettings.SettingThree.B" = "B under 3rd"
"AppSettings.SettingThree.B.X" = "Z under B under 3rd"
"AppSettings.SettingThree.B.Y" = "Y under B under 3rd"
}
# read config file
$data = Get-Content $file -Raw | ConvertFrom-Json
# loop through settings
$settings.GetEnumerator() | ForEach-Object {
$key = $_.Key
$value = $_.Value
$command = "`$data.$key = $value"
Write-Verbose $command
# update value of object property
Invoke-Expression -Command $command
}
$data | ConvertTo-Json -Depth 10 | Out-File $file -Encoding "UTF8"

How to sort an object by keys using powershell

I have the following json file and I want it sorted by the keys/names. But so far I have been unable to figure out how to actually sort the json object by it's key/name.
Origional Settings.json
{
"files.trimTrailingWhitespace": true,
"workbench.startupEditor": "newUntitledFile",
"editor.tabSize": 4,
"editor.formatOnSave": true,
"editor.detectIndentation": false,
"editor.trimAutoWhitespace": true
}
Code:
# Get Json File
$JsonFile = 'C:\Settings.json'
# Convert from Json File to Json Object
$Json = Get-Content $JsonFile | Out-String | ConvertFrom-Json
# Sort Json Object (Does Not Work!!!)
$Json = $Json | Sort-Object -Property Name
#Convert Json Object to Json File
$Json | ConvertTo-Json -depth 100 | Set-Content $JsonFile
New Settings.Json
{
"editor.detectIndentation": false,
"editor.formatOnSave": true,
"editor.tabSize": 4,
"editor.trimAutoWhitespace": true
"files.trimTrailingWhitespace": true,
"workbench.startupEditor": "newUntitledFile"
}
$json | Select-Object ($json | Get-Member -MemberType NoteProperty).Name | ConvertTo-Json
Answer was here: Powershell sort PSObject alphabetically
This issue was that the json file did not have a collection to sort but was a single object whose properties I wanted to sort. Below is the code that works.
# Build an ordered hashtable of the property-value pairs.
$SortedByProperties = [ordered] #{}
Get-Member -Type NoteProperty -InputObject $Json | Sort-Object Name |
ForEach-Object { $SortedByProperties[$_.Name] = $Json.$($_.Name) }
# Create a new object that receives the sorted properties.
$JsonFileSorted = New-Object PSCustomObject
Add-Member -InputObject $JsonFileSorted -NotePropertyMembers $SortedByProperties
$JsonFileSorted | ConvertTo-Json -depth 100 | Set-Content $JsonFile

ConvertFrom-JSON to object

It looks like the way I am expecting this to work doesn't. I want multiple objects returned, but it seems to be returning just one. It is beyond me how I do it.
A very simple JSON file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"storageAccountName": {
"value": "sa01"
},
"virtualNetworkName": {
"value": "nvn01"
}
}
}
I want to dynamically add the parameters and their values into a nice pscustomobject (that would look like the following with the above data):
ParameterName | Value
===========================
storageAccountName | sa01
virtualNetworkName | nvn01
What I don't understand is why the following returns one object:
$TemplateParametersFile = "C:\Temp\deploy-Project-Platform.parameters.json"
$content = Get-Content $TemplateParametersFile -Raw
$JsonParameters = ConvertFrom-Json -InputObject $content
$JsonParameters.parameters | Measure-Object
Whilst writing this, I eventually found a solution that get's what I want, which I'll post in the answer section. Feel free to school me and improve...
I would do things a little differently, skipping the hashtable, and using the hidden PSObject property. So, picking up after you have the JSON data stored in $content, I would do something like this:
#Convert JSON file to an object
$JsonParameters = ConvertFrom-Json -InputObject $content
#Create new PSObject with no properties
$oData = New-Object PSObject
#Loop through properties of the $JsonParameters.parameters object, and add them to the new blank object
$JsonParameters.parameters.psobject.Properties.Name |
ForEach{
Add-Member -InputObject $oData -NotePropertyName $_ -NotePropertyValue $JsonParameters.parameters.$_.Value
}
$oData
By the way, it had issues converting the JSON you posted, I had to add quotes around the two values, such as "value": "sa01".
Using the same JSON file as shown above:
<#
# Read in JSON from file on disk
$TemplateParametersFile = "C:\Temp\deploy-Project-Platform.parameters.json"
$content = Get-Content $TemplateParametersFile -Raw
#>
#Retrieve JSON file from Azure storage account.
$TemplateParametersFile = "https://{storageAccountName}.blob.core.windows.net/{SomeContainer}/deploy-Project-Platform.parameters.json"
$oWc = New-Object System.Net.WebClient
$webpage = $oWc.DownloadData($TemplateParametersFile)
$content = [System.Text.Encoding]::ASCII.GetString($webpage)
#Convert JSON file to an object (IMHO- Sort of!)
$JsonParameters = ConvertFrom-Json -InputObject $content
#Build hashtable - easier to add new items - the whole purpose of this script
$oDataHash = #{}
$JsonParameters.parameters | Get-Member -MemberType NoteProperty | ForEach-Object{
$oDataHash += #{
$_.name = $JsonParameters.parameters."$($_.name)" | Select -ExpandProperty Value
}
}
#Example: adding a single item to the hashtable
$oDataHash.Add("VirtualMachineName","aDemoAdd")
#Convert hashtable to pscustomobject
$oData = New-Object -TypeName PSCustomObject
$oData | Add-Member -MemberType ScriptMethod -Name AddNote -Value {
Add-Member -InputObject $this -MemberType NoteProperty -Name $args[0] -Value $args[1]
}
$oDataHash.Keys | Sort-Object | ForEach-Object{
$oData.AddNote($_,$oDataHash.$_)
}
$oData
And the result:
storageAccountName VirtualMachineName virtualNetworkName
------------------ ------------------ ------------------
sa01 aDemoAdd nvn01
Agreed, the question asked for a Parameter / Value pair, and this results in the parameter's name being assigned as the noteproperty, but I think it will be easier to use it this way. Of course, $oDataHash returns it as a Key/value pair.
This script also pulls the JSON file directly from an Azure storage account. No need to save to disk. If you want to save to disk, change $oWc.DownloadData() to $oWc.DownloadFile() . The commented bit at the top, reads from disk.
I am sure there are much more succinct ways to achieve the same result, and I'd love to here them. For me, at the moment this works.