I'm trying to perform some config transformations on JSON files using PowerShell.
For this there's several input json files and a transform one (one for each environment).
Input sample (AzureStorage.json):
{
"$schema": "http://datafactories.schema.management.azure.com/schemas/2015-09-01/Microsoft.DataFactory.LinkedService.json",
"name": "AzureStorage",
"properties": {
"type": "AzureStorage",
"typeProperties": {
"connectionString": "My Connection String here"
} } }
Transform:
{
"$schema": "http://datafactories.schema.management.azure.com/vsschemas/V1/Microsoft.DataFactory.Config.json",
"AzureStorage": [
{
"name": "$.properties.typeProperties.connectionString",
"value": "DefaultEndpointsProtocol=https;AccountName=mytestaccount;AccountKey=d;lasfjdalfdjfldjfdsfds;EndpointSuffix=core.windows.net"
}
],
"DataLakeStore": [
{
"name": "$.properties.typeProperties.dataLakeStoreUri",
"value": "https://mydatalake.azuredatalakestore.net/webhdfs/v1"
}
]
}
What I need to do, is to load the transform file, then traverse it, finding the names of the input files I need to transform (in this example AzureStorage.json and DataLakeStore.json).
Next, I need to replace the properties accordingly. I'm trying to do it by loading the transform file into a variable using ConvertFrom-Json, but I not sure how to traverse it afterwards.
I don't know hat exactly you need. I'm guessing access to the information within the JSON file.
What about this approach?
$json_object = Get-Content -Raw -Path '<your_path>\transform.json' | ConvertFrom-Json
$azure_storage = #('AzureStorage'; 'DataLakeStore')
ForEach ($azure in $json_object) {
ForEach ($storage in $azure_storage) {
Write-Output $azure.$storage.name
Write-Output $azure.$storage.value
}
}
Edit Due to edit I got it. You need a generic access.
Here you go:
$json_object = (Get-Content -Path '<your_path>\transform.json') -join "`n" | ConvertFrom-Json
ForEach ($object in $json_object.PsObject.Properties) {
Write-Output $object.name
Write-Output $object.value
}
Explanation:
(Get-Content -Path '<your_path>\transform.json') -join "n"` is a Microsoft's MSDN recommended way to read json files.
You need to find out where the values are. The object you are using is a "Windows PowerShell custom object" - PsObject. To access the you need to use .Properties.value. Then you have the Strings you want and you can access them using .name and .value.
Related
I have a json file, simplified version of it looks like this:
{
"Location": "EU",
"Country": {
"City": "xxx",
"Town": "xxx"
},
"Transport": {
"Train": "xxx"
}
}
I have run the ConvertFrom-Json command to convert to PSObject:
$conversion = Get-Content $path | ConvertFrom-Json
This will give me an output like this:
Location : EU
Country : #{City="xxx"; Town="xxx"}
Transport : #{Train="xxx"}
Question
How can I get the nested values to print out separately? I would want them all to print out like the "Location:EU" one
Is there a different command to ConvertFrom-Json that i should be using for this? Or do I just need to mess around with ConvertFrom-Json command a bit?
To note:
I am not just looking for a pretty print out - I would need them all separately for a script I am writing that will be looping through all the key/value pairs
I have read about the -Depth flag when using ConvertFrom-Json and does not seem to fix anything here - it seemed this was more relevant for ConvertTo-Json
In order to report all leaf properties as name-value pairs (i.e. those properties that contain primitive JSON values as opposed to containing nested objects with properties and / or arrays), you need to recursively walk the object graph:
Find helper function Get-LeafProperty below; assuming you have already defined it, you can call it as follows:
#'
{
"Location": "EU",
"Country": {
"City": "xxx",
"Town": "xxy"
},
"Transport": {
"Train": "xxz"
}
}
'# |
ConvertFrom-Json |
Get-LeafProperty
Output (the display formatting of [pscustomobject] instances with .Name and .Value properties representing all the leaf properties):
Name Value
---- -----
Location EU
City xxx
Town xxy
Train xxz
Get-LeafProperty source code:
# Walks a potentially nested [pscustomobject] graph
# as returned by ConvertFrom-Json and outputs all
# leaf properties as name-value custom objects.
function Get-LeafProperty {
param([Parameter(ValueFromPipeline)] [object] $InputObject)
process {
if ($InputObject -is [array]) { # array as single input object -> recurse
foreach ($o in $InputObject) { Get-LeafProperty $o }
}
else {
# Assumed to be a (potentially nested) [pscustomobject] instance:
# Recursively process its properties.
foreach ($p in $InputObject.psobject.properties) {
if ($p.Value -is [array]) { # array -> recurse
foreach ($o in $p.Value) { Get-LeafProperty $o }
} elseif ($p.Value -is [System.Management.Automation.PSCustomObject] ) { # nested [pscustomobject] -> recurse
Get-LeafProperty $p.Value
} else { # leaf property reached -> output name-value pair
[pscustomobject] #{ Name = $p.Name; Value = $p.Value }
}
}
}
}
}
Note: A variant of this function that outputs property name paths (e.g. Country.City) instead of just their names (e.g. City) can be found in this answer.
I have two JSON files and want to transfer collection of objects from one file to another. Suppose, the from.json file contains property which represents collection of clients:
"Clients":
[
{
"Name": "Name1",
"Age": "12"
},
{
"Name": "Name2",
"Age": "14"
}
]
to.json file contains an empty collection, "Objects: []" ,which must be filled with objects from from.json. Each objects in toJson variable must contain additional property - Id, so eventually, my "to.json" file should look like this:
"Objects":
[
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
},
{
"Id": "{new-id}",
"Name": "Name1",
"Age": "12"
}
]
I've converted two files into variables:
$fromJson = (Get-Content -Raw -Path {fromPath}) | ConvertFrom-Json
$toJson = (Get-Content -Raw -Path {toPath}) | ConvertFrom-Json
I know that objects from fromJson to toJson can be transferred in the following manner:
toJson.Objects += fromJson.Clients, but that's not enough in my case. I think that it could be done by iterating through fromJson.Clients array but have no idea how to create an object and add it into toJson.Objects collection.
Here's a more efficient solution, based on:
Use of a calculated property with Select-Object, which allows you to place the new property first in the output objects.
Instead of building the array one by one with += (which is inefficient, because a new array must technically be created behind the scenes in every iteration), the solution below lets PowerShell collect the output objects of the Select-Object call in an array automatically (the [array] type constraint is needed to ensure that an array is created even if only one object happens to be output.)
# Sample input.
$fromJson = ConvertFrom-Json '{"Clients":[{"Name":"Name1","Age":"12"},{"Name":"Name2","Age":"14"}]}'
$toJson = ConvertFrom-Json '{ "Objects": [] }'
[array] $toJson.Objects =
$fromJson.Clients |
Select-Object #{ Name='Id'; Expression = { [string] (New-Guid) } }, *
$toJson | ConvertTo-Json -Depth 3 # append | Set-Content as needed.
Kind of new to the PowerShell, but after a bit of investigation came up with the following solution:
fromJson.Clients | ForEach-Object {
$_ | Add-Member -MemberType NoteProperty -Name 'Id' -Value ([guid]::NewGuid().Guid.ToString())
$toJson += $_
}
...
$toJson | ConvertTo-Json | Out-File {to.json_path}
Frankly, don't know if that is a 'proper' way to do that, but generally it works for that particular case. For now, see no other solution.
I am reading data from a json file via powershell, with the ultimate goal of updating said file. I need to split the data in a chunk I want to keep and a chunk I want to update, and to complicate the matter further, the place where I need to split the text varies throughout a foreach-loop, thus I need that part to come from a variable.
I have tried .split/-split and .replace/-replace in numerous configurations, but it seems this is harder than one would assume in powershell.
All the below files in the same folder.
Json (json.json):
{
"Section1": {
"Heading1": [
"Thing1",
"Thing2"
]
},
"Section2": [
"Thing1",
"Thing2"
]
}
Powershell (powershell.ps1):
$originalJsonString = Get-Content -Path ".\json.json"
$SplitTarget = "Section2"
$JsonString = {This is the part that I am iffy about}
Write-Output $JsonString
The output I want from the above would be
{
"Section1": {
"Heading1": [
"Thing1",
"Thing2"
]
},
I have tried just about everything I can think of in relation to split and replace, but yet the solution alludes me.
Note in the solution above, that it is an important factor that $originalJsonString is split (or whatever) by $SplitTarget, and not "Section2", as that is also a factor in my equation.
This is my first time asking, so if I am doing something wrong I apologise.
Thank you.
Edit:
It is only fair that I add the reason I don't convert to an object.
The syntax which powershell exports when converting json to an object and back is undesirable for my use.
However, if using an object is the ONLY way, and slitting is out of the question, then another solution must be found.
Thank you.
Edit:
If objects was the way to go, I ended up finding a way more complicated way to format the .json the way I wanted it.
Thanks everyone.
What you should do is read the json file in using :
$json = Get-Content .\json.json -raw | ConvertFrom-Json
I faked that here using a 'here-string':
$json = #"
{
"Section1": {
"Heading1": [
"Thing1",
"Thing2"
]
},
"Section2": [
"Thing1",
"Thing2"
]
}
"# | ConvertFrom-Json
Next, define what you want to keep or update:
$sectionToKeep = "Section1"
$sectionToUpdate = "Section2"
To see what is in there use $json.$sectionToKeep | ConvertTo-Json
{
"Heading1": [
"Thing1",
"Thing2"
]
}
Next, update $section2 leaving everything else untouched. I am writing an object that stores an array, just like in $sectionToKeep:
$json.$sectionToUpdate = #{'Heading2' = 'Thing3', 'Thing4'}
and finally output (or write back to file) the new complete json:
$json | ConvertTo-Json
Using your example gives you this:
{
"Section1": {
"Heading1": [
"Thing1",
"Thing2"
]
},
"Section2": {
"Heading2": [
"Thing3",
"Thing4"
]
}
}
Hope that helps
Not sure to really understand your question, but I would convert the Json to an object and then filter the sub data and create a file again
$obj = Get-Content json.json -raw | ConvertFrom-Json
If you are looking to just isolate a section and create a new json file, then you can use the following:
$file = Get-Content .\json.json | ConvertFrom-Json
$file | Select-Object Section1 | ConvertTo-Json | Set-Content New.json
When I try to access the below JSON file from powershell it show the following error: Missing property name after reference operator. Here is my JSON file:
{
"Vnet": {
"Service": "VirtualNetwork",
"Name": "vnetname",
"ResourceGroupName": "RGname",
"Location": "Southeast Asia",
"Address_Prefix": "11.0.0.0/16",
"No_of_subnets": "1",
"Subnet": {
"1": {
"SubnetName":"sub1",
"Address_Prefix":"11.0.1.0/24"
}
}
}
}
Here is the simple powershell command
$file = (Get-Content "//filelocation" | Out-String) | ConvertFrom-Json
Write-Output $file.Vnet.Subnet.1.SubnetName
Note: I'm author of the JSON file and I can make any changes to it too if needed.
PowerShell seems to misinterpret the 1 as an index. Put it in quotes (to make clear it's a property name) and the code will work as you expect:
Write-Output $file.Vnet.Subnet.'1'.SubnetName
I need to use bellow command in Azure Runbooks:
New-AzureRmDataFactoryLinkedService $MyDataFactoryObject -File "PATH/TO/JSON/STRING/FILE"
I am wondering to know is it possible to pass JSON Object itself instead of passing the file path?
I mean something like this:
$StorageLinkedService_JSON_str = '{
"name": "StorageName",
"properties": {
"type": "AzureStorage",
"description": "",
"typeProperties": {
"connectionString": "connectionString"
}
}
}'
$StorageLinkedService_Obj = ConvertFrom-Json -InputObject $StorageLinkedService_JSON_str
# I need to know is it possible to use one of these lines?
# New-AzureRmDataFactoryLinkedService $MyDataFactoryObject -File $StorageLinkedService_JSON_str
# OR This:
# New-AzureRmDataFactoryLinkedService $MyDataFactoryObject -File $StorageLinkedService_Obj
Depending on where the runbook is executed, a possibility might be to write your JSON data to a temporary file:
$jsonFilePath = 'C:\Windows\Temp\StorageLinkedService_JSON_str.json'
$StorageLinkedService_JSON_str |
Out-File -FilePath $jsonFilePath
New-AzureRmDataFactoryLinkedService $MyDataFactoryObject -File $jsonFilePath