Access Properties and Keys within JSON using PowerShell - json

I have a PowerShell script and would like to access a nested key. Here is my JSON:
{
"name": "versions.json",
"versions": {
"1.0.0": {
"Component1": "1.0.0",
"Component2": "1.0.0",
"Component3": "1.0.0"
},
"2.0.0": {
"Component1": "2.0.0",
"Component2": "2.0.0",
"Component3": "2.0.0"
}
}
}
I'm unsure of how to access the values within each version (1.0.0 and 2.0.0). I know I get the property name for each "version" by using:
($json.versions.PSobject.Properties) | ForEach-Object {
"Data: $($_.Name)"
}
But how do I iterate through each of a "version" objects properties and view its value, i.e. how do I check what is contained within "1.0.0"? For "1.0.0" I would expect to see
"Component1" at 1.0.0
"Component2" at 1.0.0
"Component3" at 1.0.0

Do the same you're doing for versions for the values of its properties:
$json.versions.PSobject.Properties | ForEach-Object {
"Data: $($_.Name)"
$_.Value.PSobject.Properties | ForEach-Object {
'"{0}" at {1}' -f $_.Name, $_.Value
}
}

Related

Integrate json values into another file

I'm trying to update an existing json file from values in another json file using jq in a bash shell.
I've got a settings json file
{
"Logging": {
"MinimumLevel": {
"Default": "Information",
"Override": "Warning"
},
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "./logs/log-.txt",
"rollingInterval": "Day"
}
}
]
},
"Settings": {
"DataServerUrl": "https://address.to.server.com",
"ServerKey": "1f969476798adfe95114dd28ed3a3ff"
"ServerTimeZone": "Mountain Standard Time",
"MaxOccupantCount": 6
}
}
In an integration step, I'm attempting to incorporate values for specific environments (think dev/staging/prod) from an external json file with limited setting values. An example of such a file is
{
"DataServerUrl": "https://dev.server.addr.com",
"ServerKey": "2a4d99233efea456b95114aa23ed342ae"
}
I can get to the data using jq. I can update the data using jq if I hard-code the updates. I'm looking for something general to take in any environment settings values and update them in the base settings file. My searches suggest I can do this in a single step without knowing the specific values. A command similar to
jq -r 'to_entries[]' settings.dev.json |
while IFS= read -r key value; do
jq -r '.[$key] |= [$value]' settings.json
done
What happens is I get error messages stating jq: error: $key is not defined at <top-level> (as well as the same message for $value). The messages appear several times in pairs. settings.json is not changed. Now, this makes partial sense because the output from just jq -r 'to_entries[]' settings.dev.json looks like (empty space in this output is included as produced by the command).
"key": "DataServerUrl",
"value": "https://dev.server.addr.com"
"key": "ServerKey",
"value": "2a4d99233efea456b95114aa23ed342ae"
How do I go about iterating over the values in the environment settings file such that I can use those values to update the base settings file for further processing (i.e., publishing to the target environment)?
The simplest way is to provide both files and address the second one using input. That way, all you need is the assignment:
jq '.Settings = input' settings.json insert.json
{
"Logging": {
"MinimumLevel": {
"Default": "Information",
"Override": "Warning"
},
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "./logs/log-.txt",
"rollingInterval": "Day"
}
}
]
},
"Settings": {
"DataServerUrl": "https://dev.server.addr.com",
"ServerKey": "2a4d99233efea456b95114aa23ed342ae"
}
}
Demo
You could do something like
jq -s '.[1] as $insert | .[0].Settings |= $insert | .[0]' settings.json insert.json
Where we :
slurp both files
Save insert.json to a variable called $insert
Append (|=) $insert to .[0].Settings
Show only the first file .[0]
So the output will become:
{
"Logging": {
"MinimumLevel": {
"Default": "Information",
"Override": "Warning"
},
"WriteTo": [
{
"Name": "File",
"Args": {
"path": "./logs/log-.txt",
"rollingInterval": "Day"
}
}
]
},
"Settings": {
"DataServerUrl": "https://dev.server.addr.com",
"ServerKey": "2a4d99233efea456b95114aa23ed342ae"
}
}

Set a value for an element found with select() but return the whole json

I have a JSON like this:
{
"name": "com.company1.package1",
"version": "0.2",
"dependencies": {
"com.company1.package2": "0.1",
"com.company2.package1": "2.3"
}
}
And I want to change all dependencies with keys starting with "com.company1" to "0.2".
I managed to change it with this:
jq '.dependencies | with_entries(select(.key|startswith("com.company1"))) | .[]="0.2"'
But this only returns
{
"com.company1.package2": "0.2"
}
I would like to get the whole initial JSON with only this value changed. Like so:
{
"name": "com.company1.package1",
"version": "0.2",
"dependencies": {
"com.company1.package2": "0.2",
"com.company2.package1": "2.3"
}
}
How could go about that?
You were almost there. Use the update operator |= and inside the update a direct assignment .value = "0.2".
jq '.dependencies |= with_entries(select(.key | startswith("com.company1")).value = "0.2")'
{
"name": "com.company1.package1",
"version": "0.2",
"dependencies": {
"com.company1.package2": "0.2",
"com.company2.package1": "2.3"
}
}
Demo
You're projecting to a filtered view of your object (with all the | in your filter). Since you just want to do assignments, make sure that is at the top of your filter. select the properties you want to adjust then do the assignment.
It'll be easier if you used paths to do this, since you're filtering based on the path.
setpath(path(.dependencies[]) | select(.[1] | startswith("com.company1")); "0.2")
which produces:
{
"name": "com.company1.package1",
"version": "0.2",
"dependencies": {
"com.company1.package2": "0.2",
"com.company2.package1": "2.3"
}
}
jqplay
Here is a ruby JSON to do this:
ruby -r JSON -e 'd=JSON.parse($<.read)
d["dependencies"].each {|k,v| d["dependencies"][k]="0.2" if k[/\bcompany1\b/] }
puts d.to_json' file | jq .
# using jq to pretty print...
Or, later versions of ruby, you can do:
ruby -r JSON -e 'd=JSON.parse($<.read)
d["dependencies"].each {|k,v| d["dependencies"][k]="0.2" if k[/\bcompany1\b/] }
puts JSON.pretty_generate(d)' file
Either prints:
{
"name": "com.company1.package1",
"version": "0.2",
"dependencies": {
"com.company1.package2": "0.2",
"com.company2.package1": "2.3"
}
}

Get value by the array element name in json

I'm not sure how to name these elements properly, it'll be easier just to show it. I have following JSON:
{
"DEV": [
{
"GitEmail": "asd#asd.com"
}
],
"TEST": [
{
"GitEmail": "asd1#asd.com"
}
],
"PROD": [
{
"GitEmail": "asd2#asd.com"
}
]
}
I would like to get the "DEV" by providing it's email. How to implement that in powershell?
Something like below can help -
PS> $json = '{
"DEV": [
{
"GitEmail": "asd#asd.com"
}
],
"TEST": [
{
"GitEmail": "asd1#asd.com"
}
],
"PROD": [
{
"GitEmail": "asd2#asd.com"
}
]
}' | ConvertFrom-Json
PS> ($json.psobject.Properties | ? {$_.Value -match "asd#asd.com"}).Name
Depending on the email matches you can retrieve the environment names.
I can't promise there is an easier method, but this here is one way:
Given that you json is stored in a variable $json:
You can get every head object with $json.psobject.properties.name:
Input:
$json.psobject.properties.name
Output:
DEV
TEST
PROD
With this we can create a foreach loop and search for the Email:
foreach ($dev in $json.psobject.properties.name)
{
if($json.$dev.GitEmail -eq "asd#asd.com") {
echo $dev
}
}
I do not know any elegant way of doing it. ConvertFrom-Json does not create neat objects with easy ways to traverse them like convertfrom-xml, instead result is just a PsObject with bunch of noteproperties.
What I do in such cases is
$a= #"
{
"DEV": [
{
"GitEmail": "asd#asd.com"
}
],
"TEST": [
{
"GitEmail": "asd1#asd.com"
}
],
"PROD": [
{
"GitEmail": "asd2#asd.com"
}
]
}
"#
$JsonObject= ConvertFrom-Json -InputObject $a
$NAMES= $JsonObject|Get-Member |WHERE MemberType -EQ NOTEPROPERTY
$NAMES|Foreach-Object {IF($JsonObject.$($_.NAME).GITEMAIL -EQ 'asd#asd.com'){$_.NAME}}
Result of above is
DEV
Not pretty, not really re-usable but works.
If anyone knows a better way of going about it - I'll be happy to learn it:)

How to Call Json parameters as arguments via Powershell task in VSTS release definition?

I have Json file having the below and Powershell script calling this Json file using the below script.But I want to call the Json values as arguments from VSTS Powershell task .Please help me out how to call inturn.
Given Below webjob_list.json I have user name, password, resource group, webapp name. I would like to call these 4 parameters as arguments via VSTS Release definition using Powershell task under argument section. I don't want to put these 4 values inside Json instead want to call as arguments from Powershell script.
Json Script:
WebJob_list.json:
{
"Dev":
{
"userName": "abc-dev\\$abcjobs-dev",
"password": "SKGGfiuqHdvJ09eCmhDmE4mBnL8PSAZhnQMxzb",
"webAppName": "abcjobs-dev",
"resourceGroup": "abc-Dev"
"webJobs": [
{
"name": "webjobname",
"typeName": "continuous"
},
{
"name": "webjobname",
"typeName": "continuous"
},
{
"name": "webjobname",
"typeName": "continuous"
},
{
"name": "webjobname",
"typeName": "continuous"
}
]
}
webjobs Stop.ps1:
param(
[string]$currentEnv
)
[object]$paramObj=Get-Content "d:\a\r1\a\Jobs\WebJobs_scripts\WebJob_list.json" |ConvertFrom-Json
$userName =$paramObj.$currentEnv
$password =$paramObj.$currentEnv
$webAppName =$paramObj.$currentEnv
$resourceGroup=$paramObj.$currentEnv
[object[]]$webJobs=$paramObj.$currentEnv.webJobs
foreach($wj in $webjobs)
{
if($wj.typeName -eq "continuous")
{
Invoke-AzureRmResourceAction -ResourceGroupName $resourceGroup -ResourceType Microsoft.Web/sites/ContinuousWebJobs -ResourceName "$webAppName/$($wj.name)" -Action stop -ApiVersion 2015-08-01 -Force
Write-Host "continuous"
Write-Host $wj.name
}
}
Json file:
{
"Dev": {
"webJobs": [
{
"name": "webjobname",
"typeName": "continuous"
},
{
"name": "webjobname",
"typeName": "continuous"
},
{
"name": "webjobname",
"typeName": "continuous"
},
{
"name": "webjobname",
"typeName": "continuous"
}
]
}
}
Script:
param(
[string]$currentEnv,
[string]$username
[string]$password,
[string]$webAppName,
[string]$resourceGroup
)
[object]$paramObj=Get-Content "d:\a\r1\a\Jobs\WebJobs_scripts\WebJob_list.json" |ConvertFrom-Json
[object[]]$webJobs=$paramObj.$currentEnv.webJobs
foreach($wj in $webjobs)
{
if($wj.typeName -eq "continuous")
{
Invoke-AzureRmResourceAction -ResourceGroupName $resourceGroup -ResourceType Microsoft.Web/sites/ContinuousWebJobs -ResourceName "$webAppName/$($wj.name)" -Action stop -ApiVersion 2015-08-01 -Force
Write-Host "continuous"
Write-Host $wj.name
}
}
Your script looks like it will already read the json, but to pass variables them between tasks you need to set them in VSTS from the script itself
Write-Host "##vso[task.setvariable variable=username]$username"
It's also possible to store your variables in VSTS itself from the outset which might be a good option?
VSTS > Releases > Pipeline > New
Next create a Dev environment
Click Variables link and create your 4 x variables.
Check that they are scoped to the Dev environment.
Then in your PowerShell task create a param block to accept the variables
param(username, password, webappname, resourcegroup)
Then in the Script Arguments box for the PowerShell task add the arguments as:
-username $(username) -password $(password) -webappname $(webappname) -resourcegroup $(resourcegroup)

Recursive extraction of object values and parent key name using jq

I have a requirement to parse the output of the npm ls --global --json command, such that I get a list of all the installed npm packages in the following format:
$package;$version;js;$resolved
Where:
$package is the key containing the package name, from each dependencies object.
$version is the version value taken from each package
js is just a literal string
$resolved is the resolved value taken from each package
I have gotten as far as this command syntax and output:
$ jq --raw-output 'select( has("dependencies") ) .dependencies[] | . as $d | "parentkey" + ";" + $d.version + ";js;" + $d.resolved'`
parentkey;5.5.1;js;
parentkey;1.1.3;js;https://registry.npmjs.org/yaml-table/-/yaml-table-1.1.3.tgz
The parts that I am specificly having difficulty with are as follows:
How can I get the key name value that I am iterating over in .dependencies that contains that package name. It seems that by that point I am looking at the contents of that object itself.
How can I recurse through ALL dependency objects? At the moment I'm only looking at the top level records in the root .dependencies object. I've discovered .. recursion, but I'm not quite sure how to apply it here.
Based on the example data below, I am trying to reach the following output results:
npm;5.5.1;js;
JSONStream;1.3.1;js;https://registry.npmjs.org/JSONStream/-/JSONStream-1.3.1.tgz
jsonparse;1.3.1;js;https://registry.npmjs.org/jsonparse/-/jsonparse-1.3.1.tgz
through;2.3.8;js;https://registry.npmjs.org/through/-/through-2.3.8.tgz
yaml-table;1.1.3;js;https://registry.npmjs.org/yaml-table/-/yaml-table-1.1.3.tgz
js-yaml;3.4.6;js;https://registry.npmjs.org/js-yaml/-/js-yaml-3.4.6.tgz
argparse;1.0.9;js;https://registry.npmjs.org/argparse/-/argparse-1.0.9.tgz
Some (much reduced) sample output npm ls --global --json that I have used for the above example, is as follows:
{
"dependencies": {
"npm": {
"version": "5.5.1",
"dependencies": {
"JSONStream": {
"version": "1.3.1",
"from": "JSONStream#~1.3.1",
"resolved": "https://registry.npmjs.org/JSONStream/-/JSONStream-1.3.1.tgz",
"dependencies": {
"jsonparse": {
"version": "1.3.1",
"from": "jsonparse#^1.2.0",
"resolved": "https://registry.npmjs.org/jsonparse/-/jsonparse-1.3.1.tgz"
},
"through": {
"version": "2.3.8",
"from": "through#>=2.2.7 <3",
"resolved": "https://registry.npmjs.org/through/-/through-2.3.8.tgz"
}
}
}
}
},
"yaml-table": {
"version": "1.1.3",
"from": "yaml-table#latest",
"resolved": "https://registry.npmjs.org/yaml-table/-/yaml-table-1.1.3.tgz",
"dependencies": {
"js-yaml": {
"version": "3.4.6",
"from": "js-yaml#3.4.6",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.4.6.tgz",
"dependencies": {
"argparse": {
"version": "1.0.9",
"from": "argparse#>=1.0.2 <2.0.0",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.9.tgz"
}
}
}
}
}
}
}
By using .., it will recurse through all values in the json tree. So you'll want to filter those out by objects that have the structure you're expecting. In this case, things that have a valid dependencies object. Once you've located the objects, you could extract the values you want.
jq -r '.. | .dependencies? | objects
| to_entries[] | [.key, .value.version, "js", .value.resolved] | join(";")' input.json
produces the results:
npm;5.5.1;js;
yaml-table;1.1.3;js;https://registry.npmjs.org/yaml-table/-/yaml-table-1.1.3.tgz
JSONStream;1.3.1;js;https://registry.npmjs.org/JSONStream/-/JSONStream-1.3.1.tgz
jsonparse;1.3.1;js;https://registry.npmjs.org/jsonparse/-/jsonparse-1.3.1.tgz
through;2.3.8;js;https://registry.npmjs.org/through/-/through-2.3.8.tgz
js-yaml;3.4.6;js;https://registry.npmjs.org/js-yaml/-/js-yaml-3.4.6.tgz
argparse;1.0.9;js;https://registry.npmjs.org/argparse/-/argparse-1.0.9.tgz