powershell json format variable in tmsl script - json

I use powershell and i want to put variable in json format into TMSL create role script. I want to put this variable into TablePermission parameter
json variable:
$filter=
[
{
"name": "DimGeography",
"filterExpression": "DimGeography[CountryRegionCode] = \"US\" "
}
]
TMSL script:
$Query='
{
"create": {
"parentObject": {
"database": "AW Internet Sales"
},
"role": {
"name": "test_filter",
"modelPermission": "read",
"tablePermissions": "'+$filter+'"
}
}
}'
This is script which I want to parametrize
{
"createOrReplace": {
"object": {
"database": "AW Internet Sales",
"role": "SalesManagerUS"
},
"role": {
"name": "SalesManagerUS",
"modelPermission": "read",
"tablePermissions": [
{
"name": "DimGeography",
"filterExpression": "DimGeography[CountryRegionCode] = \"US\" "
}
]
}
}
}
How can I put $filter to $Query to receive working script like above ?

You could try the below snippet :
$filter=
'[
{
"name": "DimGeography",
"filterExpression": "DimGeography[CountryRegionCode] = \"US\" "
}
]'
function populate-tsml ($filter)
{
$Query='
{
"create": {
"parentObject": {
"database": "AW Internet Sales"
},
"role": {
"name": "test_filter",
"modelPermission": "read",
"tablePermissions": '+$filter+'
}
}
}'
return $Query
}
$Query = populate-tsml -filter $filter
OUTPUT :

Thank you!
It wasn't necesseary to define a function, a little changes in quotation marks was enough.
Script below works:
$filter=
'[
{
"name": "DimGeography",
"filterExpression": "DimGeography[CountryRegionCode] = \"US\" "
}
]'
$Query='
{
"create": {
"parentObject": {
"database": "AW Internet Sales"
},
"role": {
"name": "test_filter",
"modelPermission": "read",
"tablePermissions": '+$filter+'
}
}
}'

Related

Recursively replace value with $path in JQ

Assuming that i have a complex JsonObject
{
"parent": {
"name": "value",
"child": {
"child_value1": "value",
"child_value2": "value",
"child_value3": ["value1","value2"],
"child_value4": {
"child_child_value1":"value"
}
}
}
}
I want to replace the value of each key, with the name of key prefixed with $
{
"parent": {
"name": "$name",
"child": {
"child_value1": "$child_child_value1",
"child_value2": "$child_child_value2",
"child_value3": ["$child_child_value3_0","$child_child_value3_1"],
"child_value4": {
"child_child_value1":"$child_child_value4_child_child_value1"
}
}
}
}
Is there a way to do it recursively?
EDIT
This is the currently configuration file that I am using
{
"apis": {
"order": {
"base_url": "$mapping_base_url"
},
"payment": {
"base_url": "$admin_base_url"
}
},
"features": {
"authentication": {
"authProviders": true,
"registration": false
}
},
"availableLocales":["en","es"]
}
This is the result using the recomended jq expression:
. |= reduce paths(strings) as $p (.; setpath($p; "$" + ($p | join("_"))))
{
"apis": {
"order": {
"base_url": "$apis_order_base_url"
},
"payment": {
"base_url": "$apis_payment_base_url"
}
},
"features": {
"authentication": {
"authProviders": true,
"registration": false
}
},
"availableLocales": [
"$availableLocales_0",
"$availableLocales_1"
]
}
You're looking for something like this:
.parent |=
reduce paths(strings) as $p (.;
setpath($p; "$" + ($p | join("_")))
)
Online demo

jq command to add onto a map

Is there a command to use jq to add onto this type of map?
append an array of maps using keys (ie, vm1, vm2, vm3)
Note: I have an existing vm_map {} in a json file and i want to add to the vm_map
this is my new_json.json file
{
"gcs_config": [
{
"bucket_name": "somebucket",
"bucket_readers": [],
"bucket_writers": []
}
],
"label_application": "someapp",
"label_environment": "dev",
"lits_vm_zone": "somezone",
"project_id": "someproject",
"region": "someregion",
"storage_bucket_required": true,
"vm_map" : {}
}
expected: using jq to add onto vm_maps map. I will have an empty vm_map and each time it runs, i will add a x amount of new entries.
{
"gcs_config": [
{
"bucket_name": "somebucket",
"bucket_readers": [],
"bucket_writers": []
}
],
"label_application": "someapp",
"label_environment": "dev",
"lits_vm_zone": "zone-a",
"project_id": "someproject",
"region": "someregion",
"storage_bucket_required": true,
"vm_map": {
"vm1": {
"host": "vm1",
"network": "10.1.1.1",
"name": "vm1"
},
"vm2": {
"host": "123",
"network": "10.1.12",
"name": "vm2"
}
}
}
The file you describe is not valid JSON. I'm assuming you mean
{
"vm_map": {
"vm1": {
"host": "vm1",
"network": "xxxxx",
"name": "xxxxxxx"
},
"vm2": {
"host": "vm2",
"network": "xxxxx",
"name": "xxxxxxx"
}
}
}
You can use this:
jq \
--arg VMHOST "$VMHOST" \
--arg NETWORK_IP "$NETWORK_IP" \
--arg VM_NAME "$VM_NAME" \
'
.vm_map[ $VMHOST ] = {
host: $VMHOST,
network: $NETWORK_IP,
name: $VM_NAME
}
'

Powershell exporting data from nested JSON

I need help with looping through the following JSON file and pull in all the attributes under "snapshot_groupSnapshotChildren""entitySnapshot_properties" For example the "Test", "Guacamole" and the "Heartbeat on Guacamole" are dynamic and the depth can also vary from one subgroup to another subgroup.
JSON File
{
"snapshot_groupSnapshotChildren": {
"Test": {
"entitySnapshot_properties": {
"_dependsCondition": "good",
"_nextID": "2",
"_name": "Test",
"_externalId": "baa97724-9ff8-46ad-b23a-d37d283905d7",
"objcategory": "",
"_id": "1951498570",
"_class": "SubGroup",
"_group": "1951498570",
"_enabled": "true",
"_ownerID": "__SiteScopeRoot__"
},
"snapshot_groupSnapshotChildren": {
"Guacamole": {
"entitySnapshot_properties": {
"_dependsCondition": "good",
"_nextID": "2",
"_name": "Guacamole",
"_externalId": "f08b1069-1943-479d-bb83-7668d833fa58",
"objcategory": "",
"_id": "1951498571",
"_class": "SubGroup",
"_group": "1951498571",
"_enabled": "true",
"_ownerID": "1951498570"
},
"snapshot_groupSnapshotChildren": {},
"entitySnapshot_name": "Guacamole",
"snapshot_alertSnapshotChildren": {},
"entitySnapshot_url": "",
"snapshot_monitorSnapshotChildren": {
"Heartbeat on Guacamole": {
"entitySnapshot_properties": {
"_prioritySelection": "MEASURMENT",
"_prevKeyAttrMap": "-84.-19.0.5.115.114.0.19.106.97.118.97.46.117.116.105.108.46.65.114.114.97.121.76.105.115.116.120.-127.-46.29.-103.-57.97.-99.3.0.1.73.0.4.115.105.122.101.120.112.0.0.0.0.119.4.0.0.0.0.120.",
"_name": "Heartbeat on Guacamole",
"_frequency": "600",
"_externalId": "80c24825-c540-4e9d-8203-df83333e0a55",
"_reportTopology": "true",
"_eventPreferenceId": "CommonEventInstancePreferences_default",
"_ownerID": "1951498571"
},
"monitor_snapshot_hostName": "<hostname>",
"monitor_snapshot_fullyQualifiedTarget": "<hostFQDN>",
"entitySnapshot_name": "Heartbeat on Guacamole",
"snapshot_alertSnapshotChildren": {},
"entitySnapshot_url": ""
}
}
}
},
"entitySnapshot_name": "Test",
"snapshot_alertSnapshotChildren": {},
"entitySnapshot_url": "",
"snapshot_monitorSnapshotChildren": {}
}
},
"entitySnapshot_name": "SiteScopeRoot",
"snapshot_alertSnapshotChildren": {},
"snapshot_preferenceSnapShot": {},
"entitySnapshot_url": "",
"snapshot_monitorSnapshotChildren": {}
}
Here is my draft code for looping through, which returns an error that "snapshot_groupSnapshotChildren" dosent exist.
$info = Get-Content -Path .\input.json
If($info -eq $null){
Write-Host "Unhandled Exception parsing data for $sitescopeFQDNhost`nExit Script" -ForegroundColor Red -BackgroundColor Black
Exit
}
else{
$ResJsonObj = $info | ConvertFrom-Json #-Depth 99
foreach ($t in $ResJsonObj.PSObject.Properties)
{
Write-Host $t.name
Write-Host $t.value
}
}
Let me know how i can loop through the nested json file to strip out the atributes values under each "entitySnapshot_properties", I a intrested to fetch the following attribute values
Test -->_name, _class
2.Guacamole -->_name,_ownerid
Heartbeat on Guacamole --> _name,monitor_snapshot_hostName

How to add a json object to multiple documents in a Elastic index using _update_by_query?

I need to update several documents in my Elasticsearch index and I tried the following using the the _update_by_query plugin.
What I need to do is to add a new field to several existing documents matching a certain condition. The new field is a nested JSON. So after adding it document source should look like
_source: {
...existing fields,
"new_field" : {
"attrName1" : "value",
"attrName2" : "value",
}
}
I tried using the _update_by_query API to get this done. But so far I only could add String fields and arrays with it. When trying to add a JSON with the following query it gives me an error.
Query
curl -XPOST "http://xxx.xxx.xxx.xxx:pppp/my_index_name/_update_by_query" -d'
{
"query": {
"bool": {
"must": [
{
"term": {
"team.keyword": "search_phrase"
}
}
]
}
},
"script" : {
"inline":"ctx._source.field_name = {\"a\":\"b\"}"
}
}'
Error
{
"error": {
"root_cause": [
{
"type": "script_exception",
"reason": "compile error",
"script_stack": [
"ctx._source.field_name = {\"a\":\"b\"}",
" ^---- HERE"
],
"script": "ctx._source.field_name = {\"a\":\"b\"}",
"lang": "painless"
}
],
"type": "script_exception",
"reason": "compile error",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "invalid sequence of tokens near ['{'].",
"caused_by": {
"type": "no_viable_alt_exception",
"reason": null
}
},
"script_stack": [
"ctx._source.field_name = {\"a\":\"b\"}",
" ^---- HERE"
],
"script": "ctx._source.field_name = {\"a\":\"b\"}",
"lang": "painless"
},
"status": 500
}
So far I could only add Strings as a new field. What is the correct way to achieve this?
Instead of direct assignment, use params to achieve the same.
{
"query": {
"bool": {
"must": [
{
"term": {
"team.keyword": "search_phrase"
}
}
]
}
},
"script": {
"inline": "ctx._source.field_name = params.new_field",
"params": {
"new_field": {
"a": "b"
}
}
}
}

ARM Template DSC: configuration does not 'see' protectedSettings.configurationArguments

I need to protect a DSC configuration parameter ([pscredential]RegistrationKey), so I have put it under "settings.protectedSettings.configurationData" thus:
"protectedSettings": {
"configurationArguments": {
"RegistrationKey": {
"UserName": "PLACEHOLDER_DONOTUSE",
"Password": "[parameters('dscAutomationRegistrationKey')]"
}
},
"configurationUrlSasToken": "[parameters('artifactsLocationSasToken')]"
}
I get the error:
"VM has reported a failure when processing extension 'Microsoft.Powershell.DSC'. Error message: \"The DSC Extension failed to execute: Mandatory
parameter RegistrationKey is missing.
If I move RegistrationKey out of "settings.protectedSettings.configurationArguments", into "settings.configurationArguments", it works, therefore, I assume there is nothing wrong with the syntax, so I believe it is to do with PsDscAllowPlainTextPassword = $true that wasn't included in the DSC configuration.
(I tried to include the configuration block in the PS1 file, but this threw an error, suggesting this can't be done)
I have now written a configurationdata .psd1 file, containing the following:
$ConfigData = #{
AllNodes = #(
#{
NodeName = "*"
PsDscAllowPlainTextPassword = $true
}
)
}
and referenced it in settings.configurationdata.url.
This now results in the same error as before: VM has reported a failure...
The ARM template is called from PowerShell:
$oAutomationAccount = Get-AzureRmAutomationAccount -ResourceGroupName $AAresourceGroupName -Name $AutomationAccountName
$RegistrationInfo = $oAutomationAccount | Get-AzureRmAutomationRegistrationInfo
$DscRegKeyString = $RegistrationInfo.PrimaryKey
$ssDscAutomationRegistrationKey = (ConvertTo-SecureString -string $DscRegKeyString -AsPlainText -Force)
#Automation Account EndPoint Uri
$DscRegistrationUrl = $RegistrationInfo.Endpoint
$params = #{
artifactsLocationSasToken = $TemplateSas
vmName = "XYZ"
dscAutomationRegistrationKey = $ssDscAutomationRegistrationKey
dscAutomationRegistrationUrl = $DscRegistrationUrl
dscNodeConfigurationName = "CreateAFolder.localhost"
dscTimeStamp = (Get-Date -f "MM/dd/yyyy H:mm:ss tt") #"MM/dd/yyyy H:mm:ss tt"
dscResourceUrl = $DscResourceUrl
dscConfigurationUrl = $DscConfigurationUrl
dscResourceScript = $DscResourceScriptName
dscResourceFunction = "ConfigureLCMforAAPull"
#sequenceId = $sequenceId
}
New-AzureRmResourceGroupDeployment #params `
-Name "$TemplateInstance-$branch" `
-ResourceGroupName $DeploymentResourceGroup.ResourceGroupName `
-Mode Incremental `
-DeploymentDebugLogLevel All `
-TemplateUri $TemplateUri `
-Verbose
Where I believe the parameters are being passed as the correct types.
What am I doing wrong?
Reference template: https://github.com/Azure/azure-quickstart-templates/blob/master/dsc-extension-azure-automation-pullserver/azuredeploy.json
Updated to use a newer DSC schema:https://blogs.msdn.microsoft.com/powershell/2016/02/26/arm-dsc-extension-settings/
this is the template I've been using for node onboarding:
{
"name": "xxx",
"type": "Microsoft.Compute/virtualMachines/extensions",
"location": "[parameters('location')]",
"apiVersion": "2015-06-15",
"dependsOn": [
"xxx"
],
"properties": {
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.22",
"autoUpgradeMinorVersion": false,
"protectedSettings": {
"Items": {
"registrationKeyPrivate": "[parameters('registrationData')]"
}
},
"settings": {
"ModulesUrl": "https://github.com/Azure/azure-quickstart-templates/raw/master/dsc-extension-azure-automation-pullserver/UpdateLCMforAAPull.zip",
"SasToken": "",
"ConfigurationFunction": "UpdateLCMforAAPull.ps1\\ConfigureLCMforAAPull",
"Properties": [
{
"Name": "RegistrationKey",
"Value": {
"UserName": "PLACEHOLDER_DONOTUSE",
"Password": "PrivateSettingsRef:registrationKeyPrivate"
},
"TypeName": "System.Management.Automation.PSCredential"
},
{
"Name": "RegistrationUrl",
"Value": "xxx",
"TypeName": "System.String"
},
{
"Name": "NodeConfigurationName",
"Value": "xxx",
"TypeName": "System.String"
},
{
"Name": "ConfigurationMode",
"Value": "ApplyAndMonitor",
"TypeName": "System.String"
},
{
"Name": "ConfigurationModeFrequencyMins",
"Value": 15,
"TypeName": "System.Int32"
},
{
"Name": "RefreshFrequencyMins",
"Value": 30,
"TypeName": "System.Int32"
},
{
"Name": "RebootNodeIfNeeded",
"Value": true,
"TypeName": "System.Boolean"
},
{
"Name": "ActionAfterReboot",
"Value": "ContinueConfiguration",
"TypeName": "System.String"
},
{
"Name": "AllowModuleOverwrite",
"Value": true,
"TypeName": "System.Boolean"
},
{
"Name": "Timestamp",
"Value": "MM/dd/yyyy H:mm:ss tt",
"TypeName": "System.String"
}
]
}
}
}
I know its using an old format, but that works so, meh.