Terraform dynamic json content - json

Environment : Terraform 0.12.X
If variable var.cloudfront_distributionId is not empty then this should add cloudfront policy
else should not
locals {
codebuild_policy = {
Version = "2012-10-17"
Statement = [
{
Effect = "Allow",
Resource = [
"*"
],
Action = [
"ec2:*"
]
},
{
#count = length(var.cloudfront_distributionId) != 0 ? 1 : 0
effect = "Allow",
Action = [
"cloudfront:ListInvalidations",
"cloudfront:GetInvalidation",
"cloudfront:CreateInvalidation"
],
Resource = [
for cf_id in var.cloudfront_distributionId:
"arn:aws:cloudfront::xxxxxxxx:distribution/${cf_id}"
]
}
]
}
}
resource "aws_iam_role_policy" "codebuild" {
count = local.env_build_resource.count
name = "${var.namespace}-${var.environment}-${var.service_name}"
role = element(aws_iam_role.codebuild.*.name, 0)
policy = jsonencode(local.codebuild_policy)
}
Example 1
cloudfront_distributionId = ["1", "2"]
then
Resource = ["1", "2"]. which is working
Example 2
cloudfront_distributionId = [] is empty
It is creating, werein it should not get created.
+ {
+ Action = [
+ "cloudfront:ListInvalidations",
+ "cloudfront:GetInvalidation",
+ "cloudfront:CreateInvalidation",
]
+ Resource = []
+ count = 0
+ effect = "Allow"
},
How can I achieve this ?

There are some parts missing in your code, thus its difficult to verify the answer form me, but I think the following could be attempted:
resource "aws_iam_role_policy" "codebuild" {
# remaining attributes
policy = length(var.cloudfront_distributionId) > 0 ? jsonencode(local.codebuild_policy): null
}

This is not really a JSON-specific question but rather a general question about conditionally including items in a list, because from Terraform's perspective that Statement attribute of your object is just a normal list value.
One way to think of this is as the concatenation of two lists, one of which might be empty. We can concatenate lists using the concat function, and then use normal conditional expressions to decide the length of the second list to concatenate:
locals {
codebuild_policy = {
Version = "2012-10-17"
Statement = concat(
[
{
Effect = "Allow",
Resource = [
"*"
],
Action = [
"ec2:*"
]
},
],
length(var.cloudfront_distributionId) != 0 ? [
{
Effect = "Allow",
Action = [
"cloudfront:ListInvalidations",
"cloudfront:GetInvalidation",
"cloudfront:CreateInvalidation"
],
Resource = [
for cf_id in var.cloudfront_distributionId :
"arn:aws:cloudfront::xxxxxxxx:distribution/${cf_id}"
]
}
] : [],
)
}
}

Related

Accessing the first object in a tuple using terraform

I am trying to access the first key on a given tuple. The key's name is dynamic and may change so it cannot be accessed using a static value and has to be done either through a for loop or through the use of terraform fuctions.
I've created a small local resource that outputs the following section
skip_empty_mails = { for key, value in local.j.settings.tasks : key => value.email_notifications if value.email_notifications != {} }
The output sent back is
{
"3" = {
"on_start" = [
"foo1#aligntech.com",
"foo2#aligntech.com",
"foo3#aligntech.com",
"foo4#aligntech.com",
]
}
"4" = {
"no_alert_for_skipped_runs" = false
"on_start" = [
"foo21#aligntech.com",
"foo22#aligntech.com",
"foo23#aligntech.com",
"foo24#aligntech.com",
]
]
"on_start" = [
"foo21#aligntech.com",
"foo22#aligntech.com",
"foo23#aligntech.com",
"foo24#aligntech.com",
]
"on_success" = [
"foo21#aligntech.com",
"foo22#aligntech.com",
"foo23#aligntech.com",
"foo24#aligntech.com",
]
}
}
As seen above the key that holds all the values needs to be accessed in a way that will give me the ability to attach it to a string and use string.on_start to pull its values.
The issue is that our key's name is dynamic and may vary.
I've tried following the terraform function documentation But haven't found anything that might be of use in this case.
You may be able to replicate using the following code
locals {
json = {
"3" = {
"on_start" = [
"foo1#aligntech.com",
"foo2#aligntech.com",
"foo3#aligntech.com",
"foo4#aligntech.com",
]
},
"4" = {
"no_alert_for_skipped_runs" = false
"on_failure" = [
"foo21#aligntech.com",
"foo22#aligntech.com",
"foo23#aligntech.com",
"foo24#aligntech.com",
]
"on_start" = [
"foo21#aligntech.com",
"foo22#aligntech.com",
"foo23#aligntech.com",
"foo24#aligntech.com",
]
"on_success" = [
"foo1#foo.com",
"foo2#foo.com",
"foo3#foo.com",
"foo4#foo.com",
]
}
}
}
You can try with a combination of values, element or flatten see the documentation:
https://developer.hashicorp.com/terraform/language/functions/values
https://developer.hashicorp.com/terraform/language/functions/element
https://developer.hashicorp.com/terraform/language/functions/flatten
Below are samples:
First key extraction
locals {
json = {
"4" = {
"no_alert_for_skipped_runs" = false
"on_failure" = [
"foo1#foo.com",
"foo2#foo.com",
]
"on_start" = [
"foo1#foo.com",
"foo2#foo.com",
]
}
}
}
output "data" {
value = element(values(local.json), 1).on_start
}
the Terraform plan will be:
Changes to Outputs:
+ data = [
+ "foo1#foo.com",
+ "foo2#foo.com",
]
Extract and combine on_start item from all
locals {
json = {
"3" = {
"on_start" = [
"foo1#aligntech.com",
"foo2#aligntech.com",
]
},
"4" = {
"on_start" = [
"foo1#foo.com",
"foo2#foo.com",
]
}
}
}
output "data" {
value = flatten(values(local.json)[*].on_start)
}
the Terraform plan will be:
Changes to Outputs:
+ data = [
+ "foo1#aligntech.com",
+ "foo2#aligntech.com",
+ "foo1#foo.com",
+ "foo2#foo.com",
]

Multiple header rows exporting Google Sheets to JSON

I am trying to export google sheet to JSON formatted text so I can read it into another program. Entries are indexed by multiple headers for the row and columns. I can't share the data as it is not compliant with GDPR, so I have an example below.
E.g.
If I was constructing a table of spells in D&D to determine when they were introduced, I would have the field being School of Magic, the Subfield being the spell down the left column, and then across the header would be indexed by the edition (1 through 5), with sub headed with Base and then name of expansion, each cell is empty or just has a string saying "Yes" if it is present.
Image added to clarify.
Example table structure
This would then return an entry like this when exported
{
"School of Magic":"Necromancy",
"Spell":"Abi-Dalzim's Horrid Wilting",
"Edition":"5th",
"Book":"Elemental Evil Player's Companion"
"Elemental Evil Player's Companion": "Yes"
}
I am using this as a base to export
https://gist.githubusercontent.com/pamelafox/1878143/raw/6c23f71231ce1fa09be2d515f317ffe70e4b19aa/exportjson.js?utm_source=thenewstack&utm_medium=website&utm_campaign=platform
But I am incredibly new to JSON and I can't quite work out how to have multiple headers.
Any help here would be appreciated regarding how to adapt this or even just where to look to solve this sort of problem as I can't find documentation that points me in this direction.
Below is a link to a csv file of a similar table, hopefully I haven't just doxxed myself.
https://docs.google.com/spreadsheets/d/e/2PACX-1vSEHGJgn3x4gpyXfBYqRSoJieiZIoDSbJt_pys_TQM-SzXVJjubJbzOvmUT0cUSRRBYUpkKxPq1IOj_/pub?output=csv
The idea would be that in this example the output would show whenever a given spell was introduced within each edition. So the output would be like:
{
"School of Magic":"Necromancy",
"Spell":"Abi-Dalzim's Horrid Wilting",
"Edition":4,
"Book":"Exp2",
"Exp2": "Yes"
"Edition":5,
"Book":"Elemental Evil Player's Companion",
"Elemental Evil Player's Companion": "Yes"
}
{
"School of Magic":"Necromancy",
"Spell":"Raise Undead",
"Edition":1,
"Book":"Base",
"Base": "Yes",
"Edition":2,
"Book":"Base",
"Base": "Yes",
"Edition":3,
"Book":"Base",
"Base": "Yes",
"Edition":4,
"Book":"Base",
"Base": "Yes",
"Edition":5,
"Book":"Base",
"Base": "Yes"
}
If this makes sense? In the true data these cells contain information of interactions between the subcolumns so it is important that I can identify which sub columns and what the entry is.
Try
function table2json() {
var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
var [headers, ...rows] = sheet.getDataRange().getValues();
var items = []
rows.forEach(function(r) {
var obj={}
r.forEach(function (c, j) {
obj[headers[j]] = c
})
items.push(obj)
})
console.log(JSON.stringify(items))
}
edit
according to your new data and your spreadsheet, try
function myFunction() {
var sh = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet()
var data = sh.getDataRange().getValues()
// rebuild a complete set
var temp = ''
data.forEach((r, i) => {
if (i < 2) {
r.forEach((c, j) => {
if (j > 1) {
if (c == '') { data[i][j] = temp } else { temp = data[i][j] }
}
})
}
else {
if (r[0] == '') { data[i][0] = temp } else { temp = data[i][0] }
}
})
// extract informations
data.forEach((r, i) => {
if (i > 2) {
var result = []
if (r[0] == '') { data[i][0] = temp } else { temp = data[i][0] }
result.push([data[2][0], data[i][0]])
result.push([data[2][1], data[i][1]])
r.forEach((c, j) => {
if (j > 2) {
if (c != '') {
result.push([data[0][j], data[1][j]])
result.push(['Book', data[2][j]])
result.push([data[2][j], data[i][j]])
}
}
})
console.log((result))
}
})
}
result
[ [ 'School', 'Necromancy' ],
[ 'Spell', 'Abi-Dalzim\'s...' ],
[ 'Edition', 4 ],
[ 'Book', 'Exp2' ],
[ 'Exp2', 'Yes' ],
[ 'Edition', 5 ],
[ 'Book', 'Elemental Evil ...' ],
[ 'Elemental Evil ...', 'Yes' ] ]
[ [ 'School', 'Necromancy' ],
[ 'Spell', 'Raise Dead' ],
[ 'Edition', 2 ],
[ 'Book', 'Base' ],
[ 'Base', 'Yes' ],
[ 'Edition', 3 ],
[ 'Book', 'Base' ],
[ 'Base', 'Yes' ],
[ 'Edition', 4 ],
[ 'Book', 'Base' ],
[ 'Base', 'Yes' ],
[ 'Edition', 5 ],
[ 'Book', 'Base' ],
[ 'Base', 'Yes' ] ]

Parsing json in terraform locals

I have another question about JSON parsing. Imagine I have this JSON:
{
"all_dogs" :[
{
"name": "foo",
"groups": ["morning", "evening"]
},
{
"name": "bar",
"groups": ["evening", "saturday"]
},
{
"name": "feet",
"groups": ["afternoon"]
}
]
}
I can extract all the groups like this:
locals {
all_dogs = jsondecode(file("${path.module}/dogs.json"))
all_groups = toset(flatten(local.all_dogs.all_dogs[*].groups))
}
Now, I’m trying to create a MAP. Each group is a key of the map and values of the maps are the different dogs in that group.
So I would like to create a map like this:
afternoon = [feet]
evening= [foo, bar]
morning= [foo]
saturday= [bar]
I'm trying with something like this and I tried several options... But I can't make it work.
output "ex" {
value = flatten([
for group in local.all_groups: [
for dog in local.all_dogs : {
group = group
dog = dog
}
]
]
)
}
Later on, I would like to use that map to provision some resources. Is this eventually possible?
locals {
all_dogs = jsondecode(file("${path.module}/dogs.json"))
groups = flatten([for d in local.all_dogs.all_dogs : [for g in d.groups : { key : g, value : d.name }]])
}
The value for local groups will be a list of tuples and it will look something like this:
groups = [
{
"key" = "morning"
"value" = "foo"
},
{
"key" = "evening"
"value" = "foo"
},
{
"key" = "evening"
"value" = "bar"
},
{
"key" = "saturday"
"value" = "bar"
},
{
"key" = "afternoon"
"value" = "feet"
},
]
Now we have to create a map from this list:
output "my_map" {
value = {
for g in local.groups : g.key => g.value...
}
}
This will produce the following output:
my_map= {
"afternoon" = [
"feet",
]
"evening" = [
"foo",
"bar",
]
"morning" = [
"foo",
]
"saturday" = [
"bar",
]

Reading the JSON hierarchy (Number of subnodes can be any number) file using Powershell

Reading the JSON hierarchy (Number of subnodes can be any number) file using Powershell.
I could do it for one level but need the same for N number of loops and nested loops
Need result in Tree view, if not possible then Tabular way with the title on top of the root table
Below is the script so far I have developed:
$Filepath = 'C:\Users\Learner\Test2.JSON'
$JsonContent=gc $Filepath | ConvertFrom-Json
$RootNodeCount=$JsonContent.psobject.Properties.name.Count
$RootNodeName=$JsonContent.psobject.Properties.name
foreach ($Name in $RootNodeName) {Echo $Name--> $JsonContent.$Name}
JSON File
{
"foundation": {
"network": {
"resource_group_name": "rg-network-e2",
"name": "nonprodvnet-e2"
},
"diagnostics": {
"resource_group_name": "rg-mgmt-cu",
"storage_account_name": "Sreacnt-e2"
},
"log_analytics": {
"resource_group_name": "rg-mgmt-cu",
"workspace_name": "la-sap-e2"
},
"recovery_vault": {
"resource_group_name": "rg-mgmt-cu",
"name": "rsv-sap-e2"
},
"windows_domain": {
"domain_name": "dmn.local",
"ou_path": "CN=Computers,DC=example,DC=com",
"domain_user": "svc_domainjoin#dmn.local",
"domain_password": "Pwd01"
}
},
"control_flags": {
"enable_boot_diagnostics": true,
"enable_oms": false,
"enable_backup": true,
"windows_domain_join": false
},
"deployment": {
"resource_group": {
"name": "rg-sap-501-e2",
"location": "eastus2"
},
"tags": {
"owner": "for SAP"
},
"os_account": {
"admin_username": "locadm"
},
"proximity_placement_groups": {
"ppg-501": {}
},
"availability_sets": {
"avset-cs-501": {
"ppg_name": "ppg-501"
},
"avset-app-501": {
"ppg_name": "ppg-501"
},
"avset-db-501": {
"ppg_name": "ppg-501"
}
},
"load_balancers": {
"ilb-sap-501": {
"dv2ascs": {
"ip_address": "192.0.18.20",
"probe_port": 62000,
"subnet": "app"
},
"dv2-ers": {
"ip_address": "192.0.18.21",
"probe_port": 62102,
"subnet": "app"
}
},
"ilb-db-501": {
"dv2-hana": {
"ip_address": "192.0.17.12",
"probe_port": 62503,
"subnet": "db"
}
}
},
"server_groups": {
"ascs": {
"os_type": "linux",
"sku": "Standard_E4s_v3",
"availability_set": "avset-cs-501",
"backup_policy": "sap",
"ppg_name": "ppg-501",
"lb_refs": [
"dv2-ascs",
"dv2-ers"
],
"os_disk_size": "30",
"enable_accelerated_networking": true,
"hosts": {
"vcs501-01": {
"nics": [
[
"192.0.18.18"
]
]
},
"vcs501-02": {
"nics": [
[
"192.0.18.19"
]
]
}
},
"subnet": "app",
"image_details": {
"resource_id": null,
"marketplace_reference": {
"publisher" : "SUSE",
"offer" : "sles-sap-15-sp1",
"sku" : "gen1",
"version": "latest"
}
},
"disks": [
{
"name": "usrsap",
"disk_size": "128",
"number_of_disks": 1
}
]
},
"apps": {
"type": "app_linux",
"os_type": "linux",
"sku": "Standard_E4s_v3",
"availability_set": "avset-app-501",
"backup_policy": "sap",
"ppg_name": "ppg-501",
"enable_accelerated_networking": true,
"os_disk_size": "30",
"hosts": {
"vas501-01": {
"nics": [
[
"192.0.18.16",
"192.0.18.26"
]
]
},
"vas501-02": {
"nics": [
[
"192.0.18.17",
"192.0.18.27"
]
]
}
},
"subnet": "app",
"image_details": {
"resource_id": null,
"marketplace_reference": {
"publisher" : "SUSE",
"offer" : "sles-15-sp1",
"sku" : "gen1",
"version": "latest"
}
},
"disks": [
{
"name": "usrsap",
"disk_size": "128",
"number_of_disks": 1
}
]
},
"hana": {
"os_type": "linux",
"sku": "Standard_E32s_v3",
"availability_set": "avset-db-501",
"backup_policy": "sap",
"ppg_name": "ppg-501",
"os_disk_size": "30",
"subnet": "db",
"lb_refs": [
"dv2-hana"
],
"hosts": {
"vhs501-01": {
"nics": [
[
"192.0.17.10"
]
]
},
"vhs501-02": {
"nics": [
[
"192.0.17.11"
]
]
}
},
"image_details": {
"resource_id": null,
"marketplace_reference": {
"publisher" : "SUSE",
"offer" : "sles-sap-15-sp1",
"sku" : "gen1",
"version": "latest"
}
},
"disks": [
{
"name": "usrsap",
"disk_size": "128",
"number_of_disks": 1
},
{
"name": "hanashared",
"disk_size": "128",
"number_of_disks": 1
},
{
"name": "hanadata",
"disk_size": "128",
"number_of_disks": 3
},
{
"name": "hanalog",
"disk_size": "128",
"number_of_disks": 2
},
{
"name": "hanabackup",
"disk_size": "512",
"number_of_disks": 1
}
]
}
}
}
}
You are not really reading a Json file but actually the hierarchy of a PowerShell object which might hold a lot of (.Net) object types but in this case it is limited by the original Json structure which only consists out of three major types:
Arrays, containing multiple child objects
PSCustomObjects (unless you use the ConvertFrom-Json -AsHashTable parameter), containing child objects assigned to a specific name.
Primitives (including strings), which is basically an object ("leaf") without children.
To best way to read through a hierarchic structure of an unknown depth is to use a recursive function, meaning a function that calls itself. To give you an example for your specific question:
Function Show-Object ($Object, $Depth = 0, $Name) {
$Indent = if ($Depth++) { " " * ($Depth - 2) }
if ($Object -is [Array]) {
If ($Name) { "$Indent$Name =" }
foreach ($Item in $Object) {
Show-Object $Item $Depth
}
}
elseif ($Object -is [PSCustomObject]) {
If ($Name) { "$Indent$Name =" }
foreach ($Name in $Object.PSObject.Properties.Name) {
Show-Object $Object.$Name $Depth $Name
}
}
else {
if ($Name) { "$Indent$Name = $Object" } else { "$Indent$Object" }
}
}
As you can see, where the structure might any child objects (in case of an [Array] or an [PSCustomObject]), the Show-Object function is calling itself with a child object as the new input object, and some additional parameters as to e.g. keep track of the current $Depth.
You can call the function simply like:
Show-Object $JsonContent
Or export it to a file like:
Show-Object $JsonContent | Out-File .\text1.txt
For your $JsonContent it will result in:
foundation =
network =
resource_group_name = rg-network-e2
name = nonprodvnet-e2
diagnostics =
resource_group_name = rg-mgmt-cu
storage_account_name = Sreacnt-e2
log_analytics =
resource_group_name = rg-mgmt-cu
workspace_name = la-sap-e2
recovery_vault =
resource_group_name = rg-mgmt-cu
name = rsv-sap-e2
windows_domain =
domain_name = dmn.local
ou_path = CN=Computers,DC=example,DC=com
domain_user = svc_domainjoin#dmn.local
domain_password = Pwd01
control_flags =
enable_boot_diagnostics = True
enable_oms = False
enable_backup = True
windows_domain_join = False
deployment =
resource_group =
name = rg-sap-501-e2
location = eastus2
tags =
owner = for SAP
os_account =
admin_username = locadm
proximity_placement_groups =
ppg-501 =
availability_sets =
avset-cs-501 =
ppg_name = ppg-501
avset-app-501 =
ppg_name = ppg-501
avset-db-501 =
ppg_name = ppg-501
load_balancers =
ilb-sap-501 =
dv2ascs =
ip_address = 192.0.18.20
probe_port = 62000
subnet = app
dv2-ers =
ip_address = 192.0.18.21
probe_port = 62102
subnet = app
ilb-db-501 =
dv2-hana =
ip_address = 192.0.17.12
probe_port = 62503
subnet = db
server_groups =
ascs =
os_type = linux
sku = Standard_E4s_v3
availability_set = avset-cs-501
backup_policy = sap
ppg_name = ppg-501
lb_refs =
dv2-ascs
dv2-ers
os_disk_size = 30
enable_accelerated_networking = True
hosts =
vcs501-01 =
nics =
192.0.18.18
vcs501-02 =
nics =
192.0.18.19
subnet = app
image_details =
resource_id =
marketplace_reference =
publisher = SUSE
offer = sles-sap-15-sp1
sku = gen1
version = latest
disks =
name = usrsap
disk_size = 128
number_of_disks = 1
apps =
type = app_linux
os_type = linux
sku = Standard_E4s_v3
availability_set = avset-app-501
backup_policy = sap
ppg_name = ppg-501
enable_accelerated_networking = True
os_disk_size = 30
hosts =
vas501-01 =
nics =
192.0.18.16
192.0.18.26
vas501-02 =
nics =
192.0.18.17
192.0.18.27
subnet = app
image_details =
resource_id =
marketplace_reference =
publisher = SUSE
offer = sles-15-sp1
sku = gen1
version = latest
disks =
name = usrsap
disk_size = 128
number_of_disks = 1
hana =
os_type = linux
sku = Standard_E32s_v3
availability_set = avset-db-501
backup_policy = sap
ppg_name = ppg-501
os_disk_size = 30
subnet = db
lb_refs =
dv2-hana
hosts =
vhs501-01 =
nics =
192.0.17.10
vhs501-02 =
nics =
192.0.17.11
image_details =
resource_id =
marketplace_reference =
publisher = SUSE
offer = sles-sap-15-sp1
sku = gen1
version = latest
disks =
name = usrsap
disk_size = 128
number_of_disks = 1
name = hanashared
disk_size = 128
number_of_disks = 1
name = hanadata
disk_size = 128
number_of_disks = 3
name = hanalog
disk_size = 128
number_of_disks = 2
name = hanabackup
disk_size = 512
number_of_disks = 1

Updating nested array n1ql

The sample doc, I am trying to work with is:
{
"name": "level1",
"childLevel1": [
{
"name": "level2",
"childLevel2": [
{
"name": "level3",
"childLevel3": [
{
"name": "level4",
"attribute1": "data"
}
]
}
]
}
]
}
What we want to do is, with n1ql add one more array element to "childLevel3". I have tried the reference to https://developer.couchbase.com/documentation/server/4.5/n1ql/n1ql-language-reference/update.html
I was able to add a regular attribute to childLevel2 but after that it was not working. This is the query I have been trying out.
Update sample
set child2.childLevel3 = ARRAY_PUT(child2.childLevel3, {'attribute' : 'data2'})
FOR child2 in child1.childLevel2 when child2.name == "level3" END
for child1 in childLevel1 when childLevel1.name == 'level2' END
WHERE name == 'level1'
But it gives me error on parsing, I tried other ways too but nothing works.
Try
UPDATE sample
SET child2.childLevel3 = ARRAY_PUT(child2.childLevel3, {'attribute' : 'data2'})
FOR child2 in child1.childLevel2
FOR child1 in childLevel1
WHEN childLevel1.name == 'level2' AND child2.name == "level3"
END
WHERE name == 'level1';
I had to update an object within a nested array, recently in Couchbase. The answer above helped me to formulate this query in a bucket called "metadata". The data structure is:
{
"documentMetadata": {
"documentId": "42"
},
"results": [
{
"type": "FileStaging",
"status": "NOT STARTED"
}
],
"type": "runLog"
}
The query that worked for me:
UPDATE metadata
SET res.status = 'QUEUED'
FOR res in results
WHEN res.type = 'FileStaging'
END
WHERE type = 'runLog'
AND documentMetadata.documentId = '42'