Parameterized YAML template in Terraform - json

I am about to refactor a couple of code for a business project. Among other tings, converting from JSON to YAML templates is necessary. I use terraform for infrastructure deployment.
I have this JSON template cf_sns.json.tpl file:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"SNSTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"TopicName": "${sns_topic_name}",
"KmsMasterKeyId": "${kms_key_id}",
"DisplayName": "${sns_topic_name}",
"Subscription": [
"${sns_subscription_list}"
]
}
}
},
"Outputs" : {
"SNSTopicARN" : {
"Description": "The SNS Topic Arn",
"Value" : { "Ref" : "SNSTopic" }
}
}
}
This is a main.tf file using this template file:
data "template_file" "this" {
template = "${file("${path.module}/templates/cf_sns.json.tpl")}"
vars = {
kms_key_id = var.kms_key_id
sns_topic_name = var.sns_topic_name
sns_subscription_list = join(",", formatlist("{\"Endpoint\": \"%s\",\"Protocol\": \"%s\"}", var.sns_subscription_email_address_list, "email"))
}
}
I pass ["myemail", "myOtherEmail"] to var.sns_subscription_email_adress_list.
I had to use this approach with a cloudformation resource since Terraform does not support the email protocol for a sns subspription.
How can I refactor the cf_sns.json.tpl to a YAML file together with the data resource mentioned above in the main.tf file? Particularly, I have no clue how to properly pass the sns_subscription_list as YAML array.

That cf_sns.json.tpl is AWS CloudFormation code, if you are already using terraform just refactor that all the way, not just convert from JSON to YAML but completely get rid of that and use the proper terraform resources:
https://www.terraform.io/docs/providers/aws/r/sns_topic.html
https://www.terraform.io/docs/providers/aws/r/sns_topic_subscription.html
Here is some sample code:
resource "aws_sns_topic" "SNSTopic" {
name = var.sns_topic_name
kms_master_key_id = var.kms_key_id
display_name = var.sns_topic_name
}
output "SNSTopicARN" {
value = aws_sns_topic.SNSTopic.arn
}

Related

Terraform and tagging AWS resources from an external json file

If I had a json file like this:
{
"allMyTags": {
"owner": "john",
"department": "HR",
"city": "New York"
}
}
and my AWS provider terraform main.tf looks like this:
resource "aws_vpc" "example" {
# ... other configuration ...
tags = {
owner = "john"
}
}
How do I go about replacing everything that is in the tags section of main.tf with the external json file. The json file is a lot longer that I have put up there and I just didn't want to manually put in 20 values in the tags section of main.tf. Is there a way to "loop" thru the json file and add it in? Thanks for any help you can provide.
Assuming that you json is already loaded into TF, you could do:
resource "aws_vpc" "example" {
# ... other configuration ...
tags = jsondecode(local.myjson["allMyTags"])
}
where local.myjson is the loaded json to TF.

How to convert string to number in terraform template file

I have a terraform template file source.tpl - it's a json and it has to be JSON, because it's produced by python json library. This file has the following entry
[
{
"data": {
"address": "${NETWORK}",
"netmask": "${NETMASK}",
}
}
]
In my tf module, I render this template:
data "template_file" "source" {
template = "${file("${path.module}/source.tpl")}"
vars = {
NETWORK = element(split("/", "${var.cidr}"),0)
NETMASK = tonumber(element(split("/", "${var.cidr}"),1))
}
}
where cidr is a string - something like 10.1.1.0/24
In the rendered output I need NETMASK to be a number and NETWORK to be a string. I.e. it has to be something like:
data = {
address = "10.1.1.0"
netmask = 24
}
But I'm getting:
data = {
address = "10.1.1.0"
netmask = "24"
}
I.e. netmask is a string. How can I get rid of those quotes in terraform? Initial source.tpl should still have those quotes, because if I remove them - it becomes invalid JSON.
I understand the problem here, you're generating the template using a JSON library that cannot produce something like the following since it's invalid JSON, though this is what you want for the template to be
[
{
"data": {
"address": "${NETWORK}",
"netmask": ${NETMASK}
}
}
]
Might I recommend a little bit of preprocessing? For example
template = "${replace(file("${path.module}/source.tpl"), "\"$${NETMASK}\"", "$${NETMASK}")}"

html-to-xlsx jsreport recipe with initial xlsx template file - HTTP API request

I try to generate xlsx-to-html recipe report with JsReport HTTP API https://jsreport.net/learn/api and initial attached xlsx template.
The documentatio is missing how to build request body to pass additional xlsx file like in bleow examples:
html-to-xlsx recipe example: https://playground.jsreport.net/w/admin/QiHIBqsq
xlsx recipe example: https://playground.jsreport.net/w/anon/Hy_V2BSh-4
I tried to add in template object following object:
template: {
...
engine: "handlebars",
recipe: "html-to-xlsx",
xlsxTemplate: {
content: "base64_string_of_template_xlsx_file_........."
}
...
}
but with no success.
How should be constructed JSON body for this request?
I managed to find proper json that is consumed properly by JsReport:
{
"Template": {
"Content": "template def string..",
"Recipe": "html-to-xlsx"
"Engine": "handlebars",
"BaseXlsxTemplate": {
"Content": "base-template-string-encoded-as-base64",
},
"HtmlToXlsx": {
"InsertToXlsxTemplate": true,
"WaitForJS" = false,
"HtmlEngine" = "chrome"
}
},
"Data": "Json data...",
"Options": {...}
}

AWS CloudFormation: "Parameter [subnetIds] is invalid"

I have a AWS CodePipeline to deploy a stack in CloudFormation using a YAML template as well as a template configuration JSON file.
Relevant Template Snippet:
AWSTemplateFormatVersion: '2010-09-09'
...
Parameters:
subnetIds:
Type: List<AWS::EC2::Subnet::Id>
...
Relevant Configuration File Snippet:
{
"Parameters": {
...
"subnetIds": [
"subnet-a",
"subnet-b",
"subnet-c"
]
},
...
}
For some reason the Deploy stage (CloudFormation) keeps failing with Parameter [subnetIds] is invalid, so my question is how do I pass a list of subnetIds to the template from the configuration file?
It is explained here in the docs about list data types, such as:
List<AWS::EC2::Subnet::Id>
An array of subnet IDs, such as subnet-123a351e, subnet-456b351e.
That is to say, all List types in CloudFormation are also comma-separated strings.
Since you are using a CodePipeline Template Configuration File, you will have something like:
{
"Parameters": {
"subnetIds": "subnet-a,subnet-b,subnet-c"
}
}

How to use custom_data parameter in ARM template in Terraform?

I have an Azure ARM template that successfully bootstraps a VM from a file directory within an Azure Storage Account. I would like to get this working in Terraform, but I am really struggling getting it to work correctly.
Here is a working Azure ARM template that creates the VM and bootstraps it with files in an Azure storage account. The bootstrapping occurs by using the customData parameter.
"variables": {
"uniqueId": "[uniqueString(resourceGroup().id)]",
"customData": "[concat('storage-account=', parameters('STORAGE_ACCOUNT'), ',access-key=', parameters('ACCESS_KEY'), ',file-share=', parameters('FILE_SHARE'), ',share-directory=', parameters('SHARE_DIRECTORY'))]"
},
"resources": [
{
"apiVersion": "2016-04-30-preview",
"type": "Microsoft.Compute/virtualMachines",
"name": "MY-VM",
"location": "[resourceGroup().location]",
"properties": {
"hardwareProfile": {
"vmSize": "Standard_DS3_v2"
},
"osProfile": {
"computerName": "My-Computer-Name",
"adminUsername": "[parameters('Username')]",
"adminPassword": "[parameters('Password')]",
"customData": "[base64(variables('customData'))]"
}
}
}
Here is my non-working Terraform script that does not work when I try to do the same type of Bootstrapping.
resource "azurerm_virtual_machine" "MY-VM" {
name = "${var.vm_name}"
location = "${var.location}"
resource_group_name = "${azurerm_resource_group.rg.name}"
vm_size = "${var.vm_size}"
primary_network_interface_id = "${azurerm_network_interface.nic0.id}"
os_profile {
computer_name = "${var.vm_name}"
admin_username = "${var.adminuser}"
admin_password = "${var.adminuserpassword}"
custom_data = "${base64encode(join("", list("storage-account=", var.STORAGE_ACCOUNT, ",access-key=", var.ACCESS_KEY, ",file-share=", var.FILE_SHARE, ",share-directory=None")))}"
}
}
This is the error that I receive when I run it. If I do not use the custom_data field, the machine launches fine, but is not bootstrapped. I am out of ideas here..
azurerm_virtual_machine.MY-VM:
compute.VirtualMachinesClient#CreateOrUpdate: Failure sending
request: StatusCode=0 -- Original Error: autorest/azure: Service
returned an error. Status=400 Code="InvalidRequestFormat"
Message="Cannot parse the request." Details=[]
i dont think join works for strings? for your case you can just do
"storage-account=${var.STORAGE_ACCCOUNT},access-key=${var.ACCESS_KEY},file-share=${var.FILE_SHARE},share-directory=None"