I have a AWS CodePipeline to deploy a stack in CloudFormation using a YAML template as well as a template configuration JSON file.
Relevant Template Snippet:
AWSTemplateFormatVersion: '2010-09-09'
...
Parameters:
subnetIds:
Type: List<AWS::EC2::Subnet::Id>
...
Relevant Configuration File Snippet:
{
"Parameters": {
...
"subnetIds": [
"subnet-a",
"subnet-b",
"subnet-c"
]
},
...
}
For some reason the Deploy stage (CloudFormation) keeps failing with Parameter [subnetIds] is invalid, so my question is how do I pass a list of subnetIds to the template from the configuration file?
It is explained here in the docs about list data types, such as:
List<AWS::EC2::Subnet::Id>
An array of subnet IDs, such as subnet-123a351e, subnet-456b351e.
That is to say, all List types in CloudFormation are also comma-separated strings.
Since you are using a CodePipeline Template Configuration File, you will have something like:
{
"Parameters": {
"subnetIds": "subnet-a,subnet-b,subnet-c"
}
}
Related
I have a JSON file that is going to contain a number of different lists per client that I am deploying for. These lists are going to serve as container overrides for an ECS task that my Lambda function will be invoking. The JSON config file would look something like this:
{
"clientName": {
"environment": [
{
"name": "name1",
"value": "value1"
}
]
}
}
And my serverless.yml would look something like this:
environment:
CONTAINER_ENVIRONMENT: ${file(serverlessConfig.json):${env:CLIENT_NAME}.environment}
Which results in the following error:
Could not resolve "CONTAINER_ENVIRONMENT" environment variable: Unsupported environment variable format:
[
{
"name": "name1",
"value": "value1"
}
]
I've tried using CloudFormation intrinsic functions such as Fn::Join and Fn::ToJsonString. These both threw an error when trying to run locally using sls invoke local (the errors were the same as the above). After some digging it seems that these functions aren't compatible with the serverless environment property.
The only thing that has worked so far is storing the list as a string in a .env file, but that's not really ideal since these configs could take a number of different environment objects.
Is there any way to get this to work with the setup that I have going?
I have a Step Functions Machine the definition file of which looks as such:
"ContainerOverrides": [
{
"Name": "Foo",
"Environment": [
{
"Name": "Foo"
"Value": "Bar"
},
],
"Command.$": "States.Array($.Foo,$.Foo,$.Bar,$.Bar,$.Bar)"
}
]
},
that I am trying to rewrite into Typescript (CDK). I've gotten the following few lines.
containerOverrides: [{
containerDefinition: Foo,
environment: [
{ name: 'Foo', value: 'Bar'},
],
command: ['States.Array($.Foo,$.Foo,$.Bar,$.Bar,$.Bar)'],
}],
I'm a bit confused about how to go about this.
When I deploy the above CDK code, I get as output:
"Command": [
"States.Array($.Foo,$.Foo,$.Bar,$.Bar,$.Bar)"
],
My confusion is in regards to the following: The ContainerOverrides method doesn't accept parameters, but I need to modify a parameter (Command.$), so how can I possibly do that? I came across this post where somebody seems to have a similar issue, but when I try to apply the proposed solution, of simply writing
command: JsonPath.arrayAt('States.Array($.Foo,$.Foo,$.Bar,$.Bar,$.Bar)'
I get told that ''Cannot use JsonPath fields in an array, they must be used in objects''
TL;DR The current implementation of EcsRunTask doesn't permit this. The general-purpose CallAwsService construct does.
The EcsRunTask construct is the CDK's implementation of the ECS optimised integration. The construct only accepts an array of strings as override commands. It cannot produce substitutable output like "Command.$": "$.commands" that's needed to read the override command from the execution input. This is a limitation of the CDK implementation, not of the ECS optimized integration itself.
The cleanest solution is to use the CallAwsService construct, which implements the SDK service integration. It requires manual configuration. The API-specific config goes in the parameters prop. The prop is loosely typed as { [string]: any }. It's flexible, but it's your job to provide the expected syntax for the ecs:RunTask SDK call. Here is the relevant bit for your question:
parameters {
Overrides: {
ContainerOverrides: [
{ Command: sfn.JsonPath.array("sh", "-c", sfn.JsonPath.stringAt("$.cmd")), },
],
},
}
It produces the expected command override in the Step Functions task definition:
"Command.$": "States.Array('sh', '-c', $.cmd)"
From this article, it shows how to deploy a AWS::Serverless::Function as a 'proxy' to do routing, a/b testing, etc.
The example template is in YAML, but we need to use JSON as that is what the AWS dotnet templates are built on.
There is a line in the YAML that I cannot figure out how to translate to a JSON template:
LambdaFunctionARN: !Ref LambdaEdgeFunctionSample.Version
In a JSON template a Ref just refers to another resource or parameter. You cannot Ref to a Resource.Property. For that you use Fn::GetAtt.
I have tried Fn::GetAtt, but that errors with "Version is an unknown attribute for this resource" (in the editor) and Template error: every Fn::GetAtt object requires two non-empty parameters, the resource name and the resource attribute during deployment:
"LambdaFunctionARN":{"Fn::GetAtt":["LambdaEdgeFunctionSample","Version"]}}
Mind you, these are both using AWS::Serverless::Function underneath, so there's more transforming going on.
Specifically, when the JSON template is transformed, it does create:
"LambdaEdgeFunctionSampleVersion1cfc342538": {
"Type": "AWS::Lambda::Version",
"DeletionPolicy": "Retain",
"Properties": {
"FunctionName": {
"Ref": "LambdaEdgeFunctionSample"
}
}
},
But I cannot use LambdaEdgeFunctionSampleVersion1cfc342538 as clearly that is a transient id.
How can I accomplish the same as the YAML template in a JSON template?
Apparently, AWS SAM does a transformation on !Ref LambdaEdgeFunctionSample.Version before the template is actually processed.
So, in Json, it's the same and SAM does the transformation:
{"Ref":"LambdaEdgeFunctionSample.Version"}
transforms to
{
"Ref": "LambdaEdgeFunctionSampleVersionf76b18e3ba"
}
If I had a json file like this:
{
"allMyTags": {
"owner": "john",
"department": "HR",
"city": "New York"
}
}
and my AWS provider terraform main.tf looks like this:
resource "aws_vpc" "example" {
# ... other configuration ...
tags = {
owner = "john"
}
}
How do I go about replacing everything that is in the tags section of main.tf with the external json file. The json file is a lot longer that I have put up there and I just didn't want to manually put in 20 values in the tags section of main.tf. Is there a way to "loop" thru the json file and add it in? Thanks for any help you can provide.
Assuming that you json is already loaded into TF, you could do:
resource "aws_vpc" "example" {
# ... other configuration ...
tags = jsondecode(local.myjson["allMyTags"])
}
where local.myjson is the loaded json to TF.
I am about to refactor a couple of code for a business project. Among other tings, converting from JSON to YAML templates is necessary. I use terraform for infrastructure deployment.
I have this JSON template cf_sns.json.tpl file:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"SNSTopic": {
"Type": "AWS::SNS::Topic",
"Properties": {
"TopicName": "${sns_topic_name}",
"KmsMasterKeyId": "${kms_key_id}",
"DisplayName": "${sns_topic_name}",
"Subscription": [
"${sns_subscription_list}"
]
}
}
},
"Outputs" : {
"SNSTopicARN" : {
"Description": "The SNS Topic Arn",
"Value" : { "Ref" : "SNSTopic" }
}
}
}
This is a main.tf file using this template file:
data "template_file" "this" {
template = "${file("${path.module}/templates/cf_sns.json.tpl")}"
vars = {
kms_key_id = var.kms_key_id
sns_topic_name = var.sns_topic_name
sns_subscription_list = join(",", formatlist("{\"Endpoint\": \"%s\",\"Protocol\": \"%s\"}", var.sns_subscription_email_address_list, "email"))
}
}
I pass ["myemail", "myOtherEmail"] to var.sns_subscription_email_adress_list.
I had to use this approach with a cloudformation resource since Terraform does not support the email protocol for a sns subspription.
How can I refactor the cf_sns.json.tpl to a YAML file together with the data resource mentioned above in the main.tf file? Particularly, I have no clue how to properly pass the sns_subscription_list as YAML array.
That cf_sns.json.tpl is AWS CloudFormation code, if you are already using terraform just refactor that all the way, not just convert from JSON to YAML but completely get rid of that and use the proper terraform resources:
https://www.terraform.io/docs/providers/aws/r/sns_topic.html
https://www.terraform.io/docs/providers/aws/r/sns_topic_subscription.html
Here is some sample code:
resource "aws_sns_topic" "SNSTopic" {
name = var.sns_topic_name
kms_master_key_id = var.kms_key_id
display_name = var.sns_topic_name
}
output "SNSTopicARN" {
value = aws_sns_topic.SNSTopic.arn
}