I am following a direction on Hashicorp's site regarding wrapping a CF Template in Terraform. There's a fair amount to the whole code, but the CF Template works, so the issue is with the "wrapping"...
Terraform plan gives me this error output:
terraform plan
Error: aws_cloudformation_stack.Momma: "template_body" contains an invalid JSON: invalid character 'A' looking for beginning of object key string
Error: aws_cloudformation_stack.Momma: : invalid or unknown key: source
So it seems that the "AWSTemplateFormatVersion" line is what it does not like. Hence the'A' it is picking up, I guess.
This is the Hashicorp page I am following, I'm wondering if there are any escape characters that are appropriate or, if anyone can see any immediate formatting issues with my JSON?
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudformation_stack
terraform {}
provider "aws" {
version = "= 2.61"
region = "ap-southeast-2"
}
resource "aws_cloudformation_stack" "Momma" {
source = "../../aws_modules/aws-db-event-subscription"
name = "Momma-Stack"
template_body = <<STACK
{
AWSTemplateFormatVersion: 2010-09-09
Description: Team Server
Metadata:
'AWS::CloudFormation::Interface':
ParameterGroups:
- Label:
default: Deployment Options
Parameters:
- Environment
- KeyPairName
- VPCID
- Subnet1ID
- RemoteAccessCIDR
- Owner
ParameterLabels:
KeyPairName:
Default: Key Pair Name
RemoteAccessCIDR:
Default: External Access CIDR
VPCID:
Default: VPC ID
Owner:
Default: MommaTeam....
Thank you for any guidance offered.
There are at least two issues that are apparent:
source = "../../aws_modules/aws-db-event-subscription" is invalid. There is no attribute called source in aws_cloudformation_stack. You can remove it.
Your template_body should not begin with { in:
template_body = <<STACK
{
This is because you are using YAML for your template, not JSON.
Related
We have a couple of legacy applications we're migrating to ec2 and these use a bunch of application configuration parameters. I need to be able to store each config as an individual parameter per application.
I'm trying the following but clearly not doing it right as it appends all values to a single parameter per application:
locals {
application = {
"application1" = { app_shortcode = "app1"},
"application2" = { app_shortcode = "app2"}
}
resource "aws_ssm_parameter" "application_parameters" {
for_each = local.application
name = each.key
value = jsonencode(file("${path.module}/${each.key}/ssm_param.json"))
}
my app1's ssm_param.json is something like
{
"app1_config1": "config_value_1",
"app1_config2": "config_value_2",
"app1_config3": "config_value_3"
}
and app2's ssm_param.jsonis
{
"app2_config_a": "config_value_a",
"app2_config_b": "config_value_b",
"app2_config_c": "config_value_c"
}
The current output is a single parameter like this for each application:
"{\r\n \"app2_config_a\": \"config_value_a\",\r\n \"app2_config_b\": \"config_value_b\"\r\n, \r\n \"app2_config_c\": \"config_value_c\"\r\n}"
Looking for suggestions please.
I solved this by using a slightly different approach (not quite the same as my initial one but this works for me for now):
used a ssm_params.yaml as below (the project team was kind enough to give me the config settings as yaml output)
parameter:
app1:
name: app1_config1
description: "application config test"
type: "String"
value: "some_randoM_value"
app1:
name: app_config2
description: "another test"
type: "SecureString"
value: "some_random123_value###"
app2:
name: app_config_2
description: "config test"
type: "String"
value: "some_randoM_value_2"
locals {
params = yamldecode(file("${path.module}/ssm_params.yaml"))
}
resource "aws_ssm_parameter" "app_params" {
for_each = local.params.parameter
name = each.value.name
type = each.value.type
value = each.value.value
}
I'm having issues iterating over a list of objects within a template interpreted by the templatefile function.
I have the following var:
variable "destinations" {
description = "A list of EML Channel Destinations."
type = list(object({
id = string
url = string
}))
}
This is passed in to the templatefile function as destinations. The snippet of template relevant is this:
Destinations:
%{ for dest in destinations ~}
- Id: ${dest.id}
Settings:
URL: ${dest.url}
%{ endfor }
When planning Terraform this gives an error of:
Error: "template_body" contains an invalid YAML: yaml: line 26: did not find expected key
I have tried switching the template code to the following:
Destinations:
%{ for id, url in destinations ~}
- Id: ${id}
Settings:
URL: ${url}
%{ endfor }
Which gives a different error:
Call to function "templatefile" failed:
../../local-tfmodules/eml/templates/eml.yaml.tmpl:25,20-23: Invalid template
interpolation value; Cannot include the given value in a string template:
string required., and 2 other diagnostic(s).
[!] something went wrong when creating the environment TF plan
I get the impression my iterating over the data type here is somehow incorrect but I cannot fathom how and I cannot find any docs about this at all.
Here is a cut down example of how I'm calling this module:
module "eml" {
source = "../../local-tfmodules/eml"
name = "my_eml"
destinations = [
{
id = "6"
url = "https://example.com"
},
{
id = "7"
url = "https://example.net"
}
]
<cut>
}
I've just found (after crafting a small Terraform module to test templatefile output only) that the original config DOES work (at least in TF v0.12.29).
The errors given are a bit of a Red Herring - the issue is to do with indentation within the template, e.g. instead of:
Destinations:
%{ for destination in destinations ~}
- Id: ${destination.id}
Settings:
URL: ${destination.url}
%{ endfor ~}
it should be:
Destinations:
%{~ for destination in destinations ~}
- Id: ${destination.id}
Settings:
URL: ${destination.url}
%{~ endfor ~}
Notice the extra tilde's (~) at the beginning of the Terraform directives. This makes the Yaml alignment work correctly (you get some lines incorrectly indented and some blank lines). After this the original code in my question works as I expected it to & produces valid yaml.
You can't pass var.destinations as a list of maps to the template. It must be list/set of strings.
But you could do the following:
templatefile("eml.yaml.tmpl",
{
ids = [for v in var.destinations: v.id]
urls = [for v in var.destinations: v.url]
}
)
where eml.yaml.tmpl is
Destinations:
%{ for id, url in zipmap(ids, urls) ~}
- Id: ${id}
Settings:
URL: ${url}
%{ endfor ~}
Since you are aiming to generate a YAML result, I suggest following the advice in the templatefile documentation about generating JSON or YAML from a template.
Using the yamlencode function will guarantee that the result is always valid YAML, without you having to worry about correctly positioning newlines or quoting/escaping strings that might contain special characters.
Write your templatefile call like this:
templatefile("${path.module}/templates/eml.yaml.tmpl", {
destinations = var.destinations
})
Then, in the eml.yaml.tmpl, make the entire template be the result of calling yamlencode, like this:
${yamlencode({
Destinations = [
for dest in destinations : {
Id = dest.id
Settings = {
URL = dest.url
}
}
]
})
Notice that the argument to yamlencode is Terraform expression syntax rather than YAML syntax, because in this case it's Terraform's responsibility to do the YAML encoding, and all you need to do is provide a suitable value for Terraform to encode, following the mappings from Terraform types to YAML types given in the yamldecode documentation.
In Artillery, how can I capture the attribute of a random index in a JSON array returned from a GET, so my subsequent POSTs are evenly distributed across the resources?
https://artillery.io/docs/http-reference/#extracting-and-reusing-parts-of-a-response-request-chaining
I'm using serverless artillery to run a load test, which under the hood uses artillery.io .
A lot of my scenarios look like this:
-
get:
url: "/resource"
capture:
json: "$[0].id"
as: "resource_id"
-
post:
url: "/resource/{{ resource_id }}/subresource"
json:
body: "Example"
Get a list of resources, and then POST to one of those resources.
As you can see, I am using capture to capture an ID from the JSON response. My problem is that it is always getting the id from the first index of the array.
This will mean in my load test I end up absolutely battering one single resource rather than hitting them evenly which will be a more likely scenario.
I would like to be able to do something like:
capture:
json: "$[RANDOM].id
as: "resource_id"
but I have been unable to find anything in the JSONPath definition that would allow me to do so.
Define setResourceId function in custom JS code and to tell Artillery to load your custom code, set config.processor to the JS file path:
processor: "./custom-code.js"
- get:
url: "/resource"
capture:
json: "$"
as: "resources"
- function: "setResourceId"
- post:
url: "/resource/{{ resourceId }}/subresource"
json:
body: "Example"
custom-code.js file containing the below function
function setResourceId(context, next) {
const randomIndex = Math.round(Math.random() * context.vars.resources.length);
context.vars.resourceId = context.vars.resources[randomIndex].id;
}
Using this version:
------------ Version Info ------------
Artillery: 1.7.9
Artillery Pro: not installed (https://artillery.io/pro)
Node.js: v14.6.0
OS: darwin/x64
The answer above didn't work for me.
I got more info from here, and got it working with the following changes:
function setResourceId(context, events, done) {
const randomIndex = Math.round(Math.random() * (context.vars.resources.length - 1));
context.vars.resourceId = context.vars.resources[randomIndex].id;
return done();
}
module.exports = {
setResourceId: setResourceId
}
I have a parse-server hosted by heroku, which has an index.js file utilized for its configuration. I want to use Mailgun to up functionality for the user to request a password reset, and I have set up the config file, following this answer, as follows:
var api = new ParseServer({
appName: 'App Name',
publicServerURL: 'https://<name>.herokuapp.com/parse',
databaseURI: databaseUri || 'mongodb://localhost:27017/dev',
cloud: process.env.CLOUD_CODE_MAIN || __dirname + '/cloud/main.js',
appId: process.env.APP_ID || 'myAppId',
masterKey: process.env.MASTER_KEY || '', //Add your master key here. Keep it $
serverURL: process.env.SERVER_URL || 'http://localhost:1337/parse', // Don't$
liveQuery: {
classNames: ["Posts", "Comments"] // List of classes to support for query s$
},
push: JSON.parse(process.env.SERVER_PUSH || "{}"),
verifyUserEmails: true, //causing errors
emailAdapter: { //causing errors
module: 'parse-server-simple-mailgun-adapter',
options: {
fromAddress: 'parse#example.com',
domain: '<domain>',
apiKey: '<key>',
}
}
});
This code does not work, though, because of the verifyUserEmails and emailAdapter. Removing both of them removes the "JSON text did not start with array" error. Adding either one of them back in results in the error being thrown. I have no idea why, though, since I do not see any obvious reason as to how they aren't being set up in an array correctly?
Do I need to set up the cooresponding config vars in heroku in addition to having them in the config file? I considered this, but appName and publicServerURL are not set up in this way and don't give this error.
emailAdapter.options.apiKey doesn't need a comma at the end since its the last element of it's JSON.
I wouldn't be surprised that you're also leaving in the comma at the end of verifyUserEmails when you include it improperly as well.
options: {
fromAddress: 'parse#example.com',
domain: '<domain>',
apiKey: '<key>',
}
This is not valid JSON, because there is a comma at the end of the apiKey line. The last item in a JSON object does not have a comma.
For anyone that is repeatedly running into this issue, I have figured out exactly what was going wrong. Despite the error informing me that my JSON was incorrectly formatted, it turns out it was actually that the module was misnamed. According to this post, the updated module has been renamed to '#parse/simple-mailgun-adapter'. Inserting this into the index.js, after ensuring I had ran the npm install --save #parse/simple-mailgun-adapter in my local repo, fixed the issue.
I have a RESTful API that I have created recently and I won't remember how to use it in a few months. I decided to document my API using Swagger, however I'm going crazy.
I used http://editor.swagger.io/ to create the YAML file that I then convert into a JSON file Swagger can use. When I put file into Swagger UI it just gets stuck at fetching resource list: localhost/swagger.json and the console says Uncaught TypeError: Cannot read property '$ref' of undefined .
I'm using version 2.1.0-alpha.5 of Swagger UI.
Here is my spec file:
swagger: '2.0'
info:
title: TITLE
description: BLAH, BLAH, BLAH, ETC
version: "1.0b"
host: api.example.com
schemes:
- http
basePath: /v1
produces:
- application/json
paths:
/match.json:
get:
#summary: Match Data
description: Used for getting data about a match
parameters:
- name: id
in: query
description: The match ID of from a game
required: true
type: integer
format: int32
- name: key
in: query
description: API key used for authentication.
required: true
type: string
responses:
200:
description: Returns match data
schema:
type: array
items:
$ref: '#/definitions/MatchData'
default:
description: Unexpected error
schema:
$ref: '#/definitions/Error'
definitions:
MatchData:
properties:
info:
type: integer
format: int64
description: General information about the match
time:
type: integer
format: int64
description: Information about the start/end time
stats:
type: array
format: int64
description: Stats about the match
Error:
required:
- errorID
- message
properties:
errorID:
type: string
description: Error ID.
message:
type: string
description: Information about the error.
I've tested your spec, and while I'm not getting the same error you do, the spec is indeed invalid.
If you look at #/definitions/MatchData/properties/stats, you'll see that you define the type: array but you don't provide an 'items' property next to it to say which array it is (and that's mandatory). You may have intended to use type: integer like the properties above it, which goes along with the format: int64.
Since I don't know what you intended to provide, it's difficult to give an accurate solution, but if you add a comment with what you intended to do, I could provide a more detailed answer.
Upon some additional testing, I discovered that there's a bug in the UI. After you make that modification and the spec loads, the operation itself will not expand unless you click on the Expand Operations link. I've opened an issue about it, feel free to follow it there.
This problem can be due to some indentation errors in the yaml file which actually did not show up in the Swagger editor. Check all your definitions and whether they are getting displayed as expected in the preview that you can see in Swagger editor (especially check MatchData).
You can also try giving:
responses:
200:
description: Returns match data
schema:
type: array
items:
schema:
$ref: '#/definitions/MatchData'
For our case, we used Swagger-php and we have:
* #SWG\Response(
* response=200,
* description="app response"
* #SWG\Schema(
* type="array"
* )
* ),
but we missed " * #SWG\Items(ref="#/definitions/pet")". After removing "#SWG\Schema(", it works e.g.
* #SWG\Response(
* response=200,
* description="app response"
* ),