Problem passing parameters created in Terraform to CloudFormation - parameter-passing

Using guidance from a previous SO post: [https://stackoverflow.com/questions/43266506/is-it-possible-to-execute-a-cloudformation-file-in-terraform]:
I am trying to pass two parameters created in my Terraform template into a Cloudformation template as parameters:
My terraform (v.0.13.5) code:
resource "aws_cloudformation_stack" "cloudwatch-synthetics-canary" {
provider = aws.primary
name = "cloudwatch-synthetics"
parameters = {
CloudWatchSyntheticsRole = aws_iam_role.cloudwatch_synthetics_role.arn,
ResultsBucket = aws_s3_bucket.results_bucket.arn
}
template_body = file("${path.module}/cloudwatch_canary.yml")
}
resource "aws_iam_role" "cloudwatch_synthetics_role" {
provider = aws.primary
name = "CloudWatchSyntheticsRole"
description = "Allows Cloudwatch Lambda to operate in this account."
assume_role_policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowLambdaAssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
POLICY
}
resource "aws_iam_policy" "cloudwatch_synthetics_role_policy" {
provider = aws.primary
name = "CloudWatchSyntheticsRolePolicy"
path = "/"
description = "Addtional allowances for the synthetics role"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
# --- SNIPPED FOR BREVITY ---
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "cloudwatch_synthetics_role_policy_attachment" {
provider = aws.primary
role = aws_iam_role.cloudwatch_synthetics_role.name
policy_arn = aws_iam_policy.cloudwatch_synthetics_role_policy.arn
}
My Cloudformation code:
Parameters:
CanaryName:
Type: String
Default: my-canary
MaxLength: 21
HostName:
Type: String
Default: api.myhost.net
MaxLength: 128
Path:
Type: String
Default: /v1/status
MaxLength: 256
Port:
Type: Number
Default: 443
CloudWatchSyntheticsRole:
Type: AWS::IAM::Role
ResultsBucket:
Type: AWS::S3::Bucket
...
Resources:
ExecutionRoleArn:
Fn::GetAtt:
- CloudWatchSyntheticsRole
- Arn # <-- TRIED WITH AND WITHOUT THIS
In Terraform.io, the error is as follows:
Error: Creating CloudFormation stack failed: ValidationError: Template error: instance of Fn::GetAtt references undefined resource CloudWatchSyntheticsRole
status code: 400, request id: 694c...
I tried making the types in the CFT as Strings, but that didn't seem to work, either. The Fn::GetAtt: also had '- Arn' below the CloudwatchSyntheticsRole to refer to the Arn, but since I'm passing that in directly, I tried removing it.
I feel like I'm close, but missing something and just need some help from some fresh eyes.

It looks like the solution, found by a co-worker of mine is pretty simple:
Make sure ALL CFT variables are in Terraform and pass any default values through Terraform as literals to Cloudformation
Remove all default parameters from CloudFormation, but still keep the Name and Type of variable at a minimum. Other constraints like 'MaxLength' are ok, too.
Example:
Terraform.tf:
resource "aws_cloudformation_stack" "cloudwatch-synthetics-canary" {
provider = aws.primary
name = "cloudwatch-synthetics"
parameters = {
CanaryName = "my-canary",
HostName = "api.server.net",
Path = "/v1/status",
Port = 443,
RoleArn = aws_iam_role.cloudwatch_synthetics_role.arn,
S3Location = "s3://${aws_s3_bucket.results_bucket.id}"
}
template_body = file("${path.module}/cloudwatch_canary.yml")
}
Cloudformation.yml:
Parameters:
CanaryName:
Type: String
MaxLength: 21
HostName:
Type: String
MaxLength: 128
Path:
Type: String
MaxLength: 256
Port:
Type: Number
RoleArn:
Type: String
S3Location:
Type: String
MaxLength: 1024

Related

Is the Ansible Inventory compatible with OpenAPI standards?

I´m trying to specify the API used by ansibles dynamic inventory.
Does anyone have experience with solving the incompatibility?
For a single host it might work as following:
The Ansible json output will be like this:
{
"ansible_host": "172.16.19.123",
"proxy": "somehost.domain.fake"
}
The openapi.yml
paths:
/api/inventory/host/:
get:
summary: Gets One Ansible Host
parameters:
- in: query
name: hostname
schema:
type: string
required: true
responses:
'200':
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/SingleHost'
components:
schemas:
SingleHost: # A Single Host for ansible-inventory --host [hostname] requests
type: object
properties:
ansible_host:
type: string
description: IP-Address of the host
ansible_port:
type: integer
description: ssh-port number of host
But when specifiyng the endpoint for ansible-inventory --list it gets tricky already..
example of a ansible inventory yml
{
"HostGroupName-A": {
"hosts": [
"Host-A"
]
},
"HostGroupName-B": {
"hosts": [
"Host-A",
"Host-B"
]
}
}
Should I just avoid using openapi to specify this?

Intermittent Cloud SQL, Export Failure - Code:2, Message:"UNKNOWN", severity:"ERROR"

When attempting to export from Cloud SQL using a RESTful call to the Google API we've been encountering random failures, approximately 1 in 10 attempts. The log extract below has been taken from Stackdriver and doesn't offer much insight into the root cause:
{
insertId: "55C4B416A7642.A1B5D8E.570939C6"
logName: "projects/my-dev-project/logs/cloudaudit.googleapis.com%2Fdata_access"
protoPayload: {
#type: "type.googleapis.com/google.cloud.audit.AuditLog"
authenticationInfo: {
principalEmail: "jsmith#mycompany.com"
}
authorizationInfo: [
0: {
authorizationLoggingOptions: {
permissionType: "DATA_READ"
}
granted: true
permission: "cloudsql.instances.export"
resource: "instances/my-db-instance"
}
]
methodName: "cloudsql.instances.export"
request: {
#type: "type.googleapis.com/cloudsql.admin.InstancesExportRequest"
exportContext: {
database: [
0: "business_db"
]
fileType: "CSV"
schemaOnly: true
selectQuery:
"SELECT
replace(IFNULL(`ID`,'<NULL_VALUE>'), '', '') AS `ID`,
replace(IFNULL(`NAME`,'<NULL_VALUE>'), '', '') AS `NAME`,
replace(IFNULL(`ADDRESS`,'<NULL_VALUE>'), '', '') AS `ADDRESS`,
replace(IFNULL(`DOB`,'<NULL_VALUE>'), '', '') AS `DOB`
FROM CUSTOMERS"
table: [
0: "CUSTOMERS"
]
uri: "gs://my-dev-project-stage/CUSTOMERS_full.csv"
}
instanceName: {
fullProjectId: "my-dev-project"
instanceId: "my-db-instance"
}
}
requestMetadata: {
callerIp: "107.178.192.158"
}
resourceName: "instances/my-db-instance"
serviceName: "cloudsql.googleapis.com"
status: {
code: 2
message: "UNKNOWN"
}
}
receiveTimestamp: "2017-10-24T13:52:53.738754270Z"
resource: {
labels: {
database_id: "my-dev-project:my-db-instance"
project_id: "my-dev-project"
region: "europe"
}
type: "cloudsql_database"
}
severity: "ERROR"
timestamp: "2017-10-24T13:52:53.317Z"
}
The only difference between this log entry and a successful export are the three property values mentioned in the post title. In a successful entry they are as follows:
status: {
}
}
receiveTimestamp: "2017-10-24T14:21:44.997050352Z"
resource: {
labels: {
database_id: "my-dev-project:my-db-instance"
project_id: "my-dev-project"
region: "europe"
}
type: "cloudsql_database"
}
severity: "INFO"
timestamp: "2017-10-24T14:21:43.757Z"
}
The intermittent nature suggests the root cause is an external factor. I.e. the exact same request with the same parameters; connection details, output file location and SQL statement will work a few minutes later. There's no consistency to which exports are failing.
We've already discovered that Cloud SQL does not support multiple simultaneous exports and we've implemented a queuing system to avoid this issue.
The data volumes don't seem to be an influencing factor either. An export will work for a 100,000 row file and then fail for 500 row file. In fact smaller or even empty data sets seem to be more susceptible to failure.
If anyone has any idea's or suggestions as to a solution or avenue of investigation I'd be interested to hear them. Thanks.

Unable to Create a CloudWatch Healthcheck via Ansible

I have a inventory file which has a RDS endpoint as :
[ems_db]
syd01-devops.ce4l9ofvbl4z.ap-southeast-2.rds.amazonaws.com
I wrote the following play book to create a Cloudwatch ALARM :
---
- name: Get instance ec2 facts
debug: var=groups.ems_db[0].split('.')[0]
register: ems_db_name
- name: Display
debug: var=ems_db_name
- name: Create CPU utilization metric alarm
ec2_metric_alarm:
state: present
region: "{{aws_region}}"
name: "{{ems_db_name}}-cpu-util"
metric: "CPUUtilization"
namespace: "AWS/RDS"
statistic: Average
comparison: ">="
unit: "Percent"
period: 300
description: "It will be triggered when CPU utilization is more than 80% for 5 minutes"
dimensions: { 'DBInstanceIdentifier' : ems_db_name }
alarm_actions: arn:aws:sns:ap-southeast-2:493552970418:cloudwatch_test
ok_actions: arn:aws:sns:ap-southeast-2:493552970418:cloudwatch_test
But this results in
TASK: [cloudwatch | Get instance ec2 facts] ***********************************
ok: [127.0.0.1] => {
"var": {
"groups.ems_db[0].split('.')[0]": "syd01-devops"
}
}
TASK: [cloudwatch | Display] **************************************************
ok: [127.0.0.1] => {
"var": {
"ems_db_name": {
"invocation": {
"module_args": "var=groups.ems_db[0].split('.')[0]",
"module_complex_args": {},
"module_name": "debug"
},
"var": {
"groups.ems_db[0].split('.')[0]": "syd01-devops"
},
"verbose_always": true
}
}
}
TASK: [cloudwatch | Create CPU utilization metric alarm] **********************
failed: [127.0.0.1] => {"failed": true}
msg: BotoServerError: 400 Bad Request
<ErrorResponse xmlns="http://monitoring.amazonaws.com/doc/2010-08-01/">
<Error>
<Type>Sender</Type>
<Code>MalformedInput</Code>
</Error>
<RequestId>f30470a3-2d65-11e6-b7cb-cdbbbb30b60b</RequestId>
</ErrorResponse>
FATAL: all hosts have already failed -- aborting
What is wrong here? What can i do to solve this ? I am new to this but surely this seems some syntax issue with me or the way i am picking up the inventory endpoint split.
The variable from debug isn't being assigned in the first debug statement, though you may be able to if you change it to a message and enclose it with quotes and double braces (untested):
- name: Get instance ec2 facts
debug: msg="{{groups.ems_db[0].split('.')[0]}}"
register: ems_db_name
However, I would use the set_fact module in that task (instead of debug) and assign that value to it. That way, you can reuse it in this and subsequent calls to a play.
- name: Get instance ec2 facts
set_fact: ems_db_name="{{groups.ems_db[0].split('.')[0]}}"
UPDATE: Add a threshold: 80.0 to the last task, and the dimensions needs to use the instance id encapsulated with double braces.

Swagger POST Json Body Parameter Schema YAML

i'm working on a RESTful API using swagger-api and swagger-editor for routes.
I can't figure out why the JSON i am sending through body, never reaches my controller.
here is my YAML
schemes:
- http
- https
produces: [application/json, multipart/form-data, application/x-www-form-urlencoded]
paths:
/projects:
x-swagger-router-controller: project
post:
description: create a new project
operationId: postProjects
consumes:
- application/json
parameters:
- name: param1
in: body
description: description
required: false
schema:
$ref: "#/definitions/Project"
responses:
"200":
description: Success
schema:
$ref: "#/definitions/Project"
default:
description: Error
schema:
$ref: "#/definitions/ErrorResponse"
definitions:
Project:
properties:
name:
type: string
required:
- name
an example of the post request i'm sending.
curl -v -X POST -H "Content-Type: application/json" -d '{"name":"test"}' http://127.0.0.1:10010/projects
and the response
{"message":"Request validation failed: Parameter (param1) failed schema validation","code":"SCHEMA_VALIDATION_FAILED","failedValidation":true,"results":{"errors":[{"code":"OBJECT_MISSING_REQUIRED_PROPERTY","message":"Missing required property: name","path":[]}],"warnings":[]},"path":["paths","/projects","post","parameters","0"],"paramName":"param1"}
If i set the parameter "name" as not required, i just received an empty response like this
{ param1:
{ path: [ 'paths', '/projects', 'post', 'parameters', '0' ],
schema:
{ name: 'param1',
in: 'body',
description: 'description',
required: false,
schema: [Object] },
originalValue: {},
value: {} } }
I have no clue why since other format such as header, path or formdata works fine.
I always receive an empty object. req.swagger.params has no value.
I tried several schema but even the simplest is not working.
i can tell from the header that 'content-type': 'application/json'.
So the content type is set, the schema validates a simple string argument named "name". Everything should be ok. but still not.
This issue has been fixed.
It wasn't swagger related.
I built an API with nodeJs and i realized i didn't have a middleware to handle body parameter.
So because i missed a step before enabled the swagger middleware, i couldn't do anything with body parameter.
The main reason behind getting null values when sending json data to the API backend is the paramater path most of the time you give and the naming you give the parameter.
You also have to declare explicitly the type of schema you expect
You have to set parameter name to body and set in: body so that it picks the body object as a JSON
An example is here. You can try it out
/auth/register:
post:
tags:
- Auth
parameters:
- in: body
name: user
description: Create a new user.
schema:
type: object
required:
- firstName
- lastName
- email
- password
- confirmPassword
properties:
firstName:
type: string
lastName:
type: string
email:
type: string
password:
type: string
confirmPassword:
type: string
example:
firstName: Jane
lastName: Doe
email: janedoe#gmail.com
password: pass
confirmPassword: pass
responses:
"200":
description: OK

Deprecated: `config.adapters.default` Sails.js

When I lift my sails project I get these warnings on terminal
debug: Deprecated: config.adapters.default debug: (see
http://links.sailsjs.org/docs/config/connections) debug: For now, I'll
pretend you set config.models.connection.
debug: Deprecated: config.adapters debug: (see
http://links.sailsjs.org/docs/config/connections) debug: For now, I'll
pretend you set config.connections.
debug: Deprecated: config.adapters.*.module debug: (see
http://links.sailsjs.org/docs/config/connections) debug: For now, I'll
pretend you set config.connections["disk"].adapter.
And my data is not stored in mysql DB.
This is my api model code
module.exports = {
schema:true,
tableName: 'BusStop',
adapters: 'mysql-adapter',
migrate: 'safe',
attributes: {
stopID:{
type: 'int'
},
stopName:{
type: 'string'
},
latitude:{
type: 'float'
},
longitude:{
type: 'float'
}
}
};
This is my local.js code
module.exports = {
port: process.env.PORT || 1337,
environment: process.env.NODE_ENV || 'development',
adapters:{
'default': 'disk',
disk:{
module:'sails-disk'
},
'mysql-adapter': {
module : 'mysql-sails',
host : '127.0.0.1',
user : 'root',
password : '',
database : 'PublicTransport',
schema : true
}
}
};
And this is my connections.js code
module.exports.connections = {
localDiskDb: {
adapter: 'sails-disk'
},
'mysql-adapter': {
module : 'sials-mysql',
host : '127.0.0.1',
port : 3306,
user : 'root',
password : '',
database : 'PublicTransport'
},
someMongodbServer: {
adapter: 'sails-mongo',
host: 'localhost',
port: 27017,
},
somePostgresqlServer: {
adapter: 'sails-postgresql',
host: 'YOUR_POSTGRES_SERVER_HOSTNAME_OR_IP_ADDRESS',
user: 'YOUR_POSTGRES_USER',
password: 'YOUR_POSTGRES_PASSWORD',
database: 'YOUR_POSTGRES_DB'
}
};
I want to know how can I remove these Deprecated warnings and why my data is not being stored in the Database.
Looks like the links in those deprecation warnings are incorrect. For information on how to configure Sails connections, see http://sailsjs.org/#/documentation/reference/sails.config/sails.config.connections.html.
You should be able to make those messages go away by following their instructions regarding what config keys to set.
In order to set a connection for a specific model, you now set the connection property:
module.exports = {
schema:true,
tableName: 'BusStop',
connection: 'mysql-adapter', // "adapter" is now "connection
migrate: 'safe',
attributes: {...}
}
In order to specify the adapter to use for a connection, use the adapter key, not module; you're already doing this in most of your connections, you just need to update mysql-adapter.
In your config/local.js you're using the deprecated adapters key to set up connections; use connections instead.
Lastly, in order to set a default connection for all of your models, you do as the deprecation message says and set sails.config.models.connection rather than sails.config.adapters.default; you can do so easily in the config/models.js file, or in your config/local.js like so:
module.exports = {
models: {
connection: 'mysql-adapter'
},
...more config...
}
This warning can also be generated if your project has an config/adapters.js file. If the file is empty it can be deleted and Zap the warning is gone.If it contains code then you will need to migrate it to the correct path config/models.js and config/connections.js