Unable to get subnet data using Terraform 0.12 - amazon-elastic-beanstalk

I am trying to query the values of the vpc and subnets, I was able to get the vpc id however I'm failing to get the subnet id's which are supposed to prove 2 values.
Error is already seen on the terraform plan (some details are removed to shorten your reading on my code)
# module.environment.aws_elastic_beanstalk_environment.env will be created
+ resource "aws_elastic_beanstalk_environment" "pogimo123" {
+ all_settings = (known after apply)
+ application = "pogimo123"
+ name = "pogimo123"
+ platform_arn = (known after apply)
+ queues = (known after apply)
+ solution_stack_name = "64bit Amazon Linux 2018.03 v2.9.8 running PHP 7.2"
+ tier = "WebServer"
+ setting {
+ name = "ELBScheme"
+ namespace = "aws:ec2:vpc"
+ value = "internal"
}
+ setting {
+ name = "ELBSubnets"
+ namespace = "aws:ec2:vpc"
+ value = "data.aws_subnet_ids.mysubnets.ids"
}
+ setting {
+ name = "Subnets"
+ namespace = "aws:ec2:vpc"
+ value = "data.aws_subnet_ids.mysubnets.ids"
}
+ setting {
+ name = "VPCId"
+ namespace = "aws:ec2:vpc"
+ value = "vpc-pogimo123"
}
}
If you check on the VPC it shows the value was queried fine showing the vpc id vpc-pogimo123 however on subnets I'm getting these
+ value = "data.aws_subnet_ids.subnets.ids"
It is supposed to give this value
+ value = [
+ "subnet-01293018398409233",
+ "subnet-jlkj312knasdhjalsd",
+ "subnet-908345mnsdfhs3244s",
]
Here is the ERROR
Error: ConfigurationValidationException: Configuration validation exception: Invalid option value: '["data.aws_subnet_ids.mysubnets.ids"]' (Namespace: 'aws:ec2:vpc', OptionName: 'Subnets'): The subnet 'data.aws_subnet_ids.mysubnets.ids' does not exist.
status code: 400, request id: 123h12j3a-12312-4ed3458-c234-0adnahj234hjsa
on ../modules/environment/tfenvtest.tf line 1, in resource "aws_elastic_beanstalk_environment" "tfenvtest":
1: resource "aws_elastic_beanstalk_environment" "tfenvtest" {
Here is the code I used
data "aws_vpc" "myvpc" {
filter {
name = "tag:POGIMO123"
values = ["TRUE"]
}
}
data "aws_subnet_ids" "mysubnets" {
vpc_id = data.aws_vpc.myvpc.id
filter {
name = "tag:Name"
values = ["*_POGIMO123"]
}
}
resource "aws_elastic_beanstalk_application" "tftest" {
name = "pogimo123"
description = "pogimo123"
}
resource "aws_elastic_beanstalk_environment" "tfenvtest" {
name = "tf-test-name"
application = "${aws_elastic_beanstalk_application.tftest.name}"
solution_stack_name = "64bit Amazon Linux 2018.03 v2.9.8 running PHP 7.2"
}
setting {
namespace = "aws:ec2:vpc"
name = "VPCId"
value = data.aws_vpc.myvpc.id
}
setting {
namespace = "aws:ec2:vpc"
name = "ELBSubnets"
value = "data.aws_subnet_ids.mysubnets.ids"
}
setting {
namespace = "aws:ec2:vpc"
name = "Subnets"
value = "data.aws_subnet_ids.mysubnets.ids"
}

I tried to verify your script by launching it in my sandbox account in us-east-1 and default VPC (not your custom VPC as there is no code for it provided).
There were more issues then it was initially apparent:
setting must be in aws_elastic_beanstalk_environment
instead of "data.aws_subnet_ids.mysubnets.ids" it should be join(",", data.aws_subnet_ids.mysubnets.ids)
missing instance profile
The modified and fully working terraform script for EB is as follows:
provider "aws" {
# your data
}
data "aws_vpc" "myvpc" {
default = true
}
data "aws_subnet_ids" "mysubnets" {
vpc_id = data.aws_vpc.myvpc.id
}
resource "aws_elastic_beanstalk_application" "tftest" {
name = "pogimo123"
description = "pogimo123"
}
resource "aws_iam_role" "eb_instance_role" {
path = "/"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "eb_role_attachment" {
role = "${aws_iam_role.eb_instance_role.name}"
policy_arn = "arn:aws:iam::aws:policy/AWSElasticBeanstalkWebTier"
}
resource "aws_iam_instance_profile" "eb_instance_profile" {
role = "${aws_iam_role.eb_instance_role.id}"
# wait for the profile to exist
# it takes time
provisioner "local-exec" {
command = "sleep 30"
}
}
resource "aws_elastic_beanstalk_environment" "tfenvtest" {
name = "tf-test-name"
application = aws_elastic_beanstalk_application.tftest.name
solution_stack_name = "64bit Amazon Linux 2018.03 v2.9.8 running PHP 7.2"
setting {
namespace = "aws:ec2:vpc"
name = "VPCId"
value = data.aws_vpc.myvpc.id
}
setting {
namespace = "aws:autoscaling:launchconfiguration"
name = "IamInstanceProfile"
value = aws_iam_instance_profile.eb_instance_profile.name
}
setting {
namespace = "aws:ec2:vpc"
name = "Subnets"
value = join(",", data.aws_subnet_ids.mysubnets.ids)
}
}

Related

Create Azure policy with Terraform

I am trying to create an azure policy with terraform to add tags to resources.
I want all the resources to inherit the resource group tags.
I've been following documentations and examples here and there but I can't figure out how to have the tags being assigned on the resources.
I think I am close, I do not want to write the tags in every single resources this is not sustainable.
My code is separated in 3 different files:
main.tf
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "=3.37.0"
}
azuread = {
source = "hashicorp/azuread"
version = "2.31.0"
}
}
}
provider "azurerm" {
subscription_id = var.azure_subscription_id
tenant_id = var.azure_tenant_id
features {
resource_group {
prevent_deletion_if_contains_resources = false
}
}
}
#create azure resource group
resource "azurerm_resource_group" "rg" {
name = var.azure_rg_name
location = var.azure_resource_group_location
tags = {
costcenter = var.azure_costcenter
projectcode = var.azure_project_code
environment = var.azure_env_code
client = var.azure_client_code
}
}
#Create azure storage account
resource "azurerm_storage_account" "sa" {
name = lower("${var.azure_project_code}${var.azure_env_code}sa01")
resource_group_name = azurerm_resource_group.rg.name
location = var.azure_resource_group_location
account_tier = "Standard"
account_replication_type = "LRS"
}
#Create container in previously created sa
resource "azurerm_storage_container" "ctnr2" {
name = lower("${var.azure_project_code}${var.azure_env_code}tfstate01")
storage_account_name = azurerm_storage_account.sa.name
container_access_type = "private"
}
#create azure policy definition
resource "azurerm_policy_definition" "az_pol_def" {
name = "Append a tag and its value to resources"
policy_type = "Custom"
mode = "Indexed"
display_name = "Append a tag and its value to resources"
metadata = jsonencode({
"version" : "1.0.1",
"category" : "Tags "
}
)
policy_rule = jsonencode({
"if": {
"field": "[concat('tags[', parameters('tagName'), ']')]",
"exists": "false"
},
"then": {
"effect": "append",
"details": [
{
"field": "[concat('tags[', parameters('tagName'), ']')]",
"value": "[parameters('tagValue')]"
}
]
}
})
}
#assign azure policy created previously
resource "azurerm_resource_group_policy_assignment" "az_pol_assign" {
name = "Append a tag and its value to resources"
resource_group_id = azurerm_resource_group.rg.id
policy_definition_id = azurerm_policy_definition.az_pol_def.id
parameters = jsonencode({
"parameters": {
"tagName": {
"type": "String",
"metadata": {
"displayName": "Tag Name",
"description": "Name of the tag, such as 'environment'"
}
},
"tagValue": {
"type": "String",
"metadata": {
"displayName": "Tag Value",
"description": "Value of the tag, such as 'production'"
}
}
},
})
}
variable.tf
variable "azure_resource_group_location" {
default = "west europe"
description = "Location of the resource group."
}
variable "azure_subscription_id" {
type = string
description = "Azure Subscription Id"
}
variable "azure_tenant_id" {
type = string
description = "Azure Tenant Id"
}
variable "azure_rg_name" {
type = string
description = "Azure Resource Group Name"
}
variable "azure_costcenter" {
type = string
description = "Azure Tag Cost Center"
}
variable "azure_client_code" {
type = string
description = "Azure Tag Client"
}
variable "azure_project_code" {
type = string
description = "Azure Tag Project Code"
}
variable "azure_env_code" {
type = string
description = "Azure Tag Environment Code"
}
resource_group_name.tfvars
#Azure tenant id
azure_tenant_id ="********-****-****-****-************"
#Azure subscription
azure_subscription_id = "********-****-****-****-************"
#Azure resource group location
azure_resource_group_location = "west europe"
#Azure RG name
azure_rg_name = "resource_group_name"
#Azure tag
azure_costcenter = "missions"
#Azure tag project code
azure_project_code = "test_project"
#Azure tag client code
azure_client_code = "leanne"
#Environement tag code :
azure_env_code="dev"
I understand that "parameter_values" should be used for my tags, but I'm not sure how?
Here's an error message which might help.
Any help would be much appreciated.
Thanks in advance !
You declared the policy parameters in the policy assignment (az_pol_assign).
Instead you should declare the parameters in the policy definition (az_pol_def).
In your policy assignment you can then set the values that you want to pass as parameters:
#assign azure policy created previously
resource "azurerm_resource_group_policy_assignment" "az_pol_assign" {
name = "Append a tag and its value to resources"
resource_group_id = azurerm_resource_group.rg.id
policy_definition_id = azurerm_policy_definition.az_pol_def.id
parameters = jsonencode({
tagName = {
value = "environment"
},
tagValue = {
value = "production"
}
})
}
NOTE When you use jsonencode() you don't need to use plain JSON, you can use the simpler HashiCorp configuration language (HCL) syntax as I did in my example.

terraform using for_each to find data source

In a aws_ssoadmin_permission_set_inline_policy ressource, i'm using a for_each to parse a list of name corresponding to my data source name. It doesn't work when using the each.key but wokring when hard coding the value inline_policy = data.aws_iam_policy_document.emobg-sso-billing-admin.json
data "aws_iam_policy_document" "emobg-sso-billing-admin" {
statement {
sid = "VisualEditor0"
effect = "Allow"
actions = [
"aws-marketplace:*",
"aws-portal:*",
"budgets:*"
]
resources = [
"*",
]
}
}
data "aws_iam_policy_document" "emobg-sso-billing-audit" {
statement {
sid = "VisualEditor0"
effect = "Allow"
actions = [
"support:*",
"tag:*",
"s3:*"
]
resources = [
"*",
]
}
}
resource "aws_ssoadmin_permission_set" "emobg" {
for_each = toset(local.permission_sets_name)
name = each.key
description = each.key
instance_arn = local.sso_instance_arn
session_duration = local.session_duration
}
resource "aws_ssoadmin_permission_set_inline_policy" "emobg" {
for_each = toset(local.permission_sets_name)
inline_policy = format("data.aws_iam_policy_document.%s.json", each.key) # <-- doesn't works
# inline_policy = data.aws_iam_policy_document.emobg-sso-billing-admin.json # <-- works
instance_arn = local.sso_instance_arn
permission_set_arn = aws_ssoadmin_permission_set.emobg[each.key].arn
}
locals {
session_duration = "PT8H"
permission_sets_name = [
"emobg-sso-billing-admin",
"emobg-sso-billing-audit",
]
}
The error message is:
2022-11-01T01:19:43.923+0100 [ERROR] vertex "aws_ssoadmin_permission_set_inline_policy.emobg[\"emobg-sso-billing-admin\"]" error: "inline_policy" contains an invalid JSON policy
2022-11-01T01:19:43.923+0100 [ERROR] vertex "aws_ssoadmin_permission_set_inline_policy.emobg (expand)" error: "inline_policy" contains an invalid JSON policy
╷
│ Error: "inline_policy" contains an invalid JSON policy
│
│ with aws_ssoadmin_permission_set_inline_policy.emobg["emobg-sso-billing-admin"],
│ on permission_set.tf line 13, in resource "aws_ssoadmin_permission_set_inline_policy" "emobg":
│ 13: inline_policy = format("data.aws_iam_policy_document.%s.json", each.value)
I really don't understand what's wrong with the JSON policy because it's the same.
Maybe I missed something ?
Because you are using format("data.aws_iam_policy_document.%s.json", each.key), the policy will be literal string "data.aws_iam_policy_document.%s.json".
You have only single policy, so you have to use it directly:
inline_policy = data.aws_iam_policy_document.emobg-sso-billing-admin.json
that's why it works. You do not have more then one aws_iam_policy_document in your code.
Thanks to Marcin, who give me the anser in comment: You can't do what you want. TF does not support dynamic references to different resources.
As it was mentioned, it's not allowed from terraform to make a dynamic references, so I finally used a map even if the name of the policy is the same of the base name.
data "aws_iam_policy_document" "emobg-sso-billing-admin" {
statement {
sid = "VisualEditor0"
effect = "Allow"
actions = [
"aws-marketplace:*",
"aws-portal:*",
"budgets:*"
]
resources = [
"*",
]
}
}
data "aws_iam_policy_document" "emobg-sso-billing-audit" {
statement {
sid = "VisualEditor0"
effect = "Allow"
actions = [
"support:*",
"tag:*",
"s3:*"
]
resources = [
"*",
]
}
}
... [ ALL OTHERS DATA SOURCES POLICIES ARE LISTED HERE ]
resource "aws_ssoadmin_permission_set" "emobg" {
for_each = local.permission_set_map
name = each.key
description = each.key
instance_arn = local.sso_instance_arn
session_duration = local.session_duration
}
resource "aws_ssoadmin_permission_set_inline_policy" "emobg" {
for_each = local.inline_policies_map
inline_policy = each.value
instance_arn = local.sso_instance_arn
permission_set_arn = aws_ssoadmin_permission_set.emobg[each.key].arn
}
locals {
session_duration = "PT8H"
permission_set_map = { for ps in local.permission_sets : ps.name => ps }
inline_policies_map = { for ps in local.permission_sets : ps.name => ps.inline_policy if ps.inline_policy != "" }
}
locals {
permission_sets = [
{
name = "emobg-sso-billing-admin",
inline_policy = data.aws_iam_policy_document.emobg-sso-billing-admin.json
},
{
name = "emobg-sso-billing-audit",
inline_policy = data.aws_iam_policy_document.emobg-sso-billing-audit.json
},
{
... [ All MY POLICIES ARE LISTED HERE ]
}
]
}

Terraforms to create Azure API Management Private Endpoint

I'm trying to script creation of an Azure API Management having a Private Endpoint within a VNET Subnet.
I'm able to create it manually no problem in Azure Portal, but can't quite figure out the terraform script.
The VNET and Subnet are created in a separate process, so they are not in the Terraform script but for the API Management piece I have:
resource "azurerm_api_management" "app" {
location = var.the_location
resource_group_name = "${var.the_resource_group}"
name = "${var.the_prefix}-api-mgmt"
publisher_email = var.api_mgmt_publisher_email
publisher_name = var.api_mgmt_publisher_name
sku_name = "${var.api_mgmt_sku}_1"
tags = var.resource_tags }
resource "azurerm_private_endpoint" "endpoint" {
name = "${var.the_prefix}-api-privateendpoint"
location = var.the_location
resource_group_name = var.the_resource_group
subnet_id = var.subnetId
tags = var.resource_tags
private_service_connection {
name = "api-privateserviceconnection"
private_connection_resource_id = azurerm_api_management.app.id
is_manual_connection = false
subresource_names = [] }}
The var.subnetId is the full id of the subnet ie.
/subscriptions/{subscriptionId}/resourceGroups/OpenEHR/providers/Microsoft.Network/virtualNetworks/OpenEHR-VNET/subnets/API-Subnet
The error I get is
Error: creating Private Endpoint "i365sabppdsdevtb-api-privateendpoint" (Resource Group "i365-uks-ehsabppds-devtb-rg"): network.PrivateEndpointsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="MissingParameterOnPrivateLinkServiceConnection" Message="Private link service connection /subscriptions/8cb2b2d3-9411-46e4-926d-22d6378349bc/resourceGroups/i365-uks-ehsabppds-devtb-rg/providers/Microsoft.Network/privateEndpoints/i365sabppdsdevtb-api-privateendpoint/privateLinkServiceConnections/api-privateserviceconnection is missing required parameter 'group Id'." Details=[]
I think the error is something to so with subresource_names but I can't work out what to put in there.
I tried [ "sites" ] but then I get the error:
│ Error: creating Private Endpoint "i365sabppdsdevtb-api-privateendpoint" (Resource Group "i365-uks-ehsabppds-devtb-rg"): network.PrivateEndpointsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="PrivateEndpointBadRequest" Message="Call to Microsoft.ApiManagement/service failed. Error message: The Request has invalid groupId sites." Details=[]
Any ideas, much appreciated.
Thanks.
Issue was caused because of the private service connection resource id and sub resource names. Please use below configuration
private_connection_resource_id = azurerm_api_management.app.id
subresource_names = ["Gateway"]
Find below code snippets for references
Step1:
Copy below code from main tf file.
provider "azurerm" {
features {}
}
variable "prefix" {
default = "rg_swar"
}
resource "azurerm_resource_group" "example" {
name = "rg_swar-resources"
location = "West Europe"
}
resource "azurerm_virtual_network" "example" {
name = "example-network"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
}
resource "azurerm_subnet" "service" {
name = "service"
resource_group_name = azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.example.name
address_prefixes = ["10.0.1.0/24"]
enforce_private_link_service_network_policies = true
}
resource "azurerm_subnet" "endpoint" {
name = "endpoint"
resource_group_name = azurerm_resource_group.example.name
virtual_network_name = azurerm_virtual_network.example.name
address_prefixes = ["10.0.2.0/24"]
enforce_private_link_endpoint_network_policies = true
}
resource "azurerm_public_ip" "example" {
name = "example-pip"
sku = "Standard"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
allocation_method = "Static"
}
resource "azurerm_lb" "example" {
name = "example-lb"
sku = "Standard"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
frontend_ip_configuration {
name = azurerm_public_ip.example.name
public_ip_address_id = azurerm_public_ip.example.id
}
}
resource "azurerm_private_link_service" "example" {
name = "example-privatelink"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
nat_ip_configuration {
name = azurerm_public_ip.example.name
primary = true
subnet_id = azurerm_subnet.service.id
}
load_balancer_frontend_ip_configuration_ids = [
azurerm_lb.example.frontend_ip_configuration.0.id,
]
}
resource "azurerm_api_management" "app" {
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
name = "swar-api-mgmt"
publisher_email = "test#demo.com"
publisher_name = "Swarna Demo"
sku_name = "Developer_1"
//tags = var.resource_tags
}
resource "azurerm_private_endpoint" "example" {
name = "example-endpoint"
location = azurerm_resource_group.example.location
resource_group_name = azurerm_resource_group.example.name
subnet_id = azurerm_subnet.endpoint.id
private_service_connection {
name = "example-privateserviceconnection"
//private_connection_resource_id = azurerm_private_link_service.example.id
private_connection_resource_id = azurerm_api_management.app.id
subresource_names = ["Gateway"]
is_manual_connection = false
}
}
Step2:
run below commands
terraform plan
terraform apply -auto-approve
Review:
Above code snippet will host the services into Azure Portal.
Hope this helps!

GCP compute_engine network interface terraform error

My terraform file looks like this:
resource "google_compute_instance" "virtual_instance" {
name = "${var.instance_name}"
machine_type = "${var.instance_type}"
zone = "${var.zone}"
lifecycle {
ignore_changes = ["boot_disk.0.initialize_params.0.image"]
}
boot_disk {
initialize_params {
image = "ubuntu-os-cloud/ubuntu-1604-lts"
size = "30"
type = "pd-standard"
}
}
network_interface {
network = "default"
access_config {}
}
attached_disk {
source = "${google_compute_disk.managed_data_disk.name}"
mode = "READ_WRITE"
}
metadata {
}
}
This above code created the instance. But when i change then network_interface block as mentioned below
network_interface {
network = "${module.vpc.vpc_name}"
subnetwork = "${module.vpc.subnet_name}"
access_config {}
}
The VPC module is :
resource "google_compute_network" "vpc" {
name = "${var.name}-vpc"
auto_create_subnetworks = "false"
}
resource "google_compute_subnetwork" "subnet_public" {
name = "${var.subnet_name_public}"
ip_cidr_range = "${var.subnet_cidr_public}"
network = "${var.name}-vpc"
depends_on = ["google_compute_network.vpc"]
region = "${var.region}"
}
resource "google_compute_firewall" "firewall" {
name = "${var.name}-firewall"
network = "${google_compute_network.vpc.name}"
allow {
protocol = "icmp"
}
allow {
protocol = "tcp"
ports = ["22"]
}
source_ranges = ["0.0.0.0/0"]
}
when I changed into network_interface to custom values. It's throwing the error is
google_compute_instance.virtual_instance: Error creating network interfaces: exactly one of network or subnetwork must be provided
Please help me on this
Advance Thanks to #ydaetskcoR. If you choose custom values of network_interface. You can't mention both network and subnetwork. You will choose only subnetwork values mentioned below.
network_interface {
subnetwork = "${module.vpc.subnet_name}"
access_config {}
}

What is the correct way to add a second disk using Terraform on Google Compute

I am trying to add a second disk to a Google Compute instance using Terraform. This seems to be correct:
resource "google_compute_disk" "seconddisk" {
name = "seconddisk"
type = "pd-standard"
zone = "us-west1-a"
size = "100"
}
resource "google_compute_instance" "someinstance" {
name = "someinstance"
machine_type = "n1-standard-4"
zone = "us-west1-a"
disk {
image = "${var.image_url}"
}
disk {
disk = "${google_compute_disk.seconddisk.name}"
}
...
}
However, I get the following error:
google_compute_instance.kafka1: Error creating instance: googleapi: Error 409: The resource '...' already exists, alreadyExists.
Thoughts?
As of 2022 May, you can do following
# disk
resource "google_compute_disk" "default" {
name = "compute-disk"
}
# compute
resource "google_compute_instance" "default" {
name = "attached-disk-instance"
machine_type = "e2-medium"
zone = "us-west1-a"
boot_disk {
initialize_params {
image = "debian-cloud/debian-9"
}
}
network_interface {
network = "default"
}
lifecycle {
ignore_changes = [attached_disk]
}
}
# connect compute & disk
resource "google_compute_attached_disk" "default" {
disk = google_compute_disk.default.id
instance = google_compute_instance.default.id
}
Helpful resources
https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_attached_disk
https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance
https://github.com/terraform-google-modules/terraform-docs-samples