How to use user variables with file provisioner in Packer? - packer

I have a packer json like:
"builders": [{...}],
"provisioners": [
{
"type": "file",
"source": "packer/myfile.json",
"destination": "/tmp/myfile.json"
}
],
"variables": {
"myvariablename": "value"
}
and myfile.json is:
{
"var" : "{{ user `myvariablename`}}"
}
The variable into the file does get replaced, is a sed replacement with shell provisioner after the file the only option available here?
Using packer version 0.12.0

You have to pass these as environment variables. For example:
"provisioners": [
{
"type": "shell"
"environment_vars": [
"http_proxy={{user `proxy`}}",
],
"scripts": [
"some_script.sh"
],
}
],
"variables": {
"proxy": null
}
And in the script you can use $http_proxy

So far I've come just with the solution to use file & shell provisioner. Upload file and then replace variables in file via shell provisioner which can be fed from template variables provided by e.g. HashiCorp Vault

Yo may use OS export function to set environment and pass it to Packer
Here is a config using OS ENV_NAME value to choose local folder to copy from
export ENV_NAME=dev will set local folder to dev
{
"variables": {
...
"env_folder": "{{env `ENV_NAME`}}",
},
"builders": [{...}]
"provisioners": [
{
"type": "file",
"source": "files/{{user `env_folder`}}/",
"destination": "/tmp/"
},
{...}
]
}

User variables must first be defined in a variables section within your template. Even if you want a user variable to default to an empty string, it must be defined. This explicitness helps reduce the time it takes for newcomers to understand what can be modified using variables in your template.
The variables section is a key/value mapping of the user variable name to a default value. A default value can be the empty string. An example is shown below:
{
"variables": {
"aws_access_key": "",
"aws_secret_key": ""
},
"builders": [{
"type": "amazon-ebs",
"access_key": "{{user `aws_access_key`}}",
"secret_key": "{{user `aws_secret_key`}}",
// ...
}]
}
check this link for more information

Related

Reference value from state’s input, using JSONPath syntax in a SSMSendCommand API step through parameter which expects an array

I have on a AWS state machine the following step defined for api aws-sdk:ssm:sendCommand
{
"Type": "Task",
"Parameters": {
"DocumentName.$": "$.result.DocumentName",
"InstanceIds.$": "$..Dimensions[?(#.Name=~/.*InstanceId.*/)].Value",
"MaxErrors": "0",
"MaxConcurrency": "100%",
"CloudWatchOutputConfig": {
"CloudWatchLogGroupName": "diskspace-log",
"CloudWatchOutputEnabled": true
},
"Parameters": {
"workingDirectory": [
""
],
"executionTimeout": [
"3600"
],
"commands": [
"echo -------------------Mounting volume without signals $..Dimensions[?(#.Name=~/.*device.*/)].Value---------------------",
"echo",
"mount $..Dimensions[?(#.Name=~/.*device.*/)].Value"
]
}
}
}
The section: "commands": [] expects an array.
"commands" should accept input reference as any other parameter in the schema, so in theory will be posible to use json path parameters (Example: "size.$": "$.product.details.size") for referencing needed parameters from input.
https://docs.aws.amazon.com/step-functions/latest/dg/input-output-inputpath-params.html
This following example works without using input referencing :
"commands": [
"echo -------------------Mounting /dev/ebs---------------------",
"echo",
"mount /dev/ebs"
]
But I need to reference from input, hardcoded values won't work for me. I tried, but no working.
"commands": [
"echo -------------------Mounting volume without signals $..Dimensions[?(#.Name=~/.*device.*/)].Value---------------------",
"echo",
"mount $..Dimensions[?(#.Name=~/.*device.*/)].Value"
]
Also tried, not working also:
"commands.$": "States.Array(States.Format('echo -------------------Mounting volume without signals {} ---------------------', $..Dimensions[?(#.Name=~/.*device.*/)].Value),'echo',States.Format('mount {}', $..Dimensions[?(#.Name=~/.*device.*/)].Value))"
I believe some of the provided intrinsic functions will help on achieving the expected result but I'm lost on how to properly set up the syntax.
https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html#asl-intrsc-func-arrays
The step calls a RunShellScript type of documentCommand.
And executes the commands provided on parameters in the step of the state machine.
I got on the output:
States.Format('echo -------------------Mounting volume without signals {} ---------------------', $..Dimensions[?(#.Name=~/.*device.*/)].Value)'
Its not detecting the input reference, I expect to output.
-------------------Mounting volume without signals /dev/ebs ---------------------
and in the background execute:
mount /dev/ebs
I was able to send the commands through a Pass State Flow, here is the definition:
{
"Type": "Pass",
"Next": "SendCommand",
"ResultPath": "$.ForArgs",
"Parameters": {
"Params": {
"Args": [
{
"Arg1": "ec2-metadata -i"
},
{
"Arg2": "echo"
},
{
"Arg3.$": "States.Format('echo -------------------Mounting volume without signals {} ---------------------', States.ArrayGetItem($..Dimensions[?(#.Name=~/.*device.*/)].Value, 0))"
},
{
"Arg4": "echo"
},
{
"Arg5.$": "States.Format('mount {}', States.ArrayGetItem($..Dimensions[?(#.Name=~/.*device.*/)].Value, 0))"
},
{
"Arg6.$": "States.Format('echo Checking if device {} is mounted', States.ArrayGetItem($..Dimensions[?(#.Name=~/.*device.*/)].Value, 0))"
},
{
"Arg7.$": "States.Format('if findmnt --source \"{}\" >/dev/null', States.ArrayGetItem($..Dimensions[?(#.Name=~/.*device.*/)].Value, 0))"
},
{
"Arg8": "\tthen echo device is mounted"
},
{
"Arg9": "\telse echo device is not mounted"
},
{
"Arg10": "fi"
}
]
}
}
}
Next on the sendCommandApi:
"commands.$": "$.ForArgs.Params.Args[*][*]"

Substituting service url is arm template

I have an ARM template that deploys API's to an API Management instance
Here is an example of one API
{
"properties": {
"authenticationSettings": {
"subscriptionKeyRequired": false
},
"subscriptionKeyParameterNames": {
"header": "Ocp-Apim-Subscription-Key",
"query": "subscription-key"
},
"apiRevision": "1",
"isCurrent": true,
"subscriptionRequired": true,
"displayName": "DDD.CRM.PostLeadRequest",
"serviceUrl": "https://test1/api/FuncCreateLead?code=XXXXXXXXXX",
"path": "CRMAPI/PostLeadRequest",
"protocols": [
"https"
]
},
"name": "[concat(variables('ApimServiceName'), '/mms-crm-postleadrequest')]",
"type": "Microsoft.ApiManagement/service/apis",
"apiVersion": "2019-01-01",
"dependsOn": []
}
When I am deploying this to different environments I would like to be able to substitute the service url depending on the environment. I'm wondering the best approach?
Can I read in a config file or something like that?
At the time of deployment I have a variable that tells me the environment so I can base decisions on that. Just not sure the best way to do it
See about ARM template parameters: https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-authoring-templates#parameters They can be specified in a separate file. So you will have single template, but environment specific parameter files.

How to define a variable within a JSON file and use it within JSON file

I'm trying to find if JSON file supports defining variables and using them within that JSON file?
{
"artifactory_repo": "toplevel_virtual_NonSnapshot",
"definedVariable1": "INSTANCE1",
"passedVariable2": "${passedFromOutside}",
"products": [
{ "name": "product_${definedVariable1}_common",
"version": "1.1.0"
},
{ "name": "product_{{passedVariable2}}_common",
"version": 1.5.1
}
]
}
I know YAML files allow this but now sure if JSON file allows this behavior or not. My plan is that a user will pass "definedVariable" value from Jenkins and I'll create a target JSON file (after substi
This might help you:
{
"artifactory_repo": "toplevel_virtual_NonSnapshot",
"definedVariable1": "INSTANCE1",
"passedVariable2": `${passedFromOutside}`,
"products": [
{ "name": `product_${definedVariable1}_common`,
"version": "1.1.0"
},
{ "name": `product_${passedVariable2}_common`,
"version": 1.5.1
}
]
}
*Note the use of `` instead of ''

How to pass argument in packer provision script?

I am struggling to pass input parameter to packer provisioning script. I have tried various options but no joy.
Objective is my provision.sh should accept input parameter which I send during packer build.
packer build -var role=abc test.json
I am able to get the user variable in json file however I am unable to pass it provision script. I have to make a decision based on the input parameter.
I tried something like
"provisioners":
{
"type": "shell",
"scripts": [
"provision.sh {{user `role`}}"
]
}
But packer validation itself is failed with no such file/directory error message.
It would be real help if someone can help me on this.
Thanks in advance.
You should use the environment_vars option, see the docs Shell Provisioner - environment_vars.
Example:
"provisioners": [
{
"type": "shell"
"environment_vars": [
"HOSTNAME={{user `vm_name`}}",
"FOO=bar"
],
"scripts": [
"provision.sh"
],
}
]
If your script is already configured to use arguments; you simply need to run it as inline rather than in the scripts array.
In order to do this, the script must exist on the system already - you can accomplish this by copying it to the system with the file provisioner.
"provisioners": [
{
"type": "file",
"source": "scripts/provision.sh",
"destination": "/tmp/provision.sh"
},
{
"type": "shell",
"inline": [
"chmod u+x /tmp/provision.sh",
"/tmp/provision.sh {{user `vm_name`}}"]
}
]

How to specify metadata for GCE in packer?

I'm trying to create a GCE image from packer template.
Here is the part that I use for that purpose.
"builders": [
...
{
"type": "googlecompute",
"account_file": "foo",
"project_id": "bar",
"source_image": "centos-6-v20160711",
"zone": "us-central1-a",
"instance_name": "packer-building-image-centos6-baz",
"machine_type": "n1-standard-1",
"image_name": "centos6-some-box-name",
"ssh_username": "my_username",
"metadata": {
"startup-script-log-dest": "/opt/script.log",
"startup-script": "/opt/startup.sh",
"some_other_custom_metadata_key": "some_value"
},
"ssh_pty": true
}
],
...
I have also created the required files. Here is that part
"provisioners": [
...
{
"type": "file",
"source": "{{user `files_path`}}/startup.sh",
"destination": "/opt/startup.sh"
},
...
{
"type": "shell",
"execute_command": "sudo sh '{{.Path}}'",
"inline": [
...
"chmod ugo+x /opt/startup.sh"
]
}
...
Everything works for me without "metadata" field. I can create image/instance with provided parameters. but when I try to create an instance from the image, I can't find the provided metadata and respectively I can't run my startup script, set logging file and other custom metadata.
Here is the source that I use https://www.packer.io/docs/builders/googlecompute.html#metadata.
Any suggestion will be helpful.
Thanks in advance
The metadata tag startup-script should contain the actuall script not a path. Provisioners run after the startup script has been executed (at least started).
Instead use startup_script_file in Packer and supply a path to a startup script.