I'm trying to run serve frontend/dist -l 4000 from PM2. This is supposed to serve a Vue app on port 4000.
In my ecosystem.config.js, I have:
{
name: 'parker-frontend',
max_restarts: 5,
script: 'serve',
args: 'frontend/dist -l 4000',
instances: 1,
},
But when I do pm2 start, in the logs I have the following message:
Exposing /var/lib/jenkins/workspace/parker/frontend/dist directory on port NaN
Whereas if I run the same command: serve frontend/dist -l 4000, it runs just fine on port 4000.
After running serve frontend/dist -l 5000 I got an error in the PM2 logs.
In it's call stack I've found:
at Object.<anonymous> (/usr/lib/node_modules/pm2/lib/API/Serve.js:242:4)
Notice the path: /usr/lib/node_modules/pm2/lib/API/Serve.js
There is another command that's called serve in pm2 itself that was ran instead of the correct one. This is not the npm i -g serve I installed before. This is due to how Node package resolution works - it prioritizes local modules first.
To use the globally installed version (the correct one), you need to specify the exact path to your global serve.
To find out the path - on Linux, you can just do:
$ which serve
/usr/local/bin/serve
Then put the path in your ecosystem.config.js script property.
Final working ecosystem.config.js:
{
name: 'parker-frontend',
script: '/usr/local/bin/serve', //pm2 has it's own 'serve' which doesn't work, make sure to use global
args: 'frontend/dist -l 5000',
instances: 1,
},
```
The scripts portion of my package.json currently looks like this:
"scripts": {
"start": "node ./script.js server"
}
...which means I can run npm start to start the server. So far so good.
However, I would like to be able to run something like npm start 8080 and have the argument(s) passed to script.js (e.g. npm start 8080 => node ./script.js server 8080). Is this possible?
npm 2 and newer
It's possible to pass args to npm run since npm 2 (2014). The syntax is as follows:
npm run <command> [-- <args>]
Note the -- separator, used to separate the params passed to npm command itself, and the params passed to your script.
With the example package.json:
"scripts": {
"grunt": "grunt",
"server": "node server.js"
}
here's how to pass the params to those scripts:
npm run grunt -- task:target // invokes `grunt task:target`
npm run server -- --port=1337 // invokes `node server.js --port=1337`
Note: If your param does not start with - or --, then having an explicit -- separator is not needed; but it's better to do it anyway for clarity.
npm run grunt task:target // invokes `grunt task:target`
Note below the difference in behavior (test.js has console.log(process.argv)): the params which start with - or -- are passed to npm and not to the script, and are silently swallowed there.
$ npm run test foobar
['C:\\Program Files\\nodejs\\node.exe', 'C:\\git\\myrepo\\test.js', 'foobar']
$ npm run test -foobar
['C:\\Program Files\\nodejs\\node.exe', 'C:\\git\\myrepo\\test.js']
$ npm run test --foobar
['C:\\Program Files\\nodejs\\node.exe', 'C:\\git\\myrepo\\test.js']
$ npm run test -- foobar
['C:\\Program Files\\nodejs\\node.exe', 'C:\\git\\myrepo\\test.js', 'foobar']
$ npm run test -- -foobar
['C:\\Program Files\\nodejs\\node.exe', 'C:\\git\\myrepo\\test.js', '-foobar']
$ npm run test -- --foobar
['C:\\Program Files\\nodejs\\node.exe', 'C:\\git\\myrepo\\test.js', '--foobar']
The difference is clearer when you use a param actually used by npm:
$ npm test --help // this is disguised `npm --help test`
npm test [-- <args>]
aliases: tst, t
To get the parameter value, see this question. For reading named parameters, it's probably best to use a parsing library like yargs or minimist; nodejs exposes process.argv globally, containing command line parameter values, but this is a low-level API (whitespace-separated array of strings, as provided by the operating system to the node executable).
You asked to be able to run something like npm start 8080. This is possible without needing to modify script.js or configuration files as follows.
For example, in your "scripts" JSON value, include--
"start": "node ./script.js server $PORT"
And then from the command-line:
$ PORT=8080 npm start
I have confirmed that this works using bash and npm 1.4.23. Note that this work-around does not require GitHub npm issue #3494 to be resolved.
You could also do that:
In package.json:
"scripts": {
"cool": "./cool.js"
}
In cool.js:
console.log({ myVar: process.env.npm_config_myVar });
In CLI:
npm --myVar=something run-script cool
Should output:
{ myVar: 'something' }
Update: Using npm 3.10.3, it appears that it lowercases the process.env.npm_config_ variables? I'm also using better-npm-run, so I'm not sure if this is vanilla default behavior or not, but this answer is working. Instead of process.env.npm_config_myVar, try process.env.npm_config_myvar
jakub.g's answer is correct, however an example using grunt seems a bit complex.
So my simpler answer:
- Sending a command line argument to an npm script
Syntax for sending command line arguments to an npm script:
npm run [command] [-- <args>]
Imagine we have an npm start task in our package.json to kick off webpack dev server:
"scripts": {
"start": "webpack-dev-server --port 5000"
},
We run this from the command line with npm start
Now if we want to pass in a port to the npm script:
"scripts": {
"start": "webpack-dev-server --port process.env.port || 8080"
},
running this and passing the port e.g. 5000 via command line would be as follows:
npm start --port:5000
- Using package.json config:
As mentioned by jakub.g, you can alternatively set params in the config of your package.json
"config": {
"myPort": "5000"
}
"scripts": {
"start": "webpack-dev-server --port process.env.npm_package_config_myPort || 8080"
},
npm start will use the port specified in your config, or alternatively you can override it
npm config set myPackage:myPort 3000
- Setting a param in your npm script
An example of reading a variable set in your npm script. In this example NODE_ENV
"scripts": {
"start:prod": "NODE_ENV=prod node server.js",
"start:dev": "NODE_ENV=dev node server.js"
},
read NODE_ENV in server.js either prod or dev
var env = process.env.NODE_ENV || 'prod'
if(env === 'dev'){
var app = require("./serverDev.js");
} else {
var app = require("./serverProd.js");
}
As of npm 2.x, you can pass args into run-scripts by separating with --
Terminal
npm run-script start -- --foo=3
Package.json
"start": "node ./index.js"
Index.js
console.log('process.argv', process.argv);
I had been using this one-liner in the past, and after a bit of time away from Node.js had to try and rediscover it recently. Similar to the solution mentioned by #francoisrv, it utilizes the npm_config_* variables.
Create the following minimal package.json file:
{
"name": "argument",
"version": "1.0.0",
"scripts": {
"argument": "echo \"The value of --foo is '${npm_config_foo}'\""
}
}
Run the following command:
npm run argument --foo=bar
Observe the following output:
The value of --foo is 'bar'
All of this is nicely documented in the npm official documentation:
https://docs.npmjs.com/using-npm/config
Note: The Environment Variables heading explains that variables inside scripts do behave differently to what is defined in the documentation. This is true when it comes to case sensitivity, as well whether the argument is defined with a space or equals sign.
Note: If you are using an argument with hyphens, these will be replaced with underscores in the corresponding environment variable. For example, npm run example --foo-bar=baz would correspond to ${npm_config_foo_bar}.
Note: For non-WSL Windows users, see #Doctor Blue's comments below... TL;DR replace ${npm_config_foo} with %npm_config_foo%.
Use process.argv in your code then just provide a trailing $* to your scripts value entry.
As an example try it with a simple script which just logs the provided arguments to standard out echoargs.js:
console.log('arguments: ' + process.argv.slice(2));
package.json:
"scripts": {
"start": "node echoargs.js $*"
}
Examples:
> npm start 1 2 3
arguments: 1,2,3
process.argv[0] is the executable (node), process.argv[1] is your script.
Tested with npm v5.3.0 and node v8.4.0
Most of the answers above cover just passing the arguments into your NodeJS script, called by npm. My solution is for general use.
Just wrap the npm script with a shell interpreter (e.g. sh) call and pass the arguments as usual. The only exception is that the first argument number is 0.
For example, you want to add the npm script someprogram --env=<argument_1>, where someprogram just prints the value of the env argument:
package.json
"scripts": {
"command": "sh -c 'someprogram --env=$0'"
}
When you run it:
% npm run -s command my-environment
my-environment
If you want to pass arguments to the middle of an npm script, as opposed to just having them appended to the end, then inline environment variables seem to work nicely:
"scripts": {
"dev": "BABEL_ARGS=-w npm run build && cd lib/server && nodemon index.js",
"start": "npm run build && node lib/server/index.js",
"build": "mkdir -p lib && babel $BABEL_ARGS -s inline --stage 0 src -d lib",
},
Here, npm run dev passes the -w watch flag to babel, but npm run start just runs a regular build once.
For PowerShell users on Windows
The accepted answer did not work for me with npm 6.14. Neither adding no -- nor including it once does work. However, putting -- twice or putting "--" once before the arguments does the trick. Example:
npm run <my_script> -- -- <my arguments like --this>
Suspected reason
Like in bash, -- instructs PowerShell to treat all following arguments as literal strings, and not options (E.g see this answer). The issues seems to be that the command is interpreted one time more than expected, loosing the '--'. For instance, by doing
npm run <my_script> -- --option value
npm will run
<my_script> value
However, doing
npm run <my_script> "--" --option value
results in
<my_script> "--option" "value"
which works fine.
This doesn't really answer your question but you could always use environment variables instead:
"scripts": {
"start": "PORT=3000 node server.js"
}
Then in your server.js file:
var port = process.env.PORT || 3000;
I've found this question while I was trying to solve my issue with running sequelize seed:generate cli command:
node_modules/.bin/sequelize seed:generate --name=user
Let me get to the point. I wanted to have a short script command in my package.json file and to provide --name argument at the same time
The answer came after some experiments. Here is my command in package.json
"scripts: {
"seed:generate":"NODE_ENV=development node_modules/.bin/sequelize seed:generate"
}
... and here is an example of running it in terminal to generate a seed file for a user
> yarn seed:generate --name=user
> npm run seed:generate -- --name=user
FYI
yarn -v
1.6.0
npm -v
5.6.0
Note: This approach modifies your package.json on the fly, use it if you have no alternative.
I had to pass command line arguments to my scripts which were something like:
"scripts": {
"start": "npm run build && npm run watch",
"watch": "concurrently \"npm run watch-ts\" \"npm run watch-node\"",
...
}
So, this means I start my app with npm run start.
Now if I want to pass some arguments, I would start with maybe:
npm run start -- --config=someConfig
What this does is: npm run build && npm run watch -- --config=someConfig. Problem with this is, it always appends the arguments to the end of the script. This means all the chained scripts don't get these arguments(Args maybe or may not be required by all, but that's a different story.). Further when the linked scripts are called then those scripts won't get the passed arguments. i.e. The watch script won't get the passed arguments.
The production usage of my app is as an .exe, so passing the arguments in the exe works fine but if want to do this during development, it gets problamatic.
I couldn't find any proper way to achieve this, so this is what I have tried.
I have created a javascript file: start-script.js at the parent level of the application, I have a "default.package.json" and instead of maintaining "package.json", I maintain "default.package.json". The purpose of start-script.json is to read default.package.json, extract the scripts and look for npm run scriptname then append the passed arguments to these scripts. After this, it will create a new package.json and copy the data from default.package.json with modified scripts and then call npm run start.
const fs = require('fs');
const { spawn } = require('child_process');
// open default.package.json
const defaultPackage = fs.readFileSync('./default.package.json');
try {
const packageOb = JSON.parse(defaultPackage);
// loop over the scripts present in this object, edit them with flags
if ('scripts' in packageOb && process.argv.length > 2) {
const passedFlags = ` -- ${process.argv.slice(2).join(' ')}`;
// assuming the script names have words, : or -, modify the regex if required.
const regexPattern = /(npm run [\w:-]*)/g;
const scriptsWithFlags = Object.entries(packageOb.scripts).reduce((acc, [key, value]) => {
const patternMatches = value.match(regexPattern);
// loop over all the matched strings and attach the desired flags.
if (patternMatches) {
for (let eachMatchedPattern of patternMatches) {
const startIndex = value.indexOf(eachMatchedPattern);
const endIndex = startIndex + eachMatchedPattern.length;
// save the string which doen't fall in this matched pattern range.
value = value.slice(0, startIndex) + eachMatchedPattern + passedFlags + value.slice(endIndex);
}
}
acc[key] = value;
return acc;
}, {});
packageOb.scripts = scriptsWithFlags;
}
const modifiedJSON = JSON.stringify(packageOb, null, 4);
fs.writeFileSync('./package.json', modifiedJSON);
// now run your npm start script
let cmd = 'npm';
// check if this works in your OS
if (process.platform === 'win32') {
cmd = 'npm.cmd'; // https://github.com/nodejs/node/issues/3675
}
spawn(cmd, ['run', 'start'], { stdio: 'inherit' });
} catch(e) {
console.log('Error while parsing default.package.json', e);
}
Now, instead of doing npm run start, I do node start-script.js --c=somethis --r=somethingElse
The initial run looks fine, but haven't tested thoroughly. Use it, if you like for you app development.
I find it's possible to just pass variables exactly as you would to Node.js:
// index.js
console.log(process.env.TEST_ENV_VAR)
// package.json
...
"scripts": { "start": "node index.js" },
...
TEST_ENV_VAR=hello npm start
Prints out "hello"
Separate your arguments using -- from the script and add all the required arguments, we can later access them by index.
npm run start -- myemail#gmail.com 100
You can get params in node using
const params = process.argv.slice(2);
console.log(params);
Output
['myemail#gmail.com', '100']
From what I see, people use package.json scripts when they would like to run script in simpler way. For example, to use nodemon that installed in local node_modules, we can't call nodemon directly from the cli, but we can call it by using ./node_modules/nodemon/nodemon.js. So, to simplify this long typing, we can put this...
...
scripts: {
'start': 'nodemon app.js'
}
...
... then call npm start to use 'nodemon' which has app.js as the first argument.
What I'm trying to say, if you just want to start your server with the node command, I don't think you need to use scripts. Typing npm start or node app.js has the same effort.
But if you do want to use nodemon, and want to pass a dynamic argument, don't use script either. Try to use symlink instead.
For example using migration with sequelize. I create a symlink...
ln -s node_modules/sequelize/bin/sequelize sequelize
... And I can pass any arguement when I call it ...
./sequlize -h /* show help */
./sequelize -m /* upgrade migration */
./sequelize -m -u /* downgrade migration */
etc...
At this point, using symlink is the best way I could figure out, but I don't really think it's the best practice.
I also hope for your opinion to my answer.
I know there is an approved answer already, but I kinda like this JSON approach.
npm start '{"PROJECT_NAME_STR":"my amazing stuff", "CRAZY_ARR":[0,7,"hungry"], "MAGICAL_NUMBER_INT": 42, "THING_BOO":true}';
Usually I have like 1 var I need, such as a project name, so I find this quick n' simple.
Also I often have something like this in my package.json
"scripts": {
"start": "NODE_ENV=development node local.js"
}
And being greedy I want "all of it", NODE_ENV and the CMD line arg stuff.
You simply access these things like so in your file (in my case local.js)
console.log(process.env.NODE_ENV, starter_obj.CRAZY_ARR, starter_obj.PROJECT_NAME_STR, starter_obj.MAGICAL_NUMBER_INT, starter_obj.THING_BOO);
You just need to have this bit above it (I'm running v10.16.0 btw)
var starter_obj = JSON.parse(JSON.parse(process.env.npm_config_argv).remain[0]);
Anyhoo, question already answered. Thought I'd share, as I use this method a lot.
I settled for something like this, look at the test-watch script:
"scripts": {
"dev": "tsc-watch --onSuccess \"node ./dist/server.js\"",
"test": "tsc && cross-env NODE_OPTIONS=--experimental-vm-modules NODE_NO_WARNINGS=1 jest",
"test-watch": "cross-env NODE_OPTIONS=--experimental-vm-modules NODE_NO_WARNINGS=1 tsc-watch --onSuccess",
},
You invoke the test-watch script like this:
// Run all tests with odata in their name
npm run test-watch "jest odata"
npm run script_target -- < argument > Basically this is the way of passing the command line arguments but it will work only in case of when script have only one command running like I am running a command i.e. npm run start -- 4200
"script":{
"start" : "ng serve --port="
}
This will run for passing command line parameters but what if we run more then one command together like npm run build c:/workspace/file
"script":{
"build" : "copy c:/file <arg> && ng build"
}
but it will interpreter like this while running copy c:/file && ng build c:/work space/file
and we are expected something like this
copy c:/file c:/work space/file && ng build
Note :- so command line parameter only work ad expected in case of only one command in a script.
I read some answers above in which some of them are writing that you can access the command line parameter using $ symbol but this will not gonna work
Try cross-env NPM package.
Easy to use. Easy to install. Cross all platform.
Example:
set arguments for command
// package.json
"scripts": {
“test”: “node test.js”,
“test-with-env-arg”: “cross-env YourEnvVarName=strValue yarn test,
}
get arguments from process.env
// test.js
const getCommandLineArg = Boolean(process.env.YourEnvVarName === 'true') // Attention: value of process.env.* is String type, not number || boolean
i had the same issue when i need to deploy to different environments
here is the package.json pre and post the updates.
scripts:
{"deploy-sit": "sls deploy --config resources-sit.yml",
"deploy-uat": "sls deploy --config resources-uat.yml",
"deploy-dev": "sls deploy --config resources-dev.yml"}
but here is the correct method to adopt the environment variables rather than repeating ourselves
scripts:{"deploy-env": "sls deploy --config resources-$ENV_VAR.yml"}
finally you can deploy by running
ENV_VAR=dev npm run deploy-env
I tried updating a Laravel project I'm working on today using composer update
But it hung on Updating dependencies (including require-dev)
So I tried things like updating composer, dump-autoload, but nothing seemed to work. Then I ran it in verbose mode: composer update -vvv
And I noticed it hung while reading this json:
Reading path/to/Composer/repo/https---packagist.org/provider-cordoval$hamcrest-php.json from cache
I tried searching for cordoval/hamcrest-php on packagist.org and couldn't find it. This isn't listed as a dependency in my composer.json
Searching through my vendor folder, I notice the mockery/mockery package I use requires hamcrest/hamcrest-php, but I can't find anything that makes any reference to cordoval.
Any idea what's wrong and how I can fix it so that I can do the update?
Here's my composer.json:
{
"name": "laravel/laravel",
"description": "The Laravel Framework.",
"keywords": ["framework", "laravel"],
"license": "MIT",
"require": {
"laravel/framework": "4.2.*",
"iron-io/iron_mq": "dev-master",
"phpunit/phpunit": "4.2.*",
"mockery/mockery": "dev-master",
"xethron/migrations-generator": "dev-master",
"mailgun/mailgun-php": "dev-master"
},
"autoload": {
"classmap": [
"app/commands",
"app/controllers",
"app/models",
"app/database/migrations",
"app/database/seeds",
"app/tests/TestCase.php"
]
},
"scripts": {
"post-install-cmd": [
"php artisan clear-compiled",
"php artisan optimize"
],
"post-update-cmd": [
"php artisan clear-compiled",
"php artisan optimize"
],
"post-create-project-cmd": [
"php artisan key:generate"
]
},
"config": {
"preferred-install": "dist"
},
"minimum-stability": "stable"
}
Update
I've tried removing some of the packages from my composer.json, including the "mockery/mockery" package. The only change it made was that Composer would hang on a different file.
After leaving Composer running like that for quite a long time, it finally exited with an error such as the following:
/path/to/ComposerSetup/bin/composer: line 18: 1356 Segmentation fault php "${dir}/composer.phar" $*
Not sure what to do about that...
In my case, it was simply taking a very long time on my 8GB ram Mac. To check the progress and verify that it is going through the dependencies, run composer in verbose mode. This was an approach I missed in the question so worth re-stating here.
composer update -vvv
So it turns out the problem was with php's xdebug extension. After disabling it in my php.ini, composer ran without any problems.
And just to note, the hang-up wasn't actually occurring while reading files from the cache. It was the step right after where composer was trying to resolve dependencies. It just never finished that step and never printed the output. That's why no matter what I did, it always appeared to be stuck reading a file from the cache.
1st of all : Check firewall and proxy connections. If everything is ok but composer is still hanging try to clear composer cache:
composer clear-cache
https://getcomposer.org/doc/03-cli.md#clear-cache
2nd option If these steps does not repair your composer then it's possible that the system does not have enough RAM memory available (I faced this problem and the symptomps were the same that you describe). At this point you have two options:
a) Increase memory (Virtual Machines or Docker) : Your container or VM needs more available memory. Follow this guide: https://stackoverflow.com/a/44533437/3518053
b) Generate swap file (Linux) : Try creating a swap file to provide more memory:
(Above commands are from composer killed while updating)
free -m
mkdir -p /var/_swap_
cd /var/_swap_
#Here, 1M * 2000 ~= 2GB of swap memory
dd if=/dev/zero of=swapfile bs=1M count=2000
mkswap swapfile
swapon swapfile
chmod 600 swapfile
echo "/var/_swap_/swapfile none swap sw 0 0" >> /etc/fstab
#cat /proc/meminfo
free -m
Some times it is stuck because it is trying to use HTTP instead of https so just run this
composer config --global repo.packagist composer https://packagist.org
Working for me.
First Run command for auto load, then clear cache and run update.
composer dump-autoload
php artisan cache:clear
php artisan view:clear
composer update
For me the issue was with xDebug. I was using IDE's terminal, and the debugger was listening to incoming connections (as always). Turning the listening off (without requiring to disable the extension) solved the issue.
this worked for me:
composer self-update
I solved it by running command NOT IN VS CODE TERMINAL
I found this in another article, I found that doing the following below worked. It seemed to be a cache/download issue into the composer packages cache.
composer update -vvv
Then doing the following:
Add or edit your composer file to have these settings.
"repositories": [
{
"type": "composer",
"url": "https://packagist.org"
},
{ "packagist": false }
]
Restart your system.
I faced the same problem today. Going by advice, turned off xdebug, but did not help. Verified all files were present. Restarted my system, and it worked.
Check if you are running the minimum required php version
Compare with the specified required php version in the composer.json file
Open terminal run
php -v
Cross check in composer.json file see example below
"require": {
"php": "^7.1.3",
}
check the path of [xdebug] zend_extension = "file/path" in php.ini
I solved it by editing php.ini file in order to set de cacert required for ssl verification:
Download the file http://curl.haxx.se/ca/cacert.pem
Edit php.ini to set the pat:
[openssl]
; The location of a Certificate Authority (CA) file on the local filesystem
; to use when verifying the identity of SSL/TLS peers. Most users should
; not specify a value for this directive as PHP will attempt to use the
; OS-managed cert stores in its absence. If specified, this value may still
; be overridden on a per-stream basis via the "cafile" SSL stream context
; option.
openssl.cafile=C:\web\certs\cacert.pem
curl.cainfo=C:\web\certs\cacert.pem
Try again
Personally, I discovered using free that my system had 0kb of swap storage. Creating a 1GB swap file using https://linuxize.com/post/create-a-linux-swap-file/ solved the problem instantly.
My problem solved with: Change the wifi (I use my phone) - Waiting (about 5 minutes)
Here is the output.
Creating a "magento/project-community-edition" project at "/tmp/exampleproject"
Installing magento/project-community-edition (2.4.5-p1)
- Installing magento/project-community-edition (2.4.5-p1): Loading from cache
Created project in /tmp/exampleproject
Loading composer repositories with package information
Warning from https://repo.packagist.org: Support for Composer 1 is deprecated and some packages will not be available. You should upgrade to Composer 2. See https://blog.packagist.com/deprecating-composer-1-support/
Info from https://repo.packagist.org: #StandWithUkraine
Updating dependencies (including require-dev)
After waiting I saw the following output:
Updating dependencies (including require-dev)
Package operations: 546 installs, 0 updates, 0 removals
- Installing laminas/laminas-dependency-plugin (2.4.0): Loading from cache
Not sure what is the reason but I also run the following commands.
To diagnose the problem you should run the following:
composer diagnose
If you get OK from each line (it should be a warning but not important I had to), That means there is no problem with the composer. Try to switch wifi and do not forget to wait!!!
Working with Elastic Beanstalk .config files is kinda... interesting. I'm trying to use environment properties with the files: configuration option in an Elastc Beanstalk .config file. What I'd like to do is something like:
files:
"/etc/passwd-s3fs" :
mode: "000640"
owner: root
group: root
content: |
${AWS_ACCESS_KEY_ID}:${AWS_SECRET_KEY}
To create an /etc/passwd-s3fs file with content something like:
ABAC73E92DEEWEDS3FG4E:aiDSuhr8eg4fHHGEMes44zdkIJD0wkmd
I.e. use the environment properties defined in the AWS Console (Elastic Beanstalk/Configuration/Software Configuration/Environment Properties) to initialize system configuration files and such.
I've found that it is possible to use environment properties in container-command:s, like so:
container_commands:
000-create-file:
command: echo ${AWS_ACCESS_KEY_ID}:${AWS_SECRET_KEY} > /etc/passwd-s3fs
However, doing so will require me to manually set owner, group, file permissions etc. It's also much more of a hassle when dealing with larger configuration files than the Files: configuration option...
Anyone got any tips on this?
How about something like this. I will use the word "context" for dev vs. qa.
Create one file per context:
dev-envvars
export MYAPP_IP_ADDR=111.222.0.1
export MYAPP_BUCKET=dev
qa-envvars
export MYAPP_IP_ADDR=111.222.1.1
export MYAPP_BUCKET=qa
Upload those files to a private S3 folder, S3://myapp/config.
In IAM, add a policy to the aws-elasticbeanstalk-ec2-role role that allows reading S3://myapp/config.
Add the following file to your .ebextensions directory:
envvars.config
files:
"/opt/myapp_envvars" :
mode: "000644"
owner: root
group: root
# change the source when you need a different context
#source: https://s3-us-west-2.amazonaws.com/myapp/dev-envvars
source: https://s3-us-west-2.amazonaws.com/myapp/qa-envvars
Resources:
AWSEBAutoScalingGroup:
Metadata:
AWS::CloudFormation::Authentication:
S3Access:
type: S3
roleName: aws-elasticbeanstalk-ec2-role
buckets: myapp
commands:
# commands executes after files per
# http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html
10-load-env-vars:
command: . /opt/myapp_envvars
Per the AWS Developer's Guide, commands "run before the application and web server are set up and the application version file is extracted," and before container-commands. I guess the question will be whether that is early enough in the boot process to make the environment variables available when you need them. I actually wound up writing an init.d script to start and stop things in my EC2 instance. I used the technique above to deploy the script.
Credit for the “Resources” section that allows downloading from secured S3 goes to the May 7, 2014 post that Joshua#AWS made to this thread.
I am gravedigging but since I stumbled across this in the course of my travels, there is a "clever" way to do what you describe–at least in 2018, and at least since 2016. You can retrieve an environment variable by key with get-config:
/opt/elasticbeanstalk/bin/get-config environment --key YOUR_ENV_VAR_KEY
And likewise all environment variables with (as JSON or --output YAML)
/opt/elasticbeanstalk/bin/get-config environment
Example usage in a container command:
container_commands:
00_store_env_var_in_file_and_chmod:
command: "/opt/elasticbeanstalk/bin/get-config environment --key YOUR_ENV_KEY | install -D /dev/stdin /etc/somefile && chmod 640 /etc/somefile"
Example usage in a file:
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/00_do_stuff.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/bash
YOUR_ENV_VAR=$(source /opt/elasticbeanstalk/bin/get-config environment --key YOUR_ENV_VAR_KEY)
echo "Hello $YOUR_ENV_VAR"
I was introduced to get-config by Thomas Reggi in https://serverfault.com/a/771067.
I assume that AWS_ACCESS_KEY_ID and AWS_SECRET_KEY are known to you prior to the app deployment.
You can create the file on your workstation and submit it to Elastic Beanstalk instance with the code on $ git aws.push
$ cd .ebextensions
$ echo 'ABAC73E92DEEWEDS3FG4E:aiDSuhr8eg4fHHGEMes44zdkIJD0wkmd' > passwd-s3fs
In .config:
files:
"/etc/passwd-s3fs" :
mode: "000640"
owner: root
group: root
container_commands:
10-copy-passwords-file:
command: "cat .ebextensions/passwd-s3fs > /etc/passwd-s3fs"
You might have to play with the permissions or execute cat as sudo. Also, I put the file into .ebextensions for example, it can be anywhere in your project.
Hope it helps.