I'm using Google Cloud Deployment and I am trying to get external input into my template. Namely, I want to set a metadata variable on my instance (when creating the instance) but provide this value on execution.
I've tried:
gcloud deployment-manager deployments create test-api-backend --config test-api-backend.yaml --properties 'my_value=hello'
Which fails (The properties flag should only be used when passing in a template as your config file.)
I've tried:
my_value=hello gcloud deployment-manager deployments create test-api-backend --config test-api-backend.yaml
And use {{env['my_value']}} but the value isn't picked up.
I guess I could add the property in a .jinja file and re-write this file before I run everything, but it feels like a hack. That, or my idea of passing a variable from shell into Deploy Manager is a hack. I'm honestly not sure.
As the error message indicates, the command line properties can only be used with a template. They are essentially meant to replace the config yaml file.
The easiest thing to do is to just rename your yaml file to a .py or .jinja file. Then use that template as the file in the gcloud command instead of the yaml file.
In that new template file, add any defaults you would like if you don't pass them in on the command line.
For python, something like:
if 'myparam' in context.properties:
valuetouse = context.properities['myparam']
else:
valuetouse = mydefaultvalue
If the template uses another template then you'll also need to create a schema file for the new, top level template so you can do the imports there instead of the yaml file.
See the schema file in this github example.
https://github.com/GoogleCloudPlatform/deploymentmanager-samples/blob/master/examples/v2/igm-updater/ha-service.py.schema
If you want, you can ignore all the properties and just do the imports section.
Related
I'm building a rails app that pulls data from Google Analytics using the Google Api Client Library for Ruby.
I'm using OAuth2 and can get everything working in development on a local machine. My issue is that the library uses a downloaded file, client_secrets.json, to store two secret keys.
Problem:I'm using Heroku and need a way to get the file to their production servers.
I don't want to add this file to my github repo as the project is public.
If there is a way to temporarily add the file to git, push to Heroku, and remove from git that would be fine. My sense is that the keys will be in the commits and very hard to prevent from showing on github.
Tried:
As far I can tell you cannot SCP a file to Heroku via a Bash console. I believe when doing this you get a new Dyno and anything you add would be only be temporary. I tried this but couldn't get SCP to work properly, so not 100% sure about this.
Tried:
I looked at storing the JSON file in an Environment or Config Var, but couldn't get it to work. This seems like the best way to go if anyone has a thought. I often run into trouble when ruby converts JSON into a string or hash, so possibly I just need guidance here.
Tried:
Additionally I've tried to figure out a way to just pull out the keys from the JSON file, put them into Config Vars, and add the JSON file to git. I can't figure out a way to put ENV["KEY"] in a JSON file though.
Example Code
The Google library has a method that loads the JSON file to create an authorization client. The client then fetches a token (or gives a authorization url).
client_secrets = Google::APIClient::ClientSecrets.load('client_secrets.json')
auth_client = client_secrets.to_authorization
** note that the example on google page doesn't show a filename because it uses a default ENV Var thats been set to a path
I figure this would all be a lot easier if the ClientSecrets.load() method would just take JSON, a string or a hash which could go into a Config Var.
Unfortunately it always seems to want a file path. When I feed it JSON, a string or hash, it blows up. I've seen someone get around the issue with a p12 key here, but I'm not sure how to replicate that in my situation.
Haven't Tried:
My only other though (aside from moving to AWS) is to put the JSON file on AWS and have rails pull it when needed. I'm not sure if this can be done on the fly or if the file would need to be pulled down when the rails server boots up. Seems like too much work, but at this point I've spend a few hours on it so ready to attempt.
This is the specific controller I am working on:
https://github.com/dladowitz/slapafy/blob/master/app/controllers/welcome_controller.rb
By search github I found that someone had used a different method that used a JSON string as an argument rather than a file path: Google::APIClient::ClientSecrets.new(JSON.parse(ENV['GOOGLE_CLIENT_SECRETS']))
This lets me wrap up the JSON into an ENV VAR. The world makes sense again.
As discussed in this thread, rather than supplying a path to a json key file you can set three ENV variables instead:
GOOGLE_ACCOUNT_TYPE=service_account
GOOGLE_PRIVATE_KEY=XXX
GOOGLE_CLIENT_EMAIL=XXX
Source here.
I ran into this same problem using Google API. I ended up using openssl to assign a new very secret passphrase to the p12 file, storing that new file in the repo, and then putting the passphrase into app secrets and on Heroku env variables.
This way, the file is in the repo but it can't be accessed/read without the passphrase.
This post was helpful in changing the default google p12 passphrase from 'notasecret' to something secure.
def authorize!
#client.authorization = Signet::OAuth2::Client.new(
#...
:signing_key => key
)
end
def key
Google::APIClient::KeyUtils.load_from_pkcs12(key_path, ENV.fetch('P12_PASSPHRASE'))
end
def key_path
"#{Rails.root}/config/google_key.p12"
end
Using Rails 7, I encrypted the JSON credentials like so
I first ran bin/rails credentials:edit -e development
Then added my credentials:
omniauth:
google_oauth2:
client_secrets: {"web":{"client_id":"my-client-id","project_id":"my-project-id","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://oauth2.googleapis.com/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_secret":"my-client-secret","redirect_uris":["http://localhost:3000/users/auth/google_oauth2/callback","http://localhost:3000/contacts/gmail/callback"]}}
The I used it with ClientSecrets like so:
def client_secrets
#client_secrets ||= Google::APIClient::ClientSecrets.new(client_secrets_json)
end
def client_secrets_json
Rails.application.credentials.dig(:omniauth, :google_oauth2, :client_secrets)
end
I created a jmeter project with the included controller, which calls another jmx.
I need to dynamically construct the path using a csv in which there are the names of the jmx children to be called. For example C: performance $ {JmxName}.
I tried using the CSV Data Set Config but the included controller does not recognize the variable (Error displayed: File "C:\performance\${JmxName}" not found for Include Controller "Include Controller")
do you have ideas on how to reach the goal?
thank you in advance
According to the documentation:
This element does not support variables/functions in the filename field
So you will not be able to achieve this using Include Controller.
The options are in:
Include your "children" JMX scripts into the main JMX script and refer them via Module Controller where required. If you need to choose this or that scenario basing on the value of the variable from the CSV file - put the Module Controller's under the Switch Controller
If you cannot include children JMX scripts into the main one you can kick off them as separate processes using i.e. OS Process Sampler. The test results can be combined after the execution using i.e. Merge Results tool
I'm still pretty new to running anything in PyCharm more advanced than just a simple script. I'm writing a test in pytest right now and I want to have the test results output to a junit xml file; I'm thinking the best naming convention will be based on the current date/time, so I am trying to pipe in the current date using the date shell command as an environment variable as seen below:
Current Configuration:
However, when I run the configuration as-is, it just names the .xml file based on the command without actually executing it. Any ideas what I'm missing, or if this is even possible?
Thanks!
Yes, it is possible with a workaround. I don't think what you are trying to achieve is possible using a single configuration. The the value you set in Environment variables are substituted as-is and wouldn't be executed in bash prior to that.
The workaround would be use multiple configurations.
Store the following line in a bash file.
export PYTEST_EXEC_TIME=$(date '+%Y-%m-%d%H:%M:%S')
Add a bash configuration to which executes this file.
Add that configuration to the pytest configuration as a "Before Launch" configuration and use the $PYTEST_EXEC_TIME in the additional parameters.
Note: Here is a detailed answer showing step by step process of setting up a "Before Launch" configuration.
I am trying to use CSV Data Set Config to get some data from csv file to be used in jmeter script but i don't want to hardcode the file path as it will be changing according to the test environment. Is there a way i can pick this path from System properties i.e some export set in my bashrc file.
Export in my bashrc :
export NIMBUS4_PERFORMANCE_TEST_REPO=/Users/rahul/Documents/verecloud/performancetest/data/user.csv
I would suggest the following workaround:
Change "Filename" setting of the CSV Data Set Config to following:
${__BeanShell(System.getenv().get("NIMBUS4_PERFORMANCE_TEST_REPO"))}
Where:
System.getenv() - method which provides access to underlying operating system environment variables
__Beanshell() - JMeter built-in function which allows executing of arbitrary Beanshell code
you could create a softlink at some static path. For example,
say we have created a soft link to /user/data/csvs folder.
You are in say ~/Documents , there run below
ln -s /user/data/csvs
Now we can access it in the jmeter and you will also have the flexibility to modify the softlink to point to some other location too.
Only constraint i see is the pointed directory name shouldn't change.
Hope this will help!!!
You can have just users.csv if the file is in the same folder as the .jmx itself;
You can have ${location}\users.csv
And in your UserDefinedVariables you'll have
and in non-gui mode you'll refer as
%RUNNER_HOME%\Test.jmx -Jloc=%RUNNER_HOME%\users.csv -Jusers=100 -Jloop=1 -Jrampup=5
I have two scripts in the pre-build step in a Jenkins job, the first one a perl script, the second a system groovy script using the groovy plugin. I need information from the first perl script in my second groovy script. I think the best way would be to set some environment variable, and was wondering how that can be realized.
Or any other better way.
Thanks for your time.
The way to propagate environment variables among build steps is via EnvInject Plugin.
Here are some previous answers that show how to do it:
How to set environment variables in Jenkins?
Jenkins : Report results of intermediate [windows batch] build steps in email body
In your case, however, it may be simpler just to write to a file in one build step and read that file in another. To make sure you do not accidentally read from a previous version of the file you can incorporate BUILD_ID in the file name.
Using EnvInject Plugin from job configuration you should use Inject environment variables to the build process / Evaluated Groovy script.
Depending on the setup you may execute Groovy or shell command and save it in map containing environment variables:
Example
By either getting command result with execute method:
return [DATE: 'date'.execute().text]
or with Groovy equivalent if one exists:
return [DATE: new Date()]