I am trying to take a JSON input, complete some validation/transforms on it, and apply it as a terraform template. What would be most convenient is to pass the terraform template json (in env variables) definitions of resources, and have terraform apply those, however I cannot get this to work (I have a variables.tf file which uses the TF_VAR variables to set configuration, and am passing in JSON config per the terraform documetnation)
Is there a standard way of doing this? (Passing tf json definitions via env vars to terraform to apply?)
I'm not really sure about what the error is, since there is no code and no error log or example to be reproduced.
I will assume you already know how to pass and access environment variables (if not check this medium post), and you are receiving a large "string" inside terraform and want to read it as json.
The solution for it would be to use the functions jsondecode/jsonencode.
Just little reminder, when posting a question, if you paste the code with it, it makes it easier for people to understand what you are trying to do. The quality of answers will be higher.
Related
In my karate tests i need to write response id's to txt files (or any other file format such as JSON), was wondering if it has any capability to do this, I haven't seen otherwise in the documentation. In the case of no, is there a simple JavaScript function to do so?
Try the karate.write(value, filename) API but we don't encourage it. Also the file will be written only to the current "build" directory which will be target for Maven projects / stand-alone JAR.
value can be any data-type, and Karate will write the bytes (or plain-text) out. There is no built-in support for any other format.
Here is an example.
EDIT: for others coming across this answer in the future the right thing to do is:
don't write files in the first place, you never need to do this, and this question is typically asked by inexperienced folks who for some reason think that the only way to "save" a response before validation is to write it to a file. No, please don't waste your time - and please just match against the response. You can save it (or parts of it) to variables while you make other HTTP requests. And do not write your tests so that scenarios (or features) depend on other scenarios, this is a very bad practice. Also note that by default, Karate will dump all HTTP requests and responses in the log file (typically in target/karate.log) and also in the HTML report.
see if karate.write() works for you as per this answer
write a custom Java (or JS function that uses the JVM) to do what you want using Java interop
Also note that you can use karate.toCsv() to convert JSON into CSV if needed.
My justification for writing to a file is a different one. I am using karate explicitly to implement a mock. I want to expose an endpoint wherein the upstream system will send some basic data through json payload using POST/PUT method and karate will construct the subsequent payload file and stores it the specific folder, and this newly created payload file will be exposed through another GET call.
I have a CloudFormation template that consists of a Lambda function that reads messages from the SQS Queue.
Lambda function will read the message from the queue and transform it using a JSON template(Which I want it to be injected externally)
I will deploy different stacks for different products and for each product I will provide different JSON templates to be used for transformation.
I have different options but couldn't decide which one is better;
I can write all JSON files under the project and pack them together and pass related JSON name as a parameter to lambda.
I can store JSON files on S3 and pass S3 URL to lambda so I can read on runtime.
I can store JSON files on Dynamo DB and read from there using the same approach with 2
The first one seems like a better approach as I don't need to read from an external file on every lambda execution. But I will need to pack all templates together.
The last two are a more clear approach but require an external call to read JSON for every call.
Another approach could be (I'm not sure if it is possible) to inject a JSON file to Lambda on deploy from S3 bucket or sth. And Lambda function will read it like an environment variable.
As you can see from the cloudformation documentation Lambda environment variables can be only a Map of Strings, so the actual value you can pass to the function as an environment variable must be a String. You could pass your JSON as a string but the problem is that the max size for all environment variables is 4 KB.
If your templates are bigger and you don't want to call S3 or DynamoDB at runtime you could do a workaround like writing a simple shell script that copies the correct template file to the lambda folder before building and deploying the stack. This way the lambda gets deployed in a package with the code and only the desired json template.
I decided to go with S3 setup and also improved efficiency by storing Json on a global variable (after reading the first time). So I read once and use it for the lifetime of the Lambda container.
I'm not sure this is the best solution but works well enough for my scenario.
Please forgive my ignorance as I'm a jmeter noob. My webservice accepts JSON objects so I was able to write a rudimentary test where I create an HTTP Request with a JSON object in the "Post Body" portion of the http request.
Anyway, what I want to do is have the HTTP Request choose a different JSON object from a csv file or some other input mechanism so that I can randomize the types of queries that are being run during the load test. Is there a way to do this? The closest is probably using variables (section 4.11 in the user manual) but I have a feeling that's not how variables are used.
A second way I've theorized (although I haven't tried yet since I think the method above is easier) is to create a HTTP Request Default obj with a bunch of HTTP Requests with different JSON objects in them and then use a Random Controller to randomly go thru my multiple HTTP Requests on each pass.
If there's a third way, I'm all ears to learn how to use this tool. I'll continue to read and possibly experiment with plan B above. Thanks in advance for any help you can give me.
UPDATE: So I tried the second way and it seems to work. I had 3 different HTTP requests and the number of times each request gets hit varies from run to run. I still invite answers from the community since I'd like to see what the pros do for issues similar to mine.
You have partially answered your question yourself, by saying "csv file or". Here are the specifics.
You will have to use CSV data set config in your test plan to read data from CSV. In your post body, use the variables read from CSV.
Here is a screen cast showing how to use csv data set config.
As I said in the question, is it possible to pass jmeter variables (passed through -J) into a JUnit Request in order that it can be used internally to the junit test?
It sounds feasible as the Apache Junit Request docs indicate that:
... JMeter currently runs the test methods directly, rather than
leaving it to JUnit. ...
This would indicate that jmeter could control passing parameters but I've found no supporting documentation and there's no obvious mechanism through the JUnit Request sampler config.
Background
I was hoping to use CSV Data Set Config to load a user pool (as in Victor Klepikovskiy excellent top tips) and have a randomly obtained user provided to my unit-tests (for use across the thread/ loop). I've found an alternative way of doing what I want (but use my own user pool accessed from within the junit tests). I know you could manage the login through Http Request too (as above) but we amanage the rest-service calls internally to junit so that they can be run as junit tests as well. I was interested in being as non-intrusive as possible.
A second reason for doing so might be to have a single junit test method but have it parameterised to suit the scenario.
Addendum
For what it's worth, I'm happy with my current alternative approach but still interested in how you might be able to do this for future reference. I'm fairly new to jmeter but expect to see a fair bit more of it. Was also thinking of looking at the pre-processors but not sure how these could interface with the Junit Request.
I have tried to do exactly what you are doing (although, never specifically used -J).
The short answer is I have still not found a way to make it work to pass variables into a JUnit Request. I believe it is not currently supported.
At the time, the only way I saw of doing this was passing a parameter (e.g., ${__threadNum} or even from CSV Data Set Config) into the Constructor String Label and writing my constructor appropriately. However, I only got the string literal "${__threadNum}". I contacted support and they mentioned to me that this will not work. I forget exactly what they said and cannot find the email. But the main idea was that a JUnit Request is instantiated or created before the parameter in the Conostructor String Label is parsed. So that is why it will not work.
I am currently using the same alternative you are by having my test read credentials from a file.
I tested and it all works. what needs to be done is:
Source: blazemeter.com how-use-junit-jmeter
1) add the following jar files in the project libraries:
..\JMeter\apache-jmeter-3.0\lib\ext\ApacheJMeter_core.jar
..\JMeter\apache-jmeter-3.0\lib\ext\ApacheJMeter_junit.jar
2) in the class, add the reference to the sampler:
import org.apache.jmeter.protocol.java.sampler.JUnitSampler;
3) Add the code in the #Test tag to handle variables:
Click to see Code
4) Define variables in JMeter (both in user define variables and CSV)
Click to see JMeter Config
Hope it helps!
As I said in the question, is it possible to pass jmeter variables (passed through -J) into a JUnit Request in order that it can be used internally to the junit test?
It sounds feasible as the Apache Junit Request docs indicate that:
... JMeter currently runs the test methods directly, rather than
leaving it to JUnit. ...
This would indicate that jmeter could control passing parameters but I've found no supporting documentation and there's no obvious mechanism through the JUnit Request sampler config.
Background
I was hoping to use CSV Data Set Config to load a user pool (as in Victor Klepikovskiy excellent top tips) and have a randomly obtained user provided to my unit-tests (for use across the thread/ loop). I've found an alternative way of doing what I want (but use my own user pool accessed from within the junit tests). I know you could manage the login through Http Request too (as above) but we amanage the rest-service calls internally to junit so that they can be run as junit tests as well. I was interested in being as non-intrusive as possible.
A second reason for doing so might be to have a single junit test method but have it parameterised to suit the scenario.
Addendum
For what it's worth, I'm happy with my current alternative approach but still interested in how you might be able to do this for future reference. I'm fairly new to jmeter but expect to see a fair bit more of it. Was also thinking of looking at the pre-processors but not sure how these could interface with the Junit Request.
I have tried to do exactly what you are doing (although, never specifically used -J).
The short answer is I have still not found a way to make it work to pass variables into a JUnit Request. I believe it is not currently supported.
At the time, the only way I saw of doing this was passing a parameter (e.g., ${__threadNum} or even from CSV Data Set Config) into the Constructor String Label and writing my constructor appropriately. However, I only got the string literal "${__threadNum}". I contacted support and they mentioned to me that this will not work. I forget exactly what they said and cannot find the email. But the main idea was that a JUnit Request is instantiated or created before the parameter in the Conostructor String Label is parsed. So that is why it will not work.
I am currently using the same alternative you are by having my test read credentials from a file.
I tested and it all works. what needs to be done is:
Source: blazemeter.com how-use-junit-jmeter
1) add the following jar files in the project libraries:
..\JMeter\apache-jmeter-3.0\lib\ext\ApacheJMeter_core.jar
..\JMeter\apache-jmeter-3.0\lib\ext\ApacheJMeter_junit.jar
2) in the class, add the reference to the sampler:
import org.apache.jmeter.protocol.java.sampler.JUnitSampler;
3) Add the code in the #Test tag to handle variables:
Click to see Code
4) Define variables in JMeter (both in user define variables and CSV)
Click to see JMeter Config
Hope it helps!