Jmeter Variable from CSV - csv

I am running a Jmeter Test Plan with an HTTP Request, to test the performance of a web service.
In my test, I need a variable named REF to be changed in the body data of my HTTP Request.
REF can have 3000 values. So I have created a CSV file with all these 3000 values, a CSV Data Set Config, and a parameter in Jmeter named REF, and I use it in my HTTP Request like that :
<measure>
<measureRef id="${REF}"></measureRef>
<measureTime>${__time(yyyy-MM-dd'T'HH:mm:ss)}</measureTime>
<measureVal>
<value>${__Random(1,100,)}</value>
</measureVal>
</measure>
As you can see it in this XML, I use the functions time and Random of Jmeter to have different requests everytime I run the test plan.
I would like yo run the test plan automatically for all the 3000 values of REF. To do that, I tried to configure the thread group like this : Number of Threads = 3000 and Loop Control = 1.
The problem is that the test takes 3 minutes to be done. So I would like to know if there is another way to do. The thing is that I need the test to be done for all the differents 3000 values in my CSV, and I don't see another way to do. I tried to put another measure in my Body Data, like this :
<measure>
<measureRef id="${REF}"></measureRef>
<measureTime>${__time(yyyy-MM-dd'T'HH:mm:ss)}</measureTime>
<measureVal>
<value>${__Random(1,100,)}</value>
</measureVal>
</measure>
<measure>
<measureRef id="${REF}"></measureRef>
<measureTime>${__time(yyyy-MM-dd'T'HH:mm:ss)}</measureTime>
<measureVal>
<value>${__Random(1,100,)}</value>
</measureVal>
</measure>
But the value of REF that is used this the same in the two measures, and what I want is to have different values picked in the CSV.
I also tried to configure the Thread Group like this : Number of Threads = 1 and Loop Control = 3000. But it's not working, I have a lot of errors...
I want the test to be runned quickly because to test my web service, I'd like to inject the 3000 values every 1 minute (by using a Flow Control Action maybe). And if the test runs in 3 minutes, I would be too long, and all the values would not be tested...
Thank you for your help !

If you are using 2 data in 1 request then you can use max 1500 virtual users if you are not repeating your data.
If it is fine then add 2 columns in csv file like REF1 and REF2 and each column will have 1500 data. Then you can use ${REF1} and ${REF2} in your requests.

If it takes you 3 minutes to add 3000 values with 3000 users and you need this to be done in 1 minute I can think of 2 possible causes:
Your application cannot process the requests faster. You can try increasing number of threads i.e. to 9000 in the Thread Group and see whether it helps or not. If you still have this 3 minutes processing time - it's your application to blame
JMeter is not capable of sending requests fast enough.
First of all make sure to follow JMeter Best Practices
Then make sure to monitor the health of the machine where JMeter is running using i.e. JMeter PerfMon Plugin as JMeter must have enough headroom to operate
If a single machine cannot produce 3000 requests per minute - you will have to go for Distributed Testing

Related

Looking for an example of a OBD-II complete data frame

I'm developing an OBD-II reader where I want to query requests to read PID parameters with a stm32 processor. I already understand what should go on the data field, but the ID is giving me a headache. As I have read, one must send 0x7DF to broadcast a request, and each ECU will respond with his own ID. However, I have been asked to do this within the SAE J1939 protocol, which uses the 29 bit extended identifier, and I don't know what I need to add to this ID.
As I stated in the title, could someone show me some actual data from a bus using this method? I've been searching on the internet for real frames but did not have any luck so far.
I woud also appreciate if someone could shred some light to if the OBD-II communication needs some acknowledgment to work properly.
Thanks
I would suggest you to take a look on the SAE J1939 documentation, in the more specifically on the J1939/21,J1939-71 and J1939/73.
Generally, a J1939 transport protocol response sequence can be processed as follows:
Identify the BAM frame, indicating a new sequence being initiated
(via the PGN 60416 - 0xEC00 can be reach by 0x1CECFF00 )
Extract the J1939 PGN from bytes 6-8 of the BAM payload to use as the
identifier of the new frame
Construct the new data payload by concatenating bytes 2-8 of the data
transfer frames (i.e. excl. the 1st byte)
A J1939 data transfer messages with ID 1CEBFF00 (PGN 60160 or EB00).
Above, the last 3 bytes of the BAM equal E3FE00. When reordered, these equal the PGN FEE3 aka Engine Configuration 1 (EC1). Further, the payload is found by combining the the first 39 bytes across the 6 data transfer packets/fram
The administrative control device or any device issuing the vehicle use status PID should be sensitive to the run switch status (SPN 3046 - 0xFDC0 which probably can be reach by 0xCFDC000) and any other locally defined criteria for authorized use (i.e., driver log-ons) before the vehicle use status PID is used to generate an unauthorized use alarm.
Also, you can't forget to uses a read/send to extend ID message, since that is a 24-bit.
In fact, i will suggest you to use can-utils to make your a analyses even easier. A simple can-dump or can-sniffer you can see what is coming on your broadcast.
Some car's dbc https://github.com/commaai/opendbc

JMeter - How to run parallel execution with CSV as dataset?

What I am trying to accomplish => Run 50 threads in parallel using a CSV file as the dataset.
Here's how the CSV looks like (Let's say there are 50 records):
Username,Password
user1,password1
user2,password2
...,...
user50, password50
In JMeter, when I run my test case, each thread will consume 1 record of the CSV file in parallel. By that I mean, Thread 1 takes the first record (user1,password1), Thread 2 takes the second record (user2,password2), until the last record (50 in this example). And all of that happens at the same time.
I am still new to JMeter and I would like to know if this is something that is "doable" through this tool. If it is possible, your help is greatly appreciated! :)
Given default CSV Data Set Config setup which looks like:
each thread (virtual user) will take next line of the CSV file on each loop (iteration)
when the last line of the CSV file will be reached it will start over from the beginning
With regards to your "at the same time" requirement - the load pattern is controlled by Thread Group settings (number of threads, loops, ramp-up period), depending on your setup you will have concurrency from 0 to 50 active users, you can observe it using i.e. Active Threads Over Time listener
If you want to send 50 requests at exactly the same moment - consider using Synchronizing Timer
This is possible using CSV Data Set Config element. It reads the data row by row and username and password can be assigned to each thread. You can use the values with following syntax
Username ${Username}
password ${Password}
Also please note that you do not have to define the variable names in the CSV Dataset Config Element as you have them in the first row of the CSV file.

jmeter several http request reading from one csv data set config

Is it posible to have several http request read from one csv data set config?
I whant to make http request 1 to read from line 1 to 50 and http request 2 read from line 51 to 100 from the .csv file, and so on. Is this posible? or do i ahve to make more small csv files and more csv data set config.
Yes it is possible but generally not recommended. It will be much easier if smaller CSV's are used. Nevertheless you can make following changes for doing it through CSV data set config: -
Configure "Recycle on EOF" to False.
Configure "Stop thread on EOF" to False.
Sharing mode to "All threads".
Place HTTP requests in different thread group and set total number of threads to desired value. In your case the value for 1st and 2nd thread group should be 50. Also make sure that two thread groups does not start at same time. Add a startup delay for 2nd thread group.

Jmeter: How to map specific variable values from CSV file to specific thread-groups in a test plan

I have a test plan with 12 thread-groups, each one is one test scenario.I want to use unique login credentials for each thread-group. So I've created a CSV file, added CSV Data Config element to each thread-group and selected "All Threads" in "Sharing mode". Whenever I execute the test plan(All thread-groups concurrently) the thread-groups are not taking variable rows sequentially. I expected that the 1st thread-group in the test plan would consider 1st row of variables in the CSV file based on the post: JMeter test plan with different parameter for each thread
But it is not happening and I am unable to understand the pattern of variable allocation. Please help me resolve my issue.
My CSV file looks like below:
userName,password,message
userone,sample123,message1
usertwo,sample123,message2
.
.
so on...
Refer below for configuration of CSV Data Config element:
Thanks!
Threads and thread groups are different things. When you choose "All Threads" in "Sharing mode", it just means that all threads in the same thread group will share CSV. Thread groups are always independent.
You have 2 simple options:
Use one thread group and control what users are doing with controllers. For example Throughput Controller can allow you to control how many threads perform this or other script scenario within the same thread group.
Split your CSV so, that each thread group has its own CSV.
And many more complicated options, for example:
Use __CSVRead or __StringFromFile function, which allows to read one line. That way you can assign each thread group a range of lines to read, rather than reading the entire file.
If your usernames and passwords are predictable (e.g. user1, user2, etc), you could use a counter and a range for each thread group.

How to increase hadoop map tasks by implementing getSplits

I want to process multiline CSV files and for that I wrote a custom CSVInputFormat.
I would like to have about 40 threads processing CSV lines on each hadoop node. However, when I create a cluster on Amazon EMR with 5 machines (1 master and 4 cores), I can see I get only 2 map tasks running, even if there are 6 available map slots:
I implemented getSplits in my inputFormat so it would behave like NLineInputFormat. I was expecting with this I would get more thing running in parallel, but have had no effect. Also, I tried setting arguments -s,mapred.tasktracker.map.tasks.maximum=10 --args -jobconf,mapred.map.tasks=10, but no effect.
What can I do to have lines being processed in parallel? The way hadoop is running, it's not scalable, as doesn't matter how many instances I allocate to the cluster, only two map tasks will run at most.
UPDATE:
When I use a non compressed file (zip) as origin, it create more map tasks, about 17 for 1.3 million rows. Even so, I wonder why it shouldn't be more and why more mappers aren't created when data is zipped.
Change the split size to have more splits.
Configuration conf= new Cofiguration();
//set the value that increases your number of splits.
conf.set("mapred.max.split.size", "1020");
Job job = new Job(conf, "My job name");