Load testing with Cucumber and Java - cucumber-junit

Question

I need to perform load testing for my REST web service using Cucumber and Java. This REST web service accepts one input which is a String called id and returns complex JSON object.
I wrote a .feature file with Given, When and Then annotations which are defined in java.
The skeleton definition of the class and annotations are here under.
1) Feature (UserActivity.feature)
#functional #integration
Feature: User System Load Test
Scenario Outline: Load test for user data summary from third party UserSystem
Given Simultaneously multiple users are hitting XYZ services with an id=<ids>
When I invoke third party link with above id for multiple users simultaneously
Then I should get response code and response message for all users
Examples:
| ids |
| "pABC123rmqst" |
| "fakXYZ321rmv" |
| "bncMG4218jst" |
2) LoadTestStepDef.java (Feature definition)
package com.system.test.cucumber.steps;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.junit.runners.model.InitializationError;
import com.system.test.restassured.LoadTestUtil;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
public class LoadTestStepDef
{
private static Logger LOG = LogManager.getLogger( LoadTestStepDef.class );
private String id = null;
private LoadTestUtil service = null;
#Given("^Simultaneously multiple users are hitting XYZ services with an a id=\"(.*?)\"$" )
public void Simultaneously_multiple_users_are_hitting_XYZ_services_with_a_id( String id )
{
LOG.debug( "ID {}", id );
LOG.info( "ID {}", id );
this.id = id;
}
#When( "^I invoke third party link with above id for multiple users simultaneously$" )
public void invoke_third_party_link_With_Above_ID_for_multiple_users_simultaneously() throws InitializationError
{
LOG.debug( " *** Calling simulatenously {} ", id );
LOG.info( " *** Calling simulatenously {}", id );
//Create object of service
service = new LoadTestUtil();
//Set the id to the created service and invoke method
service.setData(id);
service.invokeSimultaneosCalls(10);
}
#Then( "^I should get response code and response message for all users$" )
public void Should_get_response_code_and_response_message_for_all_users()
{
LOG.info( "*** Assert for response Code" );
service.assertHeaderResponseCodeAndMessage();
}
}
3) LoadTestUtil.java
package com.system.test.restassured;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import java.util.concurrent.Callable;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import com.jayway.restassured.path.json.JsonPath;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.FutureTask;
import java.util.concurrent.TimeUnit;
public class LoadTestUtil
{
private String id = null;
private int numberofTimes;
//Create List to hold all Future<Long>
private List<JsonPath> jsonResponseList = new ArrayList<JsonPath>();
//No arg Constructor
public LoadTestUtil()
{
}
//Set data method to set the initial id
public void setData(String id)
{
LOG.info( "LoadTestUtil.setData()", id );
this.id = id;
}
//This method is used call the REST webservice N times using threads and get response
public void invokeSimultaneosCalls(int numberofTimes)
{
LOG.info( "LoadTestUtil.invokeSimultaneosCalls() - Start" );
this.numberofTimes = numberofTimes;
try
{
long start = System.nanoTime();
int numberOfThreads = Runtime.getRuntime().availableProcessors();
LOG.info("Number of processor available {}" , numberOfThreads);
//Create pool for the Executor Service with numberOfThreads.
ExecutorService executor = Executors.newFixedThreadPool(numberOfThreads);
//Create a list to hold the Future object associated with Callable
List<Future<JsonPath>> futureList = new ArrayList<Future<JsonPath>>();
//Create new RESTServiceCallTask instance
Callable<JsonPath> callable = new RESTServiceCallTask(id);
Future<JsonPath> future = null;
//Iterate N number of times to submit the callable object
for(int count=1; count<=numberofTimes;count++)
{
//Submit Callable tasks to the executor
future = executor.submit(callable);
//Add Future to the list to get return value using Future
futureList.add(future);
}
//Create a flag to monitor the thread status. Check whether all worker threads are completed or not
boolean threadStatus = true;
while (threadStatus)
{
if (future.isDone())
{
threadStatus = false;
//Iterate the response obtained from the futureList
for(Future<JsonPath> futuree : futureList)
{
try
{
//print the return value of Future, notice the output delay in console
// because Future.get() waits for task to get completed
JsonPath response = futuree.get();
jsonResponseList.add(response);
}
catch(InterruptedException ie)
{
ie.printStackTrace();
}
catch(ExecutionException ee)
{
ee.printStackTrace();
}
catch(Exception e)
{
e.printStackTrace();
}
}//End of for to iterate the futuree list
} //End of future.isDone()
} //End of while (threadStatus)
//shut down the executor service now
executor.shutdown();
//Calculate the time taken by the threads for execution
executor.awaitTermination(1, TimeUnit.HOURS); // or longer.
long time = System.nanoTime() - start;
logger.info("Tasks took " + time/1e6 + " ms to run");
long milliSeconds = time / 1000000;
long seconds, minutes, hours;
seconds = milliSeconds / 1000;
hours = seconds / 3600;
seconds = seconds % 3600;
seconds = seconds / 60;
minutes = seconds % 60;
logger.info("Task took " + hours + " hours, " + minutes + " minutes and " + seconds + " seconds to complete");
} //End of try block
catch (Exception e)
{
e.printStackTrace();
}
LOG.info("LoadTestUtil.invokeSimultaneosCalls() - jsonResponseList {} " , jsonResponseList);
System.out.println("LoadTestUtil.invokeSimultaneosCalls() - jsonResponseList {} " + jsonResponseList);
LOG.info( "*** LoadTestUtil.invokeSimultaneosCalls() - End" );
}
public void assertHeaderResponseCodeAndMessage(){
//Number of response objects available
int size = jsonResponseList.size();
LOG.info("Number of REST service calls made = ", size);
for(JsonPath jsonResponse : jsonResponseList)
{
String responseCode = jsonResponse.get( "header.response_code").toString();
String responseMessage = jsonResponse.get( "header.response_message").toString();
assertEquals( "200", responseCode);
assertEquals( "success", responseMessage);
}
}
}
4) RESTServiceCallTask.java
This class implements Callable and override the call() method.
In the call() method, the response in the form of JsonPath is returned for each call
package com.system.test.restassured;
import static com.jayway.restassured.RestAssured.basePath;
import static com.jayway.restassured.RestAssured.baseURI;
import static com.jayway.restassured.RestAssured.given;
import static com.jayway.restassured.RestAssured.port;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import com.system.test.restassured.TestUtil;
import com.jayway.restassured.path.json.JsonPath;
import com.jayway.restassured.response.Response;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
public class RESTServiceCallTask implements Callable<JsonPath>
{
private static Logger LOG = LogManager.getLogger(RESTServiceCallTask.class);
private Response response = null;
private String id;
private String environment;
//private JsonPath jsonPath;
/**
* Constructor initializes the call to third party system
*
* #param id
*/
public RESTServiceCallTask(String id)
{
LOG.info("In RESTServiceCallTask() constructor ");
this.id = id;
//Read the environment variable ENV to get the corresponding environment's REST URL to call
this.environment = System.getProperty("ENV");
baseURI = TestUtil.getbaseURL(environment);
basePath = "/bluelink/tracker/member_summary";
port = 80;
LOG.info(" *** Environment : {}, URI: {} and Resource {} ", environment, baseURI, basePath);
}
//This method is called by the threads to fire the REST service and returns JSONPath for each execution
#Override
public JsonPath call() throws Exception
{
LOG.info(" *** In call() method ");
try
{
response = given().headers("id", this.id).log().all().get();
} catch (Exception e)
{
LOG.error("System Internal Server Error", e);
}
String strResponse = this.response.asString();
LOG.info("Response : {}", strResponse);
JsonPath jsonResponse = new JsonPath(strResponse);
return jsonResponse;
}
}
5) TestUtil.java
This utility class is used to get the REST URL corresponding to the passed environment
package com.system.test.restassured;
import java.util.HashMap;
import java.util.Map;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
public class TestUtil
{
private static Logger LOG = LogManager.getLogger(TestUtil.class);
private static final Map<String, String> ENVIRONMENT_MAP = new HashMap<String, String>();
static
{
ENVIRONMENT_MAP.put("LOCAL", "http://localhost:9080");
ENVIRONMENT_MAP.put("ENV1", "http://localhost:9080");
ENVIRONMENT_MAP.put("ENV2", "http://localhost:9080");
ENVIRONMENT_MAP.put("ENV3", "http://localhost:9080");
}
public static String getbaseURL(String environment)
{
LOG.info("Environment value fetched = {}", environment);
return ENVIRONMENT_MAP.get(environment);
}
}
The problem here is that the multi-threading feature is not getting executed.
I used the MavenSurefire Plugin and tried with parallel classes and methods. In those cases also the above scenario doesn't work.
Does Cucumber support java multi-threading? If so what is wrong with the above feature definition?
Note - The same task is performed with stand alone program and able to run for 10,000 times
using 4 threads without any issues. However not able to run the above code for 2000 times using Maven. With 2000 times, the system crashed abruptly.
I am using Rational Application Developer 8.5, Websphere Server 8.0 with Maven 3.x for the above setup.
Thanks for your response.