I have to migrate around millions of blob records from multiple mysql databases to a physical location as files over WAN network.
I chose to use Spring Batch and has already made it work. However, I am struggling with a timeout error happen with random partitioned steps.
Here is some context,
There are multiple MySql database store >10m records in 20 years.
The source tables indexed two composite keys in varchar datatype (there is no ID key) so I have to use an UN-indexed column in date-time format to partitioning the records by year and week to keep the number of records per partition reasonably at average 200 records. If there is any better advice, it would be welcome!
My issue: When the records per partition is high enough, the stepExecutors will randomly failed due to time out
Could not open JDBC Con nection for transaction; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms
I have done some tweaks with the DataSource properties and Transaction properties but no luck. Can I get some advice please! Thanks
Terminal log:
org.springframework.transaction.CannotCreateTransactionException: Could not open JDBC Con
nection for transaction; nested exception is
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doBegin(DataSourceTransactionManager.java:309)
~[spring-jdbc-5.3.16.jar:5.3.16]
...
Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.
2022-03-05 10:05:43.146 ERROR 15624 --- [main] o.s.batch.core.step.AbstractStep : Encountered an error executing step managerStep in job mainJob
org.springframework.batch.core.JobExecutionException: Partition handler returned an unsuccessful step at ...
The job is marked as [FAILED] or [UNKNOWN] sometimes, and not restartable.
org.springframework.batch.core.partition.support.PartitionStep.doExecute(PartitionStep.java:112) ~[spring-batch-core-4.3.5.jar:4.3.5]
2022-03-05 10:05:43.213 INFO 15624 --- [main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=mainJob]] completed with the following parameters: [{run.id=20}] and the following status: [FAILED] in 3m13s783ms
2022-03-05 10:05:43.590 INFO 15624 --- [SpringApplicationShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - Shutdown initiated...
2022-03-05 10:05:43.624 INFO 15624 --- [SpringApplicationShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - Shutdown completed.
2022-03-05 10:05:43.626 INFO 15624 --- [SpringApplicationShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2022-03-05 10:05:43.637 INFO 15624 --- [SpringApplicationShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
Datasource builder: I have tried to increase the connection timeout and pool size, but it seems not be applied.
#Bean(name = "srcDataSource")
// #ConfigurationProperties(prefix = "spring.datasource.hikari")
public HikariDataSource dataSource() {
HikariDataSource hikariDS = new HikariDataSource();
hikariDS.setDriverClassName("com.mysql.jdbc.Driver");
hikariDS.setJdbcUrl("jdbc:mysql://dburl");
hikariDS.setUsername("dbuser");
hikariDS.setPassword("dbpwd");
// properties below does not solve the problem
hikariDS.setMaximumPoolSize(16);
hikariDS.setConnectionTimeout(30000);
// hikariDS.addDataSourceProperty("serverName",
// getConfig().getString("mysql.host"));
// hikariDS.addDataSourceProperty("port", getConfig().getString("mysql.port"));
// hikariDS.addDataSourceProperty("databaseName",
// getConfig().getString("mysql.database"));
// hikariDS.addDataSourceProperty("user", getConfig().getString("mysql.user"));
// hikariDS.addDataSourceProperty("password",
// getConfig().getString("mysql.password"));
// hikariDS.addDataSourceProperty("autoReconnect", true);
// hikariDS.addDataSourceProperty("cachePrepStmts", true);
// hikariDS.addDataSourceProperty("prepStmtCacheSize", 250);
// hikariDS.addDataSourceProperty("prepStmtCacheSqlLimit", 2048);
// hikariDS.addDataSourceProperty("useServerPrepStmts", true);
// hikariDS.addDataSourceProperty("cacheResultSetMetadata", true);
return hikariDS;
}
ManagerStep:
#Bean
public Step managerStep() {
return stepBuilderFactory.get("managerStep")
.partitioner(workerStep().getName(), dateRangePartitioner())
.step(workerStep())
// .gridSize(52) // number of worker, which is not necessary with datepartition
.taskExecutor(new SimpleAsyncTaskExecutor())
.build();
}
WorkerStep: I also tried to increase the transaction properties timeout, but not luck
#Bean
public Step workerStep() {
DefaultTransactionAttribute attribute = new DefaultTransactionAttribute();
attribute.setPropagationBehavior(Propagation.REQUIRED.value());
attribute.setIsolationLevel(Isolation.DEFAULT.value());
// attribute.setTimeout(30);
attribute.setTimeout(1000000);
return stepBuilderFactory.get("workerStep")
.<Image, Image>chunk(10)
.reader(jdbcPagingReader(null))
.processor(new ImageItemProcessor())
.writer(imageConverter())
// .listener(wrkrStepExecutionListener)
.transactionAttribute(attribute)
.build();
}
Job builder:
#Bean
public Job mainJob() {
return jobBuilderFactory.get("mainJob")
// .incrementer(new RunIdIncrementer())
.start(managerStep())
// .listener()
.build();
}
io.restassured.path.json.exception.JsonPathException: Failed to parse
the JSON document
It is throwing error in the code written for these Feature file steps:
Given Add new location with "<name>" "<language>" "<address>" parameters
Then user calls "AddPlaceAPI" with http "POST" method
Then API successful response with "status" as "OK"
The stepDefinition code for these steps
#Given("Add new location with {string} {string} {string} parameters")
public void Add_new_location_with_three_parameters(String name_G,String address_G,String language_G) throws IOException {
res=given().spec(requestSpecification())
.body(data.googleAddPlace(name_G,address_G,language_G));
}
#Then("user calls {string} with http {string} method")
public void user_calls_with_http_method(String resource_G,String http_method)
{
APIResources resourceAPI = APIResources.valueOf(resource_G);
System.out.println("Resource value" + resourceAPI.getResource());
resspec =new ResponseSpecBuilder().expectStatusCode(200).expectContentType(ContentType.JSON).build();
if(http_method.equalsIgnoreCase("POST"))
response= res.when().post(resourceAPI.getResource());
else if(http_method.equalsIgnoreCase("GET"))
response= res.when().post(resourceAPI.getResource());
}
#Then("API successful response with {string} as {string}")
public void api_successful_response_with_as(String string, String string2) {
// Write code here that turns the phrase above into concrete actions
assertEquals(getJsonPath(response,string),string2);
}
getJsonPath method Definition where it is throwing Error
public String getJsonPath(Response response,String key)
{
String resp=response.asString();
JsonPath js = new JsonPath(resp);
return js.get(key).toString();
}
enter image description here
Junit Error Log
io.restassured.path.json.exception.JsonPathException: Failed to parse the JSON document
at io.restassured.path.json.JsonPath$ExceptionCatcher.invoke(JsonPath.java:1002)
at io.restassured.path.json.JsonPath$4.doParseWith(JsonPath.java:967)
at io.restassured.path.json.JsonPath$JsonParser.parseWith(JsonPath.java:1047)
at io.restassured.path.json.JsonPath.get(JsonPath.java:202)
at resources.Utils.getJsonPath(Utils.java:56)
at stepDefinations.stepDefination.api_successful_response_with_as(stepDefination.java:117)
at ✽.API successful response with "status" as "OK"(file:///C:/Users/gaura/eclipse-workspaceNew/APIFramework/src/test/java/features/googlemapsValidation.feature:6)
Caused by: groovy.json.JsonException: Lexing failed on line: 1, column: 1, while reading '<', no possible valid JSON value or
punctuation could be recognized.
at groovy.json.JsonLexer.nextToken(JsonLexer.java:86)
at groovy.json.JsonLexer$nextToken.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:130)
at io.restassured.internal.path.json.ConfigurableJsonSlurper.parse(ConfigurableJsonSlurper.groovy:97)
at io.restassured.internal.path.json.ConfigurableJsonSlurper$parse.callCurrent(Unknown
Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:171)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
at io.restassured.internal.path.json.ConfigurableJsonSlurper.parseText(ConfigurableJsonSlurper.groovy:83)
at io.restassured.path.json.JsonPath$4$1.method(JsonPath.java:965)
at io.restassured.path.json.JsonPath$ExceptionCatcher.invoke(JsonPath.java:1000)
at io.restassured.path.json.JsonPath$4.doParseWith(JsonPath.java:967)
at io.restassured.path.json.JsonPath$JsonParser.parseWith(JsonPath.java:1047)
at io.restassured.path.json.JsonPath.get(JsonPath.java:202)
at resources.Utils.getJsonPath(Utils.java:56)
at stepDefinations.stepDefination.api_successful_response_with_as(stepDefination.java:117)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown
Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at io.cucumber.java.Invoker.invoke(Invoker.java:27)
at io.cucumber.java.JavaStepDefinition.execute(JavaStepDefinition.java:27)
at io.cucumber.core.runner.PickleStepDefinitionMatch.runStep(PickleStepDefinitionMatch.java:63)
at io.cucumber.core.runner.TestStep.executeStep(TestStep.java:64)
at io.cucumber.core.runner.TestStep.run(TestStep.java:49)
at io.cucumber.core.runner.PickleStepTestStep.run(PickleStepTestStep.java:46)
at io.cucumber.core.runner.TestCase.run(TestCase.java:51)
at io.cucumber.core.runner.Runner.runPickle(Runner.java:66)
at io.cucumber.junit.PickleRunners$NoStepDescriptions.run(PickleRunners.java:149)
at io.cucumber.junit.FeatureRunner.runChild(FeatureRunner.java:83)
at io.cucumber.junit.FeatureRunner.runChild(FeatureRunner.java:24)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at io.cucumber.junit.Cucumber.runChild(Cucumber.java:185)
at io.cucumber.junit.Cucumber.runChild(Cucumber.java:83)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at io.cucumber.junit.Cucumber$RunCucumber.evaluate(Cucumber.java:219)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:89)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:541)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:763)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:463)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:209)
I tried combination of following versions for required jar files in pom.xml file and its working fine for me. You can also try the same:
JUnit- version: 4.1.2
Cucumber - java - info.cukes - version: 1.2.4
Cucumber - junit - info.cukes - version: 1.2.3
Cucumber - pico container - info.cukes - version: 1.2.4
Cucumber - core - info.cukes - version: 1.2.6
Cucumber - jvm deps - info.cukes - version: 1.0.5
Rest Assurd - version: 3.0.2
Currently, I am trying to update the JSON response node value using BeanShell PreProcessor.
Need help in converting JSON response in Object and then updating the node value and then stringify.
I have the following JSON and I want to update the OrderTypeId and IsCredentialOff values using BeanShell PreProcessor in Jmeter. I have tried a solution in postman which worked well. whereas getting difficulty in BeanShell Scripting.
{"DeliveryCharge":0.0,"ApplyCouponToCart":true,"CartId":6970,"SessionId":"DB84A55E-82A9-4BC9-98E8-2A845FB6893A","StoreId":10061,"UserId":10011690,"AppId":10061,"ListCartItem":[{"CartItemId":55513,"UserId":0,"StoreId":0,"AppId":0,"CartId":6970,"PID":772544,"Pack":1,"ProductId":5712,"ProductImage":"https://s3.amazonaws.com/images.liquorapps.com/sm/000457-TWO-FINGERS-GOLD-TEQUILA.png","ProductName":"TWO FINGERS GOLD","UnitSize":"750 ML","Quantity":1,"UPC":"89540135328","SKU":"1014","CategoryId":2,"TypeId":15,"VarietalId":0,"UnitSizeId":12,"CountryId":7,"RegionId":32,"Price":14.99,"PriceDisplay":"$14.99","OfferPrice":14.99,"OfferPriceDisplay":"$14.99","FinalPrice":14.99,"FinalPriceDisplay":"$14.99","TaxRate":0.066250,"QuantityOrdered":1,"Remark":"","ItemTotal":14.99,"ItemTotalDisplay":"$14.99","FinalItemTotal":14.99,"FinalItemTotalDisplay":"$14.99","ItemTotalSaving":0.00,"ItemTotalSavingDisplay":"","ListDiscount":[],"ListCharge":[],"CartItemCount":0,"Deposit":0.00,"DealId":0,"DealInventory":0,"DealInventoryMessage":"","IsBottleLimitAtRetail":false,"TotalQuantity":1,"IsOfferItem":false,"SuccessMessage":"","ErrorDetail":"","ErrorMessage":"","MessageType":"","MessageTitle":"","TextType":""}],"ListDiscount":[],"ListCharge":[],"CouponCode":"","OrderTypeId":1,"AddressId":0,"PaymentTypeId":0,"TipForDriver":0.00,"ListTipForDriver":[],"SubTotal":14.99,"SubTotalDisplay":"$14.99","TotalSavings":0.0,"TotalSavingsDisplay":"-$0.00","TotalCharges":0.0,"TotalChargesDisplay":"$0.00","TotalValue":14.99,"TotalValueDisplay":"$14.99","SubTotalAfterDsicount":14.99,"SubTotalAfterDiscountDisplay":"$14.99","CartPaymentItemUser":{"UserId":10011690,"StoreId":10061,"AppId":0,"PaymentTypeId":0,"UserProfileId":"","IsDefault":false,"IsCardDefault":false,"Credential1":"","Credential2":"UdVYyaeBH+IBzsetzftdBw==","Credential3":"","IsLive":false},"DoPDate":"01/01/1900","DoPTimeSlot":"","DopUtcStartDate":"","DopUtcEndDate":"","Remark":"","DeliveryTax":0.0,"ShippingCost":0.0,"CartItemCount":1,"IsCredentialOff":false,"IsDeliverySolution":false,"DeliveryFeeMarkUp":0.0,"ShippingTax":0.0,"DeliveryTaxRate":0.0,"IsFromCheckOut":true,"IsUpgradeUser":true,"IsToCallDSP":false,"DeliveryInstruction":"","IsStoreCreditUtilize":false,"OrderRewardCredit":5.00,"OrderRewardCreditDisplay":"-$5.00","OrderTotalWithCredit":0.0,"IsStoreReward":false,"CouponRemark":"","IsShipping":false,"CartTotalQuan":1,"ShippingServiceCode":"","ActualDeliveryCharge":0.0,"EstimatedShippingCost":0.0,"ShippingDeliveryCharge":0.0,"DoordashValidateCustomer":false,"CustomerSessionId":13733,"ConvenienceFee":0.0,"IsDoCalculation":false,"SuccessMessage":"","ErrorDetail":"","ErrorMessage":"","MessageType":"","MessageTitle":"","TextType":""}
I have tried in BeanShell PreProcessor:
var CartGetDetailsObject = prev.getResponseDataAsString();
var jsonData = JSON.parse(CartGetDetailsObject);
jsonData.OrderTypeId=2;
jsonData.IsCredentialOff=true;
vars.put("vCartGetDetailsObject",JSON.stringify(jsonData));
log.info(vCartGetDetailsObject);
Error Log
2020-04-06 12:56:48,139 INFO o.a.j.e.StandardJMeterEngine: Running the test!
2020-04-06 12:56:48,145 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2020-04-06 12:56:48,153 INFO o.a.j.r.ResultCollector: Folder at F:\Office\Performance Test Cases\BottlecappsJmeterPerformanceScripts\apache-jmeter-5.0\RunScript\Date 6-Apr-2020 125648 was created
2020-04-06 12:56:48,154 INFO o.a.j.g.u.JMeterMenuBar: setRunning(true, *local*)
2020-04-06 12:56:48,958 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2020-04-06 12:56:48,958 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2020-04-06 12:56:48,958 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2020-04-06 12:56:48,958 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=0 perThread=0.0 delayedStart=false
2020-04-06 12:56:48,984 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2020-04-06 12:56:48,984 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2020-04-06 12:56:48,986 INFO o.a.j.t.JMeterThread: Thread started: Thread Group 1-1
2020-04-06 12:56:48,987 INFO o.a.j.s.FileServer: Stored: F:\Office\Performance Test Cases\BottlecappsJmeterPerformanceScripts\apache-jmeter-5.0\testrundata.csv
2020-04-06 12:56:53,982 ERROR o.a.j.u.BeanShellInterpreter: Error invoking bsh method: eval Sourced file: inline evaluation of: ``var CartGetDetailsObject = prev.getResponseDataAsString(); var jsonData = JSON. . . . '' : Typed variable declaration : Attempt to resolve method: parse() on undefined variable or class name: JSON
2020-04-06 12:56:53,982 WARN o.a.j.m.BeanShellPreProcessor: Problem in BeanShell script. org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of: ``var CartGetDetailsObject = prev.getResponseDataAsString(); var jsonData = JSON. . . . '' : Typed variable declaration : Attempt to resolve method: parse() on undefined variable or class name: JSON
2020-04-06 12:56:54,645 ERROR o.a.j.p.h.s.HTTPJavaImpl: readResponse: java.io.IOException: Server returned HTTP response code: 400 for URL: https://staging.liquorapps.com/Bcapi/api/Cart/CartUpdate
2020-04-06 12:56:54,645 ERROR o.a.j.p.h.s.HTTPJavaImpl: Cause: java.io.IOException: Server returned HTTP response code: 400 for URL: https://staging.liquorapps.com/Bcapi/api/Cart/CartUpdate
2020-04-06 12:56:54,645 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 400
2020-04-06 12:56:57,171 ERROR o.a.j.u.BeanShellInterpreter: Error invoking bsh method: eval Sourced file: inline evaluation of: ``var CartGetDetailsObject = prev.getResponseDataAsString(); var jsonData = JSON. . . . '' : Typed variable declaration : Attempt to resolve method: parse() on undefined variable or class name: JSON
2020-04-06 12:56:57,171 WARN o.a.j.m.BeanShellPreProcessor: Problem in BeanShell script. org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval Sourced file: inline evaluation of: ``var CartGetDetailsObject = prev.getResponseDataAsString(); var jsonData = JSON. . . . '' : Typed variable declaration : Attempt to resolve method: parse() on undefined variable or class name: JSON
2020-04-06 12:56:57,389 ERROR o.a.j.p.h.s.HTTPJavaImpl: readResponse: java.io.IOException: Server returned HTTP response code: 400 for URL: https://staging.liquorapps.com/Bcapi/api/Cart/CartUpdate
2020-04-06 12:56:57,389 ERROR o.a.j.p.h.s.HTTPJavaImpl: Cause: java.io.IOException: Server returned HTTP response code: 400 for URL: https://staging.liquorapps.com/Bcapi/api/Cart/CartUpdate
2020-04-06 12:56:57,389 INFO o.a.j.p.h.s.HTTPJavaImpl: Error Response Code: 400
2020-04-06 12:56:58,364 INFO o.a.j.g.a.Start: Stopping test
2020-04-06 12:56:58,377 INFO o.a.j.t.JMeterThread: Stopping: Thread Group 1-1
2020-04-06 12:56:58,377 WARN o.a.j.t.JMeterThread: Interrupting: Thread Group 1-1 sampler: LoginCustomer
2020-04-06 12:56:58,382 INFO o.a.j.t.JMeterThread: Thread finished: Thread Group 1-1
2020-04-06 12:56:58,383 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2020-04-06 12:56:58,383 INFO o.a.j.s.FileServer: Close: F:\Office\Performance Test Cases\BottlecappsJmeterPerformanceScripts\apache-jmeter-5.0\testrundata.csv
2020-04-06 12:56:58,385 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, *local*)
JSON is a standard built-in object for Web API, you can run it in Postman because Postman is an electron application (browser). JMeter isn't a browser therefore you cannot use JSON functions.
Add JSR223 PostProcessor as a child of the request which returns above JSON
Put the following code into "Script" area:
def jsonData = new groovy.json.JsonSlurper().parse(prev.getResponseData())
jsonData.OrderTypeId = 2
jsonData.IsCredentialOff = true
vars.put("vCartGetDetailsObject", new groovy.json.JsonBuilder(jsonData).toPrettyString())
log.info(vars.get('vCartGetDetailsObject'))
More information:
Apache Groovy - Parsing and producing JSON
Apache Groovy - Why and How You Should Use It
I have 4 tests for listeners, the application is built on SpringBoot and uses Testcontainers, I run mvn clean install, and I get the following:
9-08-15 18:37:01.287 INFO 8312 --- [tContainer#1-11] o.s.a.r.c.CachingConnectionFactory : Attempting to connect to: [localhost:33724]
[INFO]
[INFO] Results:
[INFO]
2019-08-15 18:37:01.288 ERROR 8312 --- [tContainer#1-11] o.s.a.r.l.SimpleMessageListenerContainer : Failed to check/redeclare auto-delete queue(s).
org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:62)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:509)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:682)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.createConnection(ConnectionFactoryUtils.java:214)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2073)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2047)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2027)
at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueProperties(RabbitAdmin.java:403)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.attemptDeclarations(AbstractMessageListenerContainer.java:1787)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.redeclareElementsIfNecessary(AbstractMessageListenerContainer.java:1768)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.initialize(SimpleMessageListenerContainer.java:1195)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1041)
at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.base/java.net.Socket.connect(Socket.java:591)
at com.rabbitmq.client.impl.SocketFrameHandlerFactory.create(SocketFrameHandlerFactory.java:60)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1102)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1054)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:994)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:462)
... 11 common frames omitted
I made an output to the log file of the mvn clean install command, and the above occurs throughout the entire log, many times, but this is not the main problem, the problem is that the container with RabbitMQ starts many times, I would like to share it between tests.
2019-08-15 18:35:26.126 INFO --- [ main] 🐳 [rabbitmq:3.7-management] : Creating container for image: rabbitmq:3.7-management
2019-08-15 18:35:54.665 INFO 8312 --- [ main] 🐳 [rabbitmq:3.7-management] : Creating container for image: rabbitmq:3.7-management
2019-08-15 18:36:15.510 INFO 8312 --- [ main] 🐳 [rabbitmq:3.7-management] : Creating container for image: rabbitmq:3.7-management
2019-08-15 18:36:31.862 INFO 8312 --- [ main] 🐳 [rabbitmq:3.7-management] : Creating container for image: rabbitmq:3.7-management
4 times, 4 listeners.
I tried using forkCount = 0 for maven-surefire-plugin and for maven-failsafe-plugin, this did not help.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<forkCount>0</forkCount>
</configuration>
</plugin>
#RunWith(SpringRunner.class)
#SpringBootTest(
value = {
"spring.cloud.discovery.enabled = false"
},
webEnvironment = RANDOM_PORT)
#ActiveProfiles(TEST_PROFILE)
public abstract class AbstractTest {
#LocalServerPort
private int port;
}
#ContextConfiguration(initializers = AbstractRabbitTest.Initializer.class)
public abstract class AbstractRabbitTest extends AbstractTest {
#ClassRule
public static RabbitMQContainer rabbitMQContainer = new RabbitMQContainer("rabbitmq:3.7-management")
.withExposedPorts(5672)
.withVhost("/")
.withUser("admin", "admin")
.withPermission("/", "admin", ".*", ".*", ".*");
public static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
TestPropertyValues values = TestPropertyValues.of(
"spring.rabbitmq.host=" + rabbitMQContainer.getContainerIpAddress(),
"spring.rabbitmq.port=" + rabbitMQContainer.getMappedPort(5672),
"spring.rabbitmq.username=admin",
"spring.rabbitmq.password=admin"
);
values.applyTo(applicationContext);
}
}
}
Every my test looks like this and uses spring-amqp:
public class CloseProcessListenerTest extends AbstractRabbitTest {
ClassRule in JUnit will execute the rule per class, not per "suite". Since you have multiple test classes that extend it, you start multiple containers during the tests.
Consider using the Singleton Container pattern and check Spring Boot example
#RunWith(MockitoJUnitRunner.class) //Class cannot be resolved to a type //#SpringBootTest public class MbankingApplicationTest {
#Mock CanmbTransactionDao dataServiceMock;
#Mock CanmbBaseDao baseDao;
#InjectMocks CanmbTransactionServiceImpl businessImpl;
#Autowired OminiController controller;
Customer customer;
#Test public void test() {
Customer cust= new Customer();
cust.setMbnumber("+919990176197");
cust.setDeviceid("abcdef");
UserProfileMaster profile = new UserProfileMaster();
profile.setChkflag(22);
profile.setStatus("ACTIVE");
cust.setUserProfile(profile);
cust.setMpin("123456");
cust.setMpinsalt("12345");
when(dataServiceMock.getUserMbAndDevice("+919990176197", "abcdef")).thenReturn(cust);
this.customer = cust;
assertEquals(cust, businessImpl.getUserMbAndDevice("+919990176197", "abcdef"));
}
#Test public void testMpin() {
Customer cust = new Customer();
cust.setMbnumber("+919990176197");
cust.setDeviceid("abcdef");
UserProfileMaster profile = new UserProfileMaster(); profile.setChkflag(22);
profile.setStatus("ACTIVE");
cust.setUserProfile(profile);
cust.setMpin("d150cb2c64171a95eb3fa1bbf2ea786aef16b04d389a1ac67a52c75e95f61e66");
cust.setMpinsalt("12345");
when(dataServiceMock.getUserMbAndDevice("+919990176197", "abcdef")).thenReturn(cust);
//assertEquals(cust, businessImpl.getUserMbAndDevice("+919990176197", "abcdef"));
MBSOMNIIntegration reqData = new MBSOMNIIntegration();
reqData.setMbnumber("+919990176197");
reqData.setDeviceid("abcdef");
reqData.setMpin("123456");
OMNIIntegration omni=new OMNIIntegration();
// businessImpl.validateOmniMpin(reqData, omni, "123");
ResponseData data= new ResponseData();
Map<String,String> ominiMap= new HashMap<>();
ominiMap.put("Msg", "verified"); ominiMap.put("statusCode", "0");
data.setStatusCode(0);
data.setTid("");
data.setData(ominiMap);
when(businessImpl.validateOmniMpin(reqData, omni, "123")).thenReturn(data);
assertEquals(data,businessImpl.validateOmniMpin(reqData, omni, "123"));
}
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.835 sec
<<< FAILURE! - in com.npst.mb.MbankingApplicationTest testMpin(com.npst.mb.MbankingApplicationTest) Time elapsed: 0.823 sec
<<< FAILURE! java.lang.AssertionError: expected: com.npst.mb.pojo.ResponseData but was: com.npst.mb.pojo.ResponseData at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:120) at org.junit.Assert.assertEquals(Assert.java:146) at com.npst.mb.MbankingApplicationTest.testMpin(MbankingApplicationTest.java:93)
Results:
Failed tests: MbankingApplicationTest.testMpin:93 expected: com.npst.mb.pojo.ResponseData but was: com.npst.mb.pojo.ResponseData
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO]
------------------------------------------------------------------------ [INFO] Total time: 12.493 s [INFO] Finished at: 2019-01-17T14:57:43+05:30 [INFO] Final Memory: 36M/346M [INFO]
------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project CapexMbankingPhase2: There are test failures. [ERROR] [ERROR] Please refer to /home/npstx/raj/Canara Projects/MBS_APP_OMNI/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException