User Login Authentication Failure - WSO2 Identity Server - mysql

Question

We're having users created under Secondary Userstore(JDBC Userstore). Similarly, we have an application called MyApplication created in API Store. When users are trying to login to that MyApplication by invoking /token API which was provided by WSO2 even with correct username (in the format of TESTDOMAIN/testuser) and password also. Sometimes login is getting failed by returning a response with 400 Bad Request:
{
"error_description": "Error when handling event : PRE_AUTHENTICATION",
"error": "invalid_grant"
}
And, in the IDM Audit.log, the error was like shown below:
WARN {AUDIT_LOG}- Initiator=wso2.system.user Action=Authentication Target=TESTDOMAIN/testuser Data=null Outcome=Failure Error={"Error Message":"Un-expected error while pre-authenticating, Error when handling event : PRE_AUTHENTICATION","Error Code":"31002"}
After 5 attempts of user login, the user is getting logged in successfully without any problem.
I'm not getting any clue and not understanding why this login failure happens randomly.
Please provide your solutions/ideas regarding this issue.
UPDATED:
After enabling user core debug logs and some other logs which seems to be relevant to this issue. During authentication failure, I could see following wso2carbon.log:
DEBUG {org.wso2.carbon.user.core.jdbc.JDBCUserStoreManager} - Error occurred while checking existence of values.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 733,140 milliseconds ago. The last packet sent successfully to the server was 733,140 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)
... 113 more
DEBUG {org.wso2.carbon.identity.oauth2.token.AccessTokenIssuer} - Error occurred while validating grant
org.wso2.carbon.identity.oauth2.IdentityOAuth2Exception: Error when handling event : PRE_AUTHENTICATION
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 733,140 milliseconds ago. The last packet sent successfully to the server was 733,140 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

Answer 1

As stated by #senthalan in the comments, let's try adding "autoReconnect=true" to the end of the connection URL.
Additionally, please verify you are having the following recommended values under connection configurations for your MySQL datasources in the master-datasources.xml. (As described in [1])
<definition type="RDBMS">
<configuration>
<url>jdbc:mysql://localhost:3306/umdb?autoReconnect=true</url>
<username>regadmin</username>
<password>regadmin</password>
<driverClassName>com.mysql.jdbc.Driver</driverClassName>
<maxActive>80</maxActive>
<maxWait>60000</maxWait>
<minIdle>5</minIdle>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
<defaultAutoCommit>false</defaultAutoCommit>
Also, we can increase the number of max_connections from the DB side as described in [2].
mysql> SET GLOBAL max_connections = 500;
Query OK, 0 rows affected (0.00 sec)
[1] https://docs.wso2.com/display/ADMIN44x/Changing+to+MySQL
[2] https://stackoverflow.com/a/19991390/2910841