I am executing same query in parallel in 11 threads and I am getting huge difference in execution time of these queries. And if I execute only one query it executes very fast.
Thread code: I used 11 threads
Thread t1 = new Thread(new Runnable() {
public void run() {
try {
QueryResult result= null;
double qstart = System.currentTimeMillis();
ClusterEnvironment env = ClusterEnvironment
.builder()
.timeoutConfig(TimeoutConfig.kvTimeout(Duration.ofMinutes(15)))
.build();
Cluster cluster = Cluster.connect("xx.xx.xx.xxx", "**********", "************");
Bucket bucket = cluster.bucket("case5");
Collection collection = bucket.defaultCollection();
double qend1 = System.currentTimeMillis();
result = cluster.query("select `user`.userID,`user`.userName,`user`.employeeCode,`user`.personalDetails.userEmail[0].emailAddress,`user`.userDateOfBirth,`user`.userAddress from case3 where `user`.userID=\"user1\" and document.documentType=\"01\" and document.featureID=\"101\" and document.tenantID=\"CUST1\"",QueryOptions.queryOptions().metrics(true));
double qend = System.currentTimeMillis();
System.out.println("1Reported execution time: "
+ result.metaData().metrics());
System.out.println("1metaData"+result.metaData());
double qout = qend - qend1;
System.out.println("1"+cluster.diagnostics());
System.out.println("1Query time = " + qout);
for (JsonObject row : result.rowsAsObject()) {
System.out.println("1Found row: " + row);
}cluster.disconnect();
}
catch (Exception e) {
// TODO: handle exception
e.printStackTrace();
}
}});
Output:
200
messageOK
{"message":"success","data":["query execution time : 677.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 19.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 18.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 18.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 129.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 39.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 17.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 197.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 41.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 20.0.....]}
200
messageOK
{"message":"success","data":["query execution time : 13.0.....]}
And for single execution its taking only 5ms.
Is their any settings to do before executing parallel queries, why their is so much variation?? Please help me out!
Related
I wrote a smart contract and deploy it in my private ethereum network
this contract method had been called many times, and it ran well util these days when our website api is attacked.
here is contract method:
function batchTransfer(
uint64 _requestId,
uint16 _count,
address[] _receivers,
uint256[] _amounts
)
external
payable
onlyAdmin
{
require(_count == _receivers.length);
require(_count == _amounts.length);
uint sum = _sum(_amounts);
require(sum <= msg.value);
for(uint16 i = 0; i < _count; i++) {
if(msg.sender != _receivers[i]) {
_receivers[i].transfer(_amounts[i]);
}
}
msg.sender.transfer(msg.value - sum);
BatchTransfer(_requestId, msg.sender, sum, _count);
}
this method is a batch transfer method.
and _count max 100
gas limt is 3,000,000
when in HIGH concurrency condition, FAIL.
ethereum log 'out of gas'.
query by transactionHash:
eth.getTransactionReceipt("0x821db62fcc95c30db902a7173e42a0a00079787e1e4f65b453a668a129ed32db")
{
blockHash: "0x773a01a6ddff0c9bb5804ba6580fb4d5bdcd692e939a4fdfa89c608ce966dd5c",
blockNumber: 413423,
contractAddress: null,
cumulativeGasUsed: 10935074,
from: "0x5eeaf2f57fbc5b50ad98493f02cbf173860a37fd",
gasUsed: 3000000,
logs: [],
logsBloom: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
status: "0x0",
to: "0x6f909df22acf8ad94a38603981e9b9317c1cec2d",
transactionHash: "0x821db62fcc95c30db902a7173e42a0a00079787e1e4f65b453a668a129ed32db",
transactionIndex: 23
}
This transaction wrote into block successfully.
BUT in LOW concurrency condition, I called this method again.
it will SUCCESS!!!
NOTICE
ALL INPUT PARAMS ARE SAME!
no 'out of gas'
no exceed block gas limit
no exceed queue
some blocks contain both success and fail transactions
I am trying to use the data received from my API call in IONIC home.ts in google map method in home.ts .But it ended up giving a null or undefined error.
How can I use the data from one method in another?
Here is my home.ts code
getResturentDetails(id){
const data = localStorage.getItem('userToken');
this.userPostData.api_token= data;
this.userPostData.resturentId= id;
this.authService.postData(this.userPostData,'resturentDetail').then((result)=>{
this.responseData = result;
console.log(this.responseData); //I can see data as expected
})
}
Output of the console log is something like this
{"id":"1","lat":"10.90" ,"lon":"89.00"}
Something like this in JSON object notation
My constructor
constructor(public nav: NavController,
public navParams:NavParams,
public tripService: TripService,
public authService:AuthServiceProvider
) {
// set sample data
this.getResturentDetails(this.navParams.get('id'));
console.log(this.responseData.lat); //saying undefined index lat
console.log(this.responseData.lon); //saying undefined index lon
this.DisplayMap(this.responseData.lat,this.responseData.lon);
}
ionViewDidLoad() {
}
So how do I handle this issue ??
And in case if my API return an array of data, in that case how to handle the issue
For example, if the API response is something like this
[{"id":"1","lat":"10.90" ,"lon":"89.00"},{"id":"2","lat":"10.90" ,"lon":"89.00"}]
Something like this in JSON format in that case how to handle the issue??
You need to change your code like this :
constructor(...) {
...
this.getResturentDetails(this.navParams.get('id')).then(result => {
this.responseData = result;
console.log(this.responseData.lat); // will not say undefined index lat
console.log(this.responseData.lon); //will not say undefined index lon
this.DisplayMap(this.responseData.lat,this.responseData.lon);
})
}
getResturentDetails(id){
const data = localStorage.getItem('userToken');
this.userPostData.api_token= data;
this.userPostData.resturentId= id;
return this.authService.postData(this.userPostData,'resturentDetail');
}
Reason why its not working is , async behaviour :
Just follow the Execution sequence No in below code , you will get the idea how the
flow will go in real time execution
// this will call getResturentDetails , and consider it may take few sec
this.getResturentDetails(this.navParams.get('id')); // Execution sequence : 1
// next line will be executed before result returns coz of async behaviour
console.log(this.responseData.lat); // Execution sequence : 6
console.log(this.responseData.lon); // Execution sequence : 7
this.DisplayMap(this.responseData.lat,this.responseData.lon); // Execution sequence : 8
getResturentDetails(id){
const data = localStorage.getItem('userToken'); // Execution sequence : 2
this.userPostData.api_token= data; // Execution sequence : 3
this.userPostData.resturentId= id; // Execution sequence : 4
// Execution sequence : 5
this.authService.postData(this.userPostData,'resturentDetail').then((result)=>{
this.responseData = result; // Execution sequence : 9
console.log(this.responseData); // Execution sequence : 10
})
}
Hi there I'm having issues with SQL updating player stats which makes very huge lag/timings drop I'm updating the stats on server stop this is my stats code:
public int getDeaths(Player p) {
if (!plugin.getConfig().getBoolean("mysql")) {
return plugin.data.getConfig().getInt("Deaths." + p.getUniqueId() + ".death");
}
if (plugin.getConfig().getBoolean("mysql")) {
int res = 0;
ResultSet result = getMainSQLConnection()
.executeQuery("SELECT * FROM `Account` WHERE playername='" + p.getName() + "'", false);
try {
if (result.next()) {
res = Integer.parseInt(result.getString("deaths"));
}
} catch (SQLException localSQLException) {
}
return res;
}
return 0;
}
public void setDeaths(Player p, int number) {
if (!plugin.getConfig().getBoolean("mysql")) {
plugin.data.getConfig().set("Deaths." + p.getUniqueId() + ".death", number);
plugin.data.save();
}
if (plugin.getConfig().getBoolean("mysql")) {
plugin.sqlConnection.executeUpdate(
"UPDATE `Account` SET deaths='" + number + "' WHERE playername='" + p.getName() + "'");
}
}
If you're getting the death count for a command or something that doesn't require the value to be returned immediately, use a asynchronous scheduler to run the code on a separate thread. For a command you'd do something like this when it is executed:
Bukkit.getScheduler().runTaskAsynchronously(plugin, () -> {
try {
int deaths = getMainSQLConnection()
.executeQuery("SELECT * FROM `Account` WHERE playername='" + p.getName() + "'", false)
.getInt("deaths");
player.sendMessage("Player Deaths: " + deaths);
} catch (SQLException ex) {
player.sendMessage(ChatColor.RED + "That player does not exist!");
}
});
Otherwise, if you need the value in the code for whatever reason, you can use a connection pool such as HikariCP. A connection pool will allow you to maintain multiple connections to your database so when you need to execute a query you don't have to establish a new connection every time (which is what will cause most of the lag).
Better yet, use an asynchronous task in tandem with the connection pool. Here's a good tutorial for learning how to use HikariCP with Bukkit: https://www.spigotmc.org/threads/tutorial-implement-mysql-in-your-plugin-with-pooling.61678
As a side note, for basically zero performance impact on the server, you can load the data asynchronously when a player logs in (using the AsyncPlayerPreLoginEvent). Then store it in memory when they actually join the server (PlayerLoginEvent or PlayerJoinEvent), and remove it when they quit. This way you access the data through memory while they're logged in rather than the database. This is much more complicated and also requires a lot of code to implement correctly though, so I'm not going to go into detail here.
I want to use datepicker to select range of dates. This range of dates is then queried in the database where the sum is counted...
My attempt:
public static int dateRange(){
int value = 0;
PreparedStatement stmt = null;
try {
Connection connection = DriverManager.getConnection("", "", "");
stmt = connection.prepareStatement("SELECT SUM(cost) FROM Items WHERE expiration_date between '" + Budget.datePicker1.getValue() + "' and '" + Budget.datePicker2.getValue() + "'");
ResultSet result = stmt.executeQuery();
result.next();
String sum = result.getString(1);
value = Integer.parseInt(sum);
} catch (Exception e){
value += 0;
}
return value;
}
It works. It returns the total if the days are there so to speak. If there are no days in the database as selected in the DatePicker then 0 pops up... But it looks messed up (catch block) and I was wondering if anyone could help me with an alternative solution?
First, since the value returned by the query is an integer value, there is no need to read it as a string and parse it. Just use result.getInt(...).
Second, if you are going to use a PreparedStatement, use it to properly set parameters, instead of building the query out of concatenated strings. Doing the latter exposes your application to SQL injection attacks.
If none of your dates are in range, then the SQL query will return NULL. Calling getInt() on column in a result set that is NULL will return 0, so you will get the result you want in that case anyway. If the previous value you got from a result set was SQL NULL, then calling result.wasNull() will return true, so if you really did need to handle that case separately, you could use that mechanism to do so.
You can do:
public static int dateRange(){
int value = 0;
// Use try-with-resources to make sure resources are released:
try (
Connection connection = DriverManager.getConnection("", "", "");
PreparedStatement stmt = connection.prepareStatement(
"SELECT SUM(cost) FROM Items WHERE expiration_date between ? and ?");
) {
stmt.setDate(1, Date.valueOf(Budget.datePicker1.getValue()));
stmt.setDate(2, Date.valueOf(Budget.datePicker2.getValue()));
ResultSet result = stmt.executeQuery();
result.next();
// Note that if there are no values in range, the SQL result will be NULL
// and getInt() will return 0 anyway.
value = result.getInt(1);
// however, if you need to explicitly check this and do something different
// if nothing is in range, do:
if (result.wasNull()) {
// nothing was in range...
}
} catch (SQLException e){
// this actually indicates something went wrong. Handle it properly.
Logger.getGlobal().log(Level.SEVERE, "Error accessing database", e);
// inform user there was a db error, etc...
}
return value;
}
With Perfect/mySQL (https://github.com/PerfectlySoft/Perfect-MySQL), the examples I've seen suggest that you cannot reuse a single database connection across multiple connections. That is, you need a single database connection per HTTP request.
Examples I've seen are here:
https://github.com/PerfectlySoft/Perfect-MySQL
and here:
https://perfect.org/docs/MySQL.html
Is this right? You must create a new db connection per request with Perfect/mySQL?
You can still reuse the connection for multiple times, but please note that the connection is not thread-safe, so must add thread locks on the same connection
import MySQL
import PerfectThread
#if os(Linux)
import Glibc
#else
import Darwin
#endif
let mysql = MySQL()
let lock = Threading.Lock()
var jobs = 10
func now(_ id: Int) {
print("Job Now #", id)
lock.doWithLock {
let x = mysql.query(statement: "SELECT now() as time")
guard x, let y = mysql.storeResults(),
let row = y.next() else {
print(mysql.errorMessage())
return
}
print(row[0] ?? "Now() FAILED")
y.close()
jobs -= 1
}
}
func user(_ id: Int) {
print("Job Usr #", id)
lock.doWithLock {
let x = mysql.query(statement: "select User from user")
guard x, let y = mysql.storeResults(),
let row = y.next() else {
print(mysql.errorMessage())
return
}
print(row[0] ?? "User() FAILED")
y.close()
jobs -= 1
}
}
_ = mysql.setOption(.MYSQL_SET_CHARSET_NAME, "utf8mb4")
guard mysql.connect(host: "127.0.0.1", user: "root", password: "your pass", db: "mysql") else {
print(mysql.errorMessage())
exit(0)
}
jobs = 10
for id in 0 ..< 5 {
Threading.dispatch {
now(id)
}
Threading.dispatch {
user(id)
}
}
while jobs > 0 {
sleep(1)
}
mysql.close()
In this example, the both functions are sharing the same connection and running for a few times. That's OK. However, if each function is running in a separated thread, then it will cause a failure.