MySQL Update query hangs - Time exceeded - mysql

Hello I have trouble running mysql update cross databases. MySQL UPDATE query hangs with error - Lock wait timeout exceeded; try restarting transaction. I know similar questions have been already raised but could not find any working solution. I am using this query inside of the PHP script. BUt when I run it within mysql server it does the same thing. When I kill the process, it completes. Here is what hangs. PS: This started like 2 days ago, before that, the query ran without problem. Temp table is filled only when admin pushes there data from html form. 'Teams to esc' table grows like 20 rows per day.
UPDATE query -
UPDATE escalations.teams_to_esc
JOIN domguard.temp_table ON escalations.teams_to_esc.ticket_number = domguard.temp_table.task_number
AND
escalations.teams_to_esc.team = domguard.temp_table.team
SET
escalations.teams_to_esc.`status` = domguard.temp_table.`status`,
escalations.teams_to_esc.`wiw_assigned` = domguard.temp_table.`person`
WHERE
(escalations.teams_to_esc.team,escalations.teams_to_esc.ticket_number) IN
(SELECT domguard.temp_table.team, domguard.temp_table.task_number
FROM domguard.temp_table);
First table - temp_table
CREATE TABLE `temp_table` (
`ID` INT(32) NOT NULL AUTO_INCREMENT,
`task_number` INT(10) NULL DEFAULT '0',
`type` VARCHAR(50) NULL DEFAULT '0',
`assign_date` DATE NULL DEFAULT NULL,
`team` VARCHAR(50) NULL DEFAULT NULL,
`person` VARCHAR(50) NULL DEFAULT NULL,
`task` VARCHAR(50) NULL DEFAULT NULL,
`comment` TEXT NULL,
`short_text` VARCHAR(500) NULL DEFAULT NULL,
`delay_comment` VARCHAR(500) NULL DEFAULT NULL,
`color` VARCHAR(500) NULL DEFAULT NULL,
`status` VARCHAR(500) NULL DEFAULT NULL,
`end_time` TIME NULL DEFAULT NULL,
`tel_it` INT(2) NULL DEFAULT '0',
`co_allocation` VARCHAR(500) NULL DEFAULT '0',
`co_allocation_text` VARCHAR(500) NULL DEFAULT '0',
`delay_code_text` VARCHAR(500) NULL DEFAULT '0',
PRIMARY KEY (`ID`)
)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB
AUTO_INCREMENT=2483
;
Second table
CREATE TABLE `teams_to_esc` (
`ID` INT(11) NOT NULL AUTO_INCREMENT,
`esc` INT(11) NOT NULL,
`team` VARCHAR(500) NOT NULL,
`ticket_number` VARCHAR(500) NOT NULL,
`closed_time` DATE NOT NULL,
`checked` VARCHAR(500) NULL DEFAULT NULL,
`reaction_from_tl` VARCHAR(50) NULL DEFAULT NULL,
`status` VARCHAR(50) NULL DEFAULT NULL,
`wiw_assigned` VARCHAR(50) NULL DEFAULT NULL,
PRIMARY KEY (`ID`),
INDEX `esc` (`esc`),
CONSTRAINT `teams_to_esc_ibfk_1` FOREIGN KEY (`esc`) REFERENCES `main_table` (`ID`)
)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB
AUTO_INCREMENT=2670
;
=== ERROR INVESTIGATING ===
Output from SHOW processlist;
17107338 root localhost \N Query 1278 Sending data UPDATE escalations.teams_to_esc JOIN domguard.temp_table ON
escalations.teams_to_esc.ticket_number
Output from Transaction settings
SELECT ##GLOBAL.tx_isolation, ##tx_isolation, ##session.tx_isolation;

Related

How can i tune this mysql query to run successfully

This is my query on mysql version 8
select sender, fullName, phoneNumber, addressState, businessName, bvn, max(date)
from tranlog t INNER JOIN agent a on t.sender = a.realId
where captureDate < '2022-03-01' and active = 'Y' and thirdparty = 0
group by sender
CREATE TABLE `agent` (
`id` bigint NOT NULL AUTO_INCREMENT,
`realId` varchar(19) DEFAULT NULL,
`active` char(1) DEFAULT NULL,
`phoneNumber` varchar(15) DEFAULT NULL,
`address` varchar(255) DEFAULT NULL,
`addressState` varchar(50) DEFAULT NULL,
`addressCity` varchar(50) DEFAULT NULL,
`fullName` varchar(255) DEFAULT NULL,
`businessName` varchar(255) DEFAULT NULL,
`corporate` bit(1) DEFAULT b'0',
`thirdparty` bit(1) NOT NULL DEFAULT b'0',
PRIMARY KEY (`id`),
KEY `id` (`fee_group`),
KEY `realId` (`realId`),
KEY `agent_password` (`password`),
KEY `agent_idx` (`active`,`thirdparty`),
) ENGINE=InnoDB AUTO_INCREMENT=29784 DEFAULT CHARSET=latin1;
CREATE TABLE `tranlog` (
`id` bigint NOT NULL AUTO_INCREMENT
`date` datetime DEFAULT NULL
`captureDate` date DEFAULT NULL
`sender` varchar(50) DEFAULT NULL
PRIMARY KEY (`id`)
KEY `tranlog_date` (`date`)
KEY `sender` (`sender`)
KEY `tranlog_capturedate_idx` (`captureDate`)
) ENGINE=InnoDB AUTO_INCREMENT=49373312 DEFAULT CHARSET=latin1"
But i keep getting 'C:windows\TEMP#sql1234_2' is full which i believe is about temporary table
I have increased the size of tmp_table_size and max_heap_size to 3G yet no reprieve, the error keeps poping up.
Any ideas on how to tune the query
Add a 'LIMIT CLAUSE' to your query. It seems you have a very large result set.

Why do this query takes more than 180 secs to run?

CREATE TABLE `tvnotif` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`pingId` int(11) DEFAULT NULL,
`token` varchar(45) COLLATE utf8_bin DEFAULT NULL,
`summary` varchar(45) COLLATE utf8_bin DEFAULT NULL,
`startTime` int(11) DEFAULT NULL,
`endTime` int(11) DEFAULT NULL,
`processed` int(1) DEFAULT '0',
`created` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`modified` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `processedIndex` (`processed`),
KEY `summaryIndex` (`summary`),
KEY `tokenIndex` (`token`)
) ENGINE=InnoDB AUTO_INCREMENT=18297898 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
CREATE TABLE `vv_us` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`athleteid` int(11) DEFAULT NULL,
`token` varchar(45) COLLATE utf8_bin DEFAULT NULL,
`secret` varchar(45) COLLATE utf8_bin DEFAULT NULL,
`active` int(1) DEFAULT '1',
`created` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`modified` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`type` varchar(45) COLLATE utf8_bin DEFAULT 'mc',
`step` varchar(45) COLLATE utf8_bin DEFAULT NULL,
`host` varchar(45) COLLATE utf8_bin DEFAULT NULL,
`server` mediumblob,
`tempcreds` mediumblob,
PRIMARY KEY (`id`),
KEY `activeIndex` (`active`),
KEY `typeIndex` (`type`)
) ENGINE=InnoDB AUTO_INCREMENT=33888 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
I am running a query which is mostly based on the table tvnotif which has at least 2 million rows of data in it, all the other tables are having less amount only. I added the index for the tables before that the query took 20 minutes to run and now its taking 160 secs.
EXPLAIN SELECT tvu.secret,COALESCE(php_timezone,"America/Los_Angeles") AS userTz,tn.*,tvu.athleteid,tvu.type FROM tvnotif AS tn
LEFT JOIN vv_us AS tvu ON tvu.token = tn.token
LEFT JOIN tbl_ath_pro AS tap ON tap.athleteid = tvu.athleteid
LEFT JOIN timezones AS tz ON tz.tz_id = tap.tz_id
WHERE tvu.active = 1 AND tn.summary = 'dailies' AND tn.processed = 0
LIMIT 300
The problem is probably your indexes... You have indexes on each field individually. What you need is a composite index on ALL 3 parts as a single index. Without, it can't pick the best one as you have 3 parts of the where clause.
Build a SINGLE index on ( processed, summary, token )
This way the query can jump directly to the processed records, directly to the summary value and then get those records and be done.
Additionally, your VV_US table should have an index on ( token, active ) so the join will be optimized on BOTH parts.

Extremely slow query- Using google sql cloud

Is there a way I can speed this up? Right now it's taking an unbelievably insane amount of time to query.
SELECT trades.*, trader1.user_name as trader1_name,
trader2.user_name as trader2_name FROM trades
LEFT JOIN logs_players trader1 ON trader1.user_id = trader1_account_id
LEFT JOIN logs_players trader2 ON trader2.user_id = trader2_account_id
ORDER BY time_added
LIMIT 20 OFFSET 0;
I've done as much as I could in terms of searching online for a solution. Or even just trying to get some more information why it's taking so long to execute.
The query takes about 45 seconds or so to complete.
Create statements:
CREATE TABLE `trades` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`trader1_account_id` int(11) DEFAULT NULL,
`trader2_account_id` int(11) DEFAULT NULL,
`trader1_value` bigint(20) DEFAULT NULL,
`trader2_value` bigint(20) DEFAULT NULL,
`trader1_ip` varchar(16) DEFAULT NULL,
`trader2_ip` varchar(16) DEFAULT NULL,
`world` int(11) DEFAULT NULL,
`x` int(11) DEFAULT NULL,
`z` int(11) DEFAULT NULL,
`level` int(11) DEFAULT NULL,
`trader1_user` varchar(12) DEFAULT NULL,
`trader2_user` varchar(12) DEFAULT NULL,
`time_added` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=utf8
CREATE TABLE `logs_players` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`user_name` varchar(20) DEFAULT NULL,
`world_stage` varchar(20) DEFAULT NULL,
`world_type` varchar(20) DEFAULT NULL,
`bank` longtext,
`inventory` longtext,
`equipment` longtext,
`total_wealth` mediumtext,
`total_play_time` mediumtext,
`rights` int(11) DEFAULT NULL,
`icon` int(11) DEFAULT NULL,
`ironmode` int(11) DEFAULT NULL,
`x` int(11) DEFAULT NULL,
`z` int(11) DEFAULT NULL,
`level` int(11) DEFAULT NULL,
`last_ip` varchar(16) DEFAULT NULL,
`last_online` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`muted_until` timestamp NULL DEFAULT NULL,
`banned_until` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=utf8
I filled a sample database with 10k rows each, and found that a few indexes were what you needed:
ALTER TABLE `logs_players` ADD INDEX(`user_id`);
ALTER TABLE `trades` ADD INDEX(`time_added`);
The main index we need is an index on user_id. Changing that we went from a query time of 20.1390 seconds, to 0.0130 seconds:
We can even get that down further, by adding an index on time_added to make sorting a lot faster, now we ended up with an impressive query time:
Do some research on indexes! A simple EXPLAIN query would show you that you're using filesort (Which is rather bad!):
After indexes, this looks a lot better:

MySQL use separate indices for JOIN and GROUP BY

I am trying to execute following query
SELECT
a.sessionID AS `sessionID`,
firstSeen, birthday, gender,
isAnonymous, LanguageCode
FROM transactions AS trx
INNER JOIN actions AS a ON a.sessionID = trx.SessionID
WHERE a.ActionType = 'PURCHASE'
GROUP BY trx.TransactionNumber
Explain provides the following output
1 SIMPLE trx ALL TransactionNumber,SessionID NULL NULL NULL 225036 Using temporary; Using filesort
1 SIMPLE a ref sessionID sessionID 98 infinitiExport.trx.SessionID 1 Using index
The problem is that I am trying to use one field for join and different field for GROUP BY.
How can I tell MySQL to use different indices for same table?
CREATE TABLE `transactions` (
`SessionID` varchar(32) NOT NULL DEFAULT '',
`date` datetime DEFAULT NULL,
`TransactionNumber` varchar(32) NOT NULL DEFAULT '',
`CustomerECommerceTrackID` int(11) DEFAULT NULL,
`SKU` varchar(45) DEFAULT NULL,
`AmountPaid` double DEFAULT NULL,
`Currency` varchar(10) DEFAULT NULL,
`Quantity` int(11) DEFAULT NULL,
`Name` tinytext NOT NULL,
`Category` varchar(45) NOT NULL DEFAULT '',
`customerInfoXML` text,
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
KEY `TransactionNumber` (`TransactionNumber`),
KEY `SessionID` (`SessionID`)
) ENGINE=InnoDB AUTO_INCREMENT=212007 DEFAULT CHARSET=utf8;
CREATE TABLE `actions` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`sessionActionDate` datetime DEFAULT NULL,
`actionURL` varchar(255) DEFAULT NULL,
`sessionID` varchar(32) NOT NULL DEFAULT '',
`ActionType` varchar(64) DEFAULT NULL,
`CustomerID` int(11) DEFAULT NULL,
`IPAddressID` int(11) DEFAULT NULL,
`CustomerDeviceID` int(11) DEFAULT NULL,
`customerInfoXML` text,
PRIMARY KEY (`id`),
KEY `ActionType` (`ActionType`),
KEY `CustomerDeviceID` (`CustomerDeviceID`),
KEY `sessionID` (`sessionID`)
) ENGINE=InnoDB AUTO_INCREMENT=15042833 DEFAULT CHARSET=utf8;
Thanks
EDIT 1: My indexes were broken, I had to add (SessionID, TransactionNumber) index to transactions table, however now, when I try to include trx.customerInfoXML table mysql stops using index
EDIT 2 Another answer does not really solved my problem because it's not standard sql syntax and generally not a good idea to force indices.
For ORM users such syntax is a unattainable luxury.
EDIT 3 I updated my indices and it solved the problem, see EDIT 1

mysql join query is slow compared with mssql

I have the following simple join query
SELECT
count(*)
FROM
DBx.caseview p2015
INNER JOIN DBy.caseview p2014 ON p2015.casenumber=p2014.casenumber;
For some reason it just leaves MySQL hanging there for a lot of time until I get tired and cancel it. On the contrary, if run exactly the same code on MSSQL with the same data set the query takes a few seconds at most.
Is there a parameter that needs to be changed on MySQL to speed up this type of queries?
Here's my table in MySQL
CREATE TABLE `caseview` (
`ID` bigint(20) NOT NULL AUTO_INCREMENT,
`CASEID` varchar(18) DEFAULT NULL,
`CASENUMBER` int(10) DEFAULT NULL,
`ACCOUNTID` varchar(18) DEFAULT NULL,
`ACCOUNT` varchar(256) DEFAULT NULL,
`ASSETID` varchar(18) DEFAULT NULL,
`SAPPRODUCTGROUP` varchar(10) DEFAULT NULL,
`PRODUCT` varchar(128) DEFAULT NULL,
`FAMILY` varchar(128) DEFAULT NULL,
`CONTACTID` varchar(18) DEFAULT NULL,
`OWNERID` varchar(18) DEFAULT NULL,
`TYPE` varchar(128) DEFAULT NULL,
`PRIORITY` varchar(24) DEFAULT NULL,
`ORIGIN` varchar(24) DEFAULT NULL,
`SUBJECT` varchar(256) DEFAULT NULL,
`STATUS` varchar(24) DEFAULT NULL,
`LASTACTIVITY` varchar(1024) DEFAULT NULL,
`INITALDESCRIPTION` varchar(1024) DEFAULT NULL,
`CLOSEDDATE` datetime DEFAULT NULL,
`CREATEDDATE` datetime DEFAULT NULL,
`LASTMODIFIEDDATE` datetime DEFAULT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `ID_UNIQUE` (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=122393 DEFAULT CHARSET=utf8;
There's two tables with exactly the same configuration as above, just different data
DBx.caseview p2015 has 197647 rows
DBy.caseview p2014 has 122392 rows
Create an Index for CASENUMBER field.
ALTER TABLE `caseview` ADD INDEX ( `CASENUMBER` ) ;
Give it a few seconds to create the index and run the query again.