I have the following SQL statement:
SELECT user_accounts.uacc_id,
user_accounts.uacc_username,
ride_rides.ride_type,
ride_rides.ride_num_seats,
ride_rides.ride_price_seat,
ride_rides.ride_accept_nm,
ride_rides.ride_split_cost,
ride_rides.ride_from,
ride_rides.ride_from_lat,
ride_rides.ride_from_lng,
ride_rides.ride_to,
ride_rides.ride_to_lat,
ride_rides.ride_to_lng,
user_profiles.upro_image_name,
ride_times.ridetms_id,
ride_times.ridetms_return,
ride_times.ridetms_depart_date,
ride_times.ridetms_depart_time,
ride_times.ridetms_return_date,
ride_times.ridetms_return_time,
depart_times.dpttme_text
FROM ride_times
LEFT JOIN ride_rides
ON ride_rides.ride_id = ride_times.ridetms_ride_fk
LEFT JOIN user_accounts
ON ride_rides.ride_uacc_fk = user_accounts.uacc_id
LEFT JOIN user_profiles
ON user_profiles.upro_uacc_fk = user_accounts.uacc_id
LEFT JOIN depart_times
ON depart_times.dpttme_id = ride_times.ridetms_depart_time
WHERE ride_times.ridetms_id = ?"
Right now, I have the query pulling a text representation of the data from ride_times.ridetms_depart_time in the last join, and it works fine. However, I need to do the same with another column in the ride_times table. I think I need to use an alias, but after reading several sources on aliases, I can't wrap my head around how to change the call.
Also, 100 brownie points for any feedback about any glaring mistakes in this call. It is my first attempt at using JOINs.
take care,
lee
Thanks to the responses I've received so far. Here is the structure of the tables involved:
CREATE TABLE `depart_times` (
`dpttme_id` int(11) NOT NULL,
`dpttme_text` varchar(50) DEFAULT NULL,
PRIMARY KEY (`dpttme_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
CREATE TABLE `ride_rides` (
`ride_id` int(11) NOT NULL AUTO_INCREMENT,
`ride_uacc_fk` int(11) NOT NULL,
`ride_date_added` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`ride_type` tinyint(4) DEFAULT NULL,
`ride_from` varchar(200) DEFAULT NULL,
`ride_from_lat` float(10,6) DEFAULT NULL,
`ride_from_lng` float(10,6) DEFAULT NULL,
`ride_to` varchar(200) DEFAULT NULL,
`ride_to_lat` float(10,6) DEFAULT NULL,
`ride_to_lng` float(10,6) DEFAULT NULL,
`ride_num_seats` tinyint(4) DEFAULT NULL,
`ride_price_seat` float DEFAULT NULL,
`ride_accept_nm` tinyint(1) DEFAULT '0' COMMENT 'accept non-monetary items',
`ride_split_cost` tinyint(1) DEFAULT '0',
`ride_notes` longtext,
PRIMARY KEY (`ride_id`)
) ENGINE=MyISAM AUTO_INCREMENT=34 DEFAULT CHARSET=latin1;
CREATE TABLE `ride_times` (
`ridetms_id` int(11) NOT NULL AUTO_INCREMENT,
`ridetms_ride_fk` int(11) DEFAULT NULL,
`ridetms_date_added` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`ridetms_depart_date` date NOT NULL DEFAULT '0000-00-00',
`ridetms_depart_time` tinyint(4) DEFAULT '0',
`ridetms_return` tinyint(1) DEFAULT '0',
`ridetms_return_date` date NOT NULL DEFAULT '0000-00-00',
`ridetms_return_time` tinyint(4) DEFAULT '0',
PRIMARY KEY (`ridetms_id`)
) ENGINE=MyISAM AUTO_INCREMENT=8 DEFAULT CHARSET=latin1;
CREATE TABLE `user_accounts` (
`uacc_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`uacc_group_fk` smallint(5) unsigned NOT NULL,
`uacc_email` varchar(100) NOT NULL,
`uacc_username` varchar(15) NOT NULL,
`uacc_password` varchar(60) NOT NULL,
`uacc_ip_address` varchar(40) NOT NULL,
`uacc_salt` varchar(40) NOT NULL,
`uacc_activation_token` varchar(40) NOT NULL,
`uacc_forgotten_password_token` varchar(40) NOT NULL,
`uacc_forgotten_password_expire` datetime NOT NULL,
`uacc_update_email_token` varchar(40) NOT NULL,
`uacc_update_email` varchar(100) NOT NULL,
`uacc_active` tinyint(1) unsigned NOT NULL,
`uacc_suspend` tinyint(1) unsigned NOT NULL,
`uacc_fail_login_attempts` smallint(5) NOT NULL,
`uacc_fail_login_ip_address` varchar(40) NOT NULL,
`uacc_date_fail_login_ban` datetime NOT NULL COMMENT 'Time user is banned until due to repeated failed logins',
`uacc_date_last_login` datetime NOT NULL,
`uacc_date_added` datetime NOT NULL,
PRIMARY KEY (`uacc_id`),
UNIQUE KEY `uacc_id` (`uacc_id`),
KEY `uacc_group_fk` (`uacc_group_fk`),
KEY `uacc_email` (`uacc_email`),
KEY `uacc_username` (`uacc_username`),
KEY `uacc_fail_login_ip_address` (`uacc_fail_login_ip_address`)
) ENGINE=InnoDB AUTO_INCREMENT=56 DEFAULT CHARSET=latin1;
CREATE TABLE `user_profiles` (
`upro_id` int(11) NOT NULL AUTO_INCREMENT,
`upro_uacc_fk` int(11) NOT NULL,
`upro_name` varchar(100) DEFAULT NULL,
`upro_blackberry_id` varchar(200) DEFAULT NULL,
`upro_yahoo_id` varchar(200) DEFAULT NULL,
`upro_skype_id` varchar(200) DEFAULT NULL,
`upro_gmail_id` varchar(200) DEFAULT NULL,
`upro_image_name` varchar(200) DEFAULT 'default.jpg',
PRIMARY KEY (`upro_id`),
UNIQUE KEY `upro_id` (`upro_id`),
KEY `upro_uacc_fk` (`upro_uacc_fk`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=25 DEFAULT CHARSET=latin1;
So, to clarify:
Right now I am pulling some text from depart_times based on ride_times.ridetms_depart_time. I need to also pull some text form depart_times based on ride_times.ridetms_return_time.
I think if you have joined all the tables already, Just put the condition with the WHERE clause Like :
WHERE
ride_times.ridetms_id = ?
AND
depart_times.column_name1 = ride_times.return_time
Does this answer your question?
Well, I must point out this line:
LEFT JOIN depart_times ON depart_times.dpttme_id = ride_times.ridetms_depart_time
Just feels like, something wrong with the schema. But, that is also a guess from my side seeing the namings..
Ok,
After LOTS more searching, I found this post:
MySQL alias for SELECT * columns
I have revised my statement to the following and it is working properly:
SELECT user_accounts.uacc_id,
user_accounts.uacc_username,
ride_rides.*,
user_profiles.upro_image_name,
ride_times.*,
dpt1.dpttme_text AS dep_text,
dpt2.dpttme_text AS ret_text
FROM ride_times
LEFT JOIN ride_rides
ON ride_rides.ride_id = ride_times.ridetms_ride_fk
LEFT JOIN user_accounts
ON ride_rides.ride_uacc_fk = user_accounts.uacc_id
LEFT JOIN user_profiles
ON user_profiles.upro_uacc_fk = user_accounts.uacc_id
LEFT JOIN depart_times AS dpt1
ON dpt1.dpttme_id = ride_times.ridetms_depart_time
LEFT JOIN depart_times AS dpt2
ON dpt2.dpttme_id = ride_times.ridetms_return_time
WHERE ride_times.ridetms_id = ?
Thanks very much to everyone who attempted to help me.
take care,
lee
Related
I have a very long running MySql query. The query simply joins two tables which are very huge
bizevents - Nearly 34 Million rows
bizevents_actions - Nearly 17 million rows
Here is the query:
select
bizevent0_.id as id1_37_,
bizevent0_.json as json2_37_,
bizevent0_.account_id as account_3_37_,
bizevent0_.createdBy as createdB4_37_,
bizevent0_.createdOn as createdO5_37_,
bizevent0_.description as descript6_37_,
bizevent0_.iconCss as iconCss7_37_,
bizevent0_.modifiedBy as modified8_37_,
bizevent0_.modifiedOn as modified9_37_,
bizevent0_.name as name10_37_,
bizevent0_.version as version11_37_,
bizevent0_.fired as fired12_37_,
bizevent0_.preCreateFired as preCrea13_37_,
bizevent0_.entityRefClazz as entityR14_37_,
bizevent0_.entityRefIdAsStr as entityR15_37_,
bizevent0_.entityRefIdType as entityR16_37_,
bizevent0_.entityRefName as entityR17_37_,
bizevent0_.entityRefType as entityR18_37_,
bizevent0_.entityRefVersion as entityR19_37_
from
BizEvent bizevent0_
left outer join BizEvent_actions actions1_ on
bizevent0_.id = actions1_.BizEvent_id
where
bizevent0_.createdOn >= '1969-12-31 19:00:01.0'
and (actions1_.action <> 'SoftLock'
and actions1_.targetRefClazz = 'com.biznuvo.core.orm.domain.org.EmployeeGroup'
and actions1_.targetRefIdAsStr = '1'
or actions1_.action <> 'SoftLock'
and actions1_.objectRefClazz = 'com.biznuvo.core.orm.domain.org.EmployeeGroup'
and actions1_.objectRefIdAsStr = '1')
order by
bizevent0_.createdOn;
Below are the table definitions -- As you see i have defined the indexes well enough on these two tables on all the search columns plus the sort column. But still my queries are running for very very long time. Appreciate any more ideas either with respective indexing.
-- bizevent definition
CREATE TABLE `bizevent` (
`id` bigint(20) NOT NULL,
`json` longtext,
`account_id` int(11) DEFAULT NULL,
`createdBy` varchar(50) NOT NULL,
`createdon` datetime(3) DEFAULT NULL,
`description` varchar(255) DEFAULT NULL,
`iconCss` varchar(50) DEFAULT NULL,
`modifiedBy` varchar(50) NOT NULL,
`modifiedon` datetime(3) DEFAULT NULL,
`name` varchar(255) NOT NULL,
`version` int(11) NOT NULL,
`fired` bit(1) NOT NULL,
`preCreateFired` bit(1) NOT NULL,
`entityRefClazz` varchar(255) DEFAULT NULL,
`entityRefIdAsStr` varchar(255) DEFAULT NULL,
`entityRefIdType` varchar(25) DEFAULT NULL,
`entityRefName` varchar(255) DEFAULT NULL,
`entityRefType` varchar(50) DEFAULT NULL,
`entityRefVersion` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `IDXk9kxuuprilygwfwddr67xt1pw` (`createdon`),
KEY `IDXsf3ufmeg5t9ok7qkypppuey7y` (`entityRefIdAsStr`),
KEY `IDX5bxv4g72wxmjqshb770lvjcto` (`entityRefClazz`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
-- bizevent_actions definition
CREATE TABLE `bizevent_actions` (
`BizEvent_id` bigint(20) NOT NULL,
`action` varchar(255) DEFAULT NULL,
`objectBizType` varchar(255) DEFAULT NULL,
`objectName` varchar(255) DEFAULT NULL,
`objectRefClazz` varchar(255) DEFAULT NULL,
`objectRefIdAsStr` varchar(255) DEFAULT NULL,
`objectRefIdType` int(11) DEFAULT NULL,
`objectRefVersion` int(11) DEFAULT NULL,
`targetBizType` varchar(255) DEFAULT NULL,
`targetName` varchar(255) DEFAULT NULL,
`targetRefClazz` varchar(255) DEFAULT NULL,
`targetRefIdAsStr` varchar(255) DEFAULT NULL,
`targetRefIdType` int(11) DEFAULT NULL,
`targetRefVersion` int(11) DEFAULT NULL,
`embedJson` longtext,
`actions_ORDER` int(11) NOT NULL,
PRIMARY KEY (`BizEvent_id`,`actions_ORDER`),
KEY `IDXa21hhagjogn3lar1bn5obl48gll` (`action`),
KEY `IDX7agsatk8u8qvtj37vhotja0ce77` (`targetRefClazz`),
KEY `IDXa7tktl678kqu3tk8mmkt1mo8lbo` (`targetRefIdAsStr`),
KEY `IDXa22eevu7m820jeb2uekkt42pqeu` (`objectRefClazz`),
KEY `IDXa33ba772tpkl9ig8ptkfhk18ig6` (`objectRefIdAsStr`),
CONSTRAINT `FKr9qjs61id11n48tdn1cdp3wot` FOREIGN KEY (`BizEvent_id`) REFERENCES `bizevent` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;>
By the way we are using Amazon RDS 5.7.33 MySql version. 16 GB RAM and 4 vCPU.
I also did a Explain Extended on the query and below is what it shows. Appreciate any help.
Initially the search of the bizevent_actions didn;t have the indexes defined. I have defined the indexes for them and tried the query but of no use.
One technique that worked for me in a similar situation was abandoning the idea of JOIN completely and switching to queries by PK. More detailed: find out which table in join would give less rows on average if you use only that table and related filter to query; get the primary keys from that table and then query the other one using WHERE pk IN ().
In your case one example would be:
SELECT
bizevent0_.id as id1_37_,
bizevent0_.json as json2_37_,
bizevent0_.account_id as account_3_37_,
...
FROM BizEvent bizevent0_
WHERE
bizevent0_.createdOn >= '1969-12-31 19:00:01.0'
AND bizevent0_.id IN (
SELECT BizEvent_id
FROM BizEvent_actions actions1_
WHERE
actions1_.action <> 'SoftLock'
and actions1_.targetRefClazz = 'com.biznuvo.core.orm.domain.org.EmployeeGroup'
and actions1_.targetRefIdAsStr = '1'
or actions1_.action <> 'SoftLock'
and actions1_.objectRefClazz = 'com.biznuvo.core.orm.domain.org.EmployeeGroup'
and actions1_.objectRefIdAsStr = '1')
ORDER BY
bizevent0_.createdOn;
This assumes that you're not actually willing to select 33+ Mio rows from BizEvent though - your code with LEFT OUTER JOIN would have done exactly this.
I have a few problems with MYSQL database that I can't solve.
My query below is taking too much time and making the system hang. I'm trying to use the "JOIN" construct to develop this. But this time my aggregation, which I'm trying to do with "SUM", reduces the query to one line. Is it ok to do this job with "JOIN"? or how should i improve this query.
This database works with a total of 22 client devices in ASP .NET application. As I mentioned above, in cases where the query time is long, when the client devices send a query to the database at the same time, the client device freezes. What I don't understand is why a query in the browser app is making all devices wait. Isn't each query processed as a separate "Thread" in MYSQL? So if a query has a return time of 10 seconds, will all clients wait 10 seconds for the query to be answered in the browser?
SELECT *,
(SELECT MachModel FROM machine WHERE MachCode=workorder.MachCode) AS MachModel,
(SELECT RawMaterialDescription FROM rawmaterials WHERE RawMaterialCode=workorder.ProductRawMaterial) AS RawMaterialDescr,
(SELECT RawMaterialColor FROM rawmaterials WHERE RawMaterialCode=workorder.ProductRawMaterial) AS RawMaterialColor,
(SELECT StaffName FROM staff WHERE AccountName=workorder.AssignStaff) AS AssignStaffName,
(SELECT StaffCode FROM staff WHERE AccountName=workorder.AssignStaff) AS AssignStaffCode,
(SELECT MachStatus FROM machine WHERE MachCode=workorder.MachCode) AS MachStatus,
(SELECT SUM(xStopTime) FROM workorderb WHERE xWoNumber=workorder.WoNumber) AS WoTotalStopTime
FROM workorder
WHERE WoStatus=3
ORDER BY PlanProdStartDate DESC, WoSortNumber, WoNumber LIMIT 100
SELECT workorder.*,machine.MachModel,machine.MachStatus,rawmaterials.RawMaterialDescription,rawmaterials.RawMaterialColor,staff.StaffName,staff.StaffCode,SUM(workorderb.xStopTime)
FROM workorder
LEFT JOIN machine ON machine.MachCode=workorder.MachCode
LEFT JOIN rawmaterials ON rawmaterials.RawMaterialCode=workorder.ProductRawMaterial
LEFT JOIN staff ON staff.AccountName=workorder.AssignStaff
LEFT JOIN workorderb ON workorderb.xWoNumber=workorder.WoNumber
WHERE workorder.WoStatus=3
ORDER BY workorder.PlanProdStartDate DESC, workorder.WoSortNumber, workorder.WoNumber LIMIT 100
CREATE TABLE `workorder` (
`WoNumber` varchar(20) NOT NULL,
`MachCode` varchar(15) NOT NULL,
`PlannedMoldCode` varchar(10) NOT NULL,
`PartyNumber` smallint(6) NOT NULL,
`PlanProdCycleTime` smallint(6) NOT NULL,
`CalAverageCycleTime` float(15,1) unsigned NOT NULL,
`ProductRawMaterial` varchar(30) NOT NULL,
`PlanProdStartDate` date NOT NULL,
`PlanProdFinishDate` int(10) unsigned NOT NULL,
`WoStartDate` datetime DEFAULT NULL,
`WoFinishDate` datetime DEFAULT NULL,
`WoWorkTime` int(10) unsigned NOT NULL,
`WoSystemProductivity` smallint(6) unsigned NOT NULL,
`AssignStaff` varchar(50) DEFAULT '',
`WoStatus` smallint(6) NOT NULL,
`WoSortNumber` int(10) unsigned NOT NULL,
`CycleCount` int(11) unsigned NOT NULL,
`ControlDate` datetime NOT NULL ON UPDATE CURRENT_TIMESTAMP,
`WoProductionStatus` smallint(6) NOT NULL DEFAULT '0',
`Creator` varchar(50) NOT NULL,
`Changer` varchar(50) NOT NULL,
`CreateDate` datetime NOT NULL,
PRIMARY KEY (`WoNumber`) USING BTREE,
KEY `WoNumber` (`WoNumber`) USING BTREE,
KEY `WoNumber_2` (`WoNumber`) USING BTREE,
KEY `WoNumber_3` (`WoNumber`) USING BTREE
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC
CREATE TABLE `machine` (
`MachCode` varchar(15) NOT NULL,
`MachModel` varchar(30) NOT NULL,
`FirstProdDate` date NOT NULL,
`MachCapacity` smallint(6) NOT NULL,
`MachStatus` smallint(6) NOT NULL,
`NowMoldOnMach` varchar(10) NOT NULL DEFAULT '',
`NowMachOperator` varchar(50) NOT NULL DEFAULT '',
`NowWorkOrder` varchar(20) NOT NULL DEFAULT '',
`IPNumber` varchar(15) NOT NULL,
`Creator` varchar(50) NOT NULL,
`Changer` varchar(50) NOT NULL,
`ControlDate` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`OperatorLoginDate` datetime DEFAULT NULL,
`Message` varchar(500) DEFAULT NULL,
`MessageReaded` smallint(6) DEFAULT '0',
`StaffName` varchar(50) DEFAULT 'OSIS',
`StaffImage` varchar(255) DEFAULT '',
`StopDesc` varchar(30) DEFAULT 'OSIS',
`StopTime` datetime DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`MachCode`) USING BTREE
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC
CREATE TABLE `rawmaterials` (
`RawMaterialCode` varchar(15) NOT NULL,
`RawMaterialDescription` varchar(30) NOT NULL,
`RawMaterialColor` varchar(30) NOT NULL,
PRIMARY KEY (`RawMaterialCode`) USING BTREE,
KEY `RawMaterialCode` (`RawMaterialCode`) USING BTREE
) ENGINE=MyISAM AUTO_INCREMENT=3 DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC
CREATE TABLE `staff` (
`StaffCode` varchar(15) DEFAULT NULL,
`StaffCardCode` varchar(10) DEFAULT NULL,
`StaffName` varchar(50) NOT NULL,
`StaffPassword` varchar(10) NOT NULL,
`StaffStatus` smallint(6) NOT NULL DEFAULT '2',
`StaffDateOfStart` date NOT NULL,
`StaffBirthDay` date DEFAULT NULL,
`StaffGender` varchar(5) DEFAULT NULL,
`StaffRoleA` smallint(6) NOT NULL,
`StaffEmail` varchar(100) NOT NULL,
`StaffImageLink` varchar(255) DEFAULT NULL,
`AccountName` varchar(50) NOT NULL,
`StaffRoleB` smallint(6) NOT NULL,
`StaffRoleD` smallint(6) NOT NULL,
`StaffRoleE` smallint(6) NOT NULL,
`StaffRoleC` smallint(6) NOT NULL,
`StaffRoleF` smallint(6) NOT NULL,
`StaffRoleG` smallint(6) NOT NULL,
`StaffRoleH` smallint(6) NOT NULL,
`StaffRoleI` smallint(6) NOT NULL,
`StaffRoleJ` smallint(6) NOT NULL,
`StaffRoleK` smallint(6) NOT NULL,
`StaffRoleL` smallint(6) NOT NULL,
`StaffRoleM` smallint(6) NOT NULL,
`StaffRoleN` smallint(6) NOT NULL,
`StaffConnection` smallint(6) NOT NULL DEFAULT '2',
`MachineWorked` varchar(15) DEFAULT NULL,
`WorkOrderWorked` varchar(20) DEFAULT NULL,
`StaffGroup` varchar(50) NOT NULL,
`Creator` varchar(50) NOT NULL,
`Changer` varchar(50) DEFAULT NULL,
PRIMARY KEY (`AccountName`) USING BTREE
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC
CREATE TABLE `workorderb` (
`xWoNumber` varchar(20) NOT NULL,
`xMachCode` varchar(15) NOT NULL,
`xPlannedMoldCode` varchar(10) NOT NULL,
`xPartyNumber` smallint(6) NOT NULL,
`xStaffName` varchar(50) NOT NULL,
`xStopCode` smallint(6) NOT NULL,
`xStopStartTime` datetime NOT NULL,
`xStopFinishTime` datetime DEFAULT NULL,
`xStopTime` int(11) DEFAULT NULL,
PRIMARY KEY (`xMachCode`,`xStopStartTime`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC
Your query with the joins was nearly there: it was just missing a GROUP BYclause.
I have replaced workorder.* with workorder.WoNumber in the SELECT and added GROUP BY workorder.WoNumber.
You can add as many columns from workorder in the SELECT as you like but you must list them in the GROUP BY.
SELECT workorder.WoNumber,machine.MachModel,machine.MachStatus,rawmaterials.RawMaterialDescription,rawmaterials.RawMaterialColor,staff.StaffName,staff.StaffCode,SUM(workorderb.xStopTime)
FROM workorder
LEFT JOIN machine ON machine.MachCode=workorder.MachCode
LEFT JOIN rawmaterials ON rawmaterials.RawMaterialCode=workorder.ProductRawMaterial
LEFT JOIN staff ON staff.AccountName=workorder.AssignStaff
LEFT JOIN workorderb ON workorderb.xWoNumber=workorder.WoNumber
WHERE workorder.WoStatus=3
GROUP BY workorder.WoNumber \* <<= ADD OTHER COLUMNS HERE AS NEEDED *\
ORDER BY workorder.PlanProdStartDate DESC, workorder.WoSortNumber, workorder.WoNumber LIMIT 100;
db<>fiddle here
Use InnoDB, not MyISAM. MyISAM locks the entire table when INSERTing; InnoDB can often allow other threads to run when inserting.
Other notes
workorder has 4 identical indexes on wonumber; keep the PK, toss the rest. Note that a PRIMARY KEY is an index. Check the other tables for redundant Keys.
Do you need the mixture of DESC and ASC in ORDER BY PlanProdStartDate DESC, WoSortNumber, WoNumber? If not, there may be an optimization here.
As Kendle suggests, JOINs would be faster since there are cases where the same table is needed twice. If values might be missing, then LEFT might be useful; it won't change the performance.
Needed:
workorderb: INDEX(xWoNumber, xStopTime)
Is xStopTime an elapsed time? Or a time of day?
I've build web application as a tool to eliminate unnecessary data in peoples table, this application mainly to filter all data of peoples who valid to get an election rights. At first, it wasn't a problem when the main table still had few rows, but it is really bad (6 seconds) when the table is filled with about 200K rows (really worse because the table will be up to 6 million rows).
I have table design like below, and I am doing a join with 4 tables (region table start from province, city, district and town). Each region table is related to each other with their own id:
CREATE TABLE `peoples` (
`id` mediumint(8) unsigned NOT NULL AUTO_INCREMENT,
`id_prov` smallint(2) NOT NULL,
`id_city` smallint(2) NOT NULL,
`id_district` smallint(2) NOT NULL,
`id_town` smallint(4) NOT NULL,
`tps` smallint(4) NOT NULL,
`urut_xls` varchar(20) NOT NULL,
`nik` varchar(20) NOT NULL,
`name` varchar(60) NOT NULL,
`place_of_birth` varchar(60) NOT NULL,
`birth_date` varchar(30) NOT NULL,
`age` tinyint(3) NOT NULL DEFAULT '0',
`sex` varchar(20) NOT NULL,
`marital_s` varchar(20) NOT NULL,
`address` varchar(160) NOT NULL,
`note` varchar(60) NOT NULL,
`m_name` tinyint(1) NOT NULL DEFAULT '0',
`m_birthdate` tinyint(1) NOT NULL DEFAULT '0' ,
`format_birthdate` tinyint(1) NOT NULL DEFAULT '0' ,
`m_sex` tinyint(1) NOT NULL DEFAULT '0' COMMENT ,
`m_m_status` tinyint(1) NOT NULL DEFAULT '0' ,
`sex_double` tinyint(1) NOT NULL DEFAULT '0',
`id_import` bigint(10) NOT NULL,
`id_workspace` tinyint(4) unsigned NOT NULL DEFAULT '0',
`stat_valid` smallint(1) NOT NULL DEFAULT '0' ,
`add_manual` tinyint(1) unsigned NOT NULL DEFAULT '0' ,
`insert_by` varchar(12) NOT NULL,
`update_by` varchar(12) DEFAULT NULL,
`mark_as_duplicate` smallint(1) NOT NULL DEFAULT '0' ,
`mark_as_trash` smallint(1) NOT NULL DEFAULT '0' ,
`in_date_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `ind_import` (`id_import`),
KEY `ind_duplicate` (`mark_as_duplicate`),
KEY `id_workspace` (`id_workspace`),
KEY `tambah_manual` (`tambah_manual`),
KEY `il` (`stat_valid`,`mark_as_trash`,`in_date_time`),
KEY `region` (`id_prov`,`id_kab`,`id_kec`,`id_kel`,`tps`),
KEY `name` (`name`),
KEY `place_of_birth` (`place_of_birth`),
KEY `ind_birth` (`birthdate`(10)),
KEY `ind_sex` (`sex`(2))
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
town:
CREATE TABLE `town` (
`id` smallint(4) NOT NULL,
`id_district` smallint(2) NOT NULL,
`id_city` smallint(2) NOT NULL,
`id_prov` smallint(2) NOT NULL,
`name_town` varchar(60) NOT NULL,
`handprint` blob,
`pps_1` varchar(60) DEFAULT NULL,
`pps_2` varchar(60) DEFAULT NULL,
`pps_3` varchar(60) DEFAULT NULL,
`tpscount` smallint(2) DEFAULT NULL,
`pps_4` varchar(60) DEFAULT NULL,
`pps_5` varchar(60) DEFAULT NULL,
PRIMARY KEY (`id_prov`,`id_kab`,`id_kec`,`id`),
KEY `name_town` (`name_town`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
and the query like
SELECT `E`.`id`, `E`.`id_prov`, `E`.`id_city`, `E`.`id_district`, `E`.`id_town`,
`B`.`name_prov`,`C`.`name_city`,`D`.`name_district`, `A`.`name_town`,
`E`.`tps`, `E`.`urut_xls`, `E`.`nik`,`E`.`name`,`E`.`place_of_birth`,
`E`.`birth_date`, `E`.age, `E`.`sex`, `E`.`marital_s`, `E`.`address`,
`E`.`note`
FROM peoples E
JOIN test_prov B ON E.id_prov = B.id
JOIN test_city C ON E.id_city = C.id
AND (C.id_prov=B.id)
JOIN test_district D ON E.id_district = D.id
AND ((D.id_city = C.id) AND (D.id_prov= B.id))
JOIN test_town A ON E.id_town = A.id
AND ((A.id_district = D.id)
AND (A.id_city = C.id)
AND (A.id_prov = B.id))
AND E.stat_valid=1
AND E.mark_as_trash=0
mark_as_trash is a mark column which only contain 1 and zero just to know if the data has been mark as a deleted record, and stat_valid is the filtered result value - if value is 1 then the data is valid to get the rights of election.
I've tried to see the explain but no column is used as an index lookup. I believe that's the problem why the application so slow in 200K rows. The query above only shows two conditions, but the application has a feature to filter by name, place of birth, birth date, age with ranges and so on.
How can I make this perform better?
Can a city be in two provinces? If not then why do you check C.id_prov=B.id if E.id_city = C.id should give you just one row?
Also it seems that your query is slow because you're selecting 200k rows. Indexes will improve performance but do you really need all the rows at once? You should use pagination (limit, offset).
The following query is timing out after 600 seconds.
update placed p
,Results r
set p.position = r.position
where p.competitor = r.competitor
AND p.date = r.date
AND REPLACE(p.time,":","") = r.time;
The structure is as follows:
'CREATE TABLE `placed` (
`idplaced` varchar(50) DEFAULT NULL,
`date` decimal(8,0) DEFAULT NULL,
`time` varchar(45) DEFAULT NULL,
`field1` varchar(45) DEFAULT NULL,
`competitor` varchar(45) DEFAULT NULL,
`field2` int(2) DEFAULT NULL,
`field3` varchar(45) DEFAULT NULL,
`field4` varchar(45) DEFAULT NULL,
`field5` decimal(6,2) DEFAULT NULL,
`field6` decimal(10,2) DEFAULT NULL,
`field7` decimal(6,2) DEFAULT NULL,
`field8` char(1) DEFAULT NULL,
`field9` varchar(45) DEFAULT NULL,
`position` char(4) DEFAULT NULL,
`field10` decimal(6,2) DEFAULT NULL,
`field11` char(1) DEFAULT NULL,
`field12` char(1) DEFAULT NULL,
`field13` decimal(6,2) DEFAULT NULL,
`field14` decimal(6,2) DEFAULT NULL,
`field15` decimal(6,2) DEFAULT NULL,
`field16` decimal(6,2) DEFAULT NULL,
`field17` decimal(6,2) DEFAULT NULL,
`field18` char(1) DEFAULT NULL,
`field19` char(20) DEFAULT NULL,
`field20` char(1) DEFAULT NULL,
`field21` char(5) DEFAULT NULL,
`field22` char(5) DEFAULT NULL,
`field23` int(11) DEFAULT NULL
PRIMARY KEY (`idplaced`),
UNIQUE KEY `date_time_competitor_field18_combo` (`date`,`time`,`competitor`,`field18`)
) ENGINE=InnoDB AUTO_INCREMENT=100688607 DEFAULT CHARSET=latin1;
CREATE TABLE `results` (
`idresults` int(11) NOT NULL AUTO_INCREMENT,
`date` char(8) DEFAULT NULL,
`time` char(4) DEFAULT NULL,
`field1` varchar(45) DEFAULT NULL,
`competitor` varchar(45) DEFAULT NULL,
`position` char(4) DEFAULT NULL,
`field2` varchar(45) DEFAULT NULL,
`field3` decimal(2,0) DEFAULT NULL,
PRIMARY KEY (`idresults`)
) ENGINE=InnoDB AUTO_INCREMENT=6644 DEFAULT CHARSET=latin1;
The PLACED table has 65,000 records, the RESULTS table has 9,000 records.
I am assuming the solution involves a JOIN statement of some descript, and I have tried taking several suggestions from this site, but am simply not finding the answer I am looking for. Simply put, I would be grateful for suggestions on this. I can put up example tables / create table code if requried.
The index cannot be used efficiently to perform the join because of your REPLACE operation.
I'd suggest creating an index with the columns in the following slightly different order:
(date, competitor, time, position)
It may also help to add this index on both tables.
It would be even better if you could modify the data in the database so that the data in the time column was stored in the same format in both tables.
First of all, you'd better send us your full tables description, using
show create table
Second, you'd better use join syntax :
update placed p
join Results r on r.competitor = p.competitor
set p.position = r.position
where p.date = r.date
AND REPLACE(p.time,":","") = r.time;
Hope this will help.
I have table named as contacts which has nearly 1.2 million records we use
MyIsam engine whenever we query this table mysql hangs down so now we are trying our hands with Innodb engine so that if it slows down, but it will not hang up for others
So we want make fast with Myisam we tried many indexes on this table but it goes down and hangs the system
What should be done to make it more faster and it should not hang up the system
This is the table:
CREATE TABLE `contacts` (
`id` varchar(36) NOT NULL,
`deleted` tinyint(1) NOT NULL default '0',
`date_entered` datetime NOT NULL default '0000-00-00 00:00:00',
`date_modified` datetime NOT NULL default '0000-00-00 00:00:00',
`modified_user_id` varchar(36) default NULL,
`assigned_user_id` varchar(36) default NULL,
`created_by` varchar(36) default NULL,
`team_id` varchar(36) default NULL,
`salutation` varchar(5) default NULL,
`first_name` varchar(100) default '',
`last_name` varchar(100) default '',
`username` varchar(25) default '',
`lead_source` varchar(100) default NULL,
`title` varchar(50) default NULL,
`department` varchar(100) default NULL,
`reports_to_id` varchar(36) default NULL,
`birthdate` date default NULL,
`do_not_call` char(3) default '0',
`phone_home` varchar(25) default NULL,
`phone_mobile` varchar(25) default NULL,
`phone_work` varchar(25) default '',
`phone_other` varchar(25) default NULL,
`phone_fax` varchar(25) default '',
`email1` varchar(100) default '',
`email2` varchar(100) default NULL,
`assistant` varchar(75) default NULL,
`assistant_phone` varchar(25) default NULL,
`email_opt_out` char(3) default 'off',
`primary_address_street` varchar(150) default NULL,
`primary_address_city` varchar(100) default NULL,
`primary_address_state` varchar(100) default NULL,
`primary_address_postalcode` varchar(20) default NULL,
`primary_address_country` varchar(100) default NULL,
`alt_address_street` varchar(150) default NULL,
`alt_address_city` varchar(100) default NULL,
`alt_address_state` varchar(100) default NULL,
`alt_address_postalcode` varchar(20) default NULL,
`alt_address_country` varchar(100) default NULL,
`description` text,
`portal_name` varchar(255) default NULL,
`portal_active` tinyint(1) NOT NULL default '0',
`portal_app` varchar(255) default NULL,
`salesforceid` varchar(36) default NULL,
`phone_direct` varchar(25) default NULL,
`invalid_email` tinyint(1) default '0',
`parent_is_lead` char(3) default 'no',
`advisory_board_member` varchar(25) default NULL,
`direct_marketing` varchar(25) default NULL,
`efx_id` varchar(36) default NULL,
`fax_opt_out` char(3) default 'off',
`ppc_keyword` varchar(50) default NULL,
`status` varchar(25) default NULL,
`web_form` varchar(50) default NULL,
`efx_export_date` datetime default NULL,
`bmtn` varchar(225) default '',
`employee_location` varchar(50) default NULL,
`pronunciation` varchar(250) default NULL,
`duplicate_of` varchar(36) default NULL,
`job_category` varchar(50) default NULL,
`last_ska_upload_key` varchar(50) default NULL,
`persid` varchar(36) default NULL,
`last_web_upload_key` varchar(50) default NULL,
`last_webinar_upload_key` varchar(50) default NULL,
`primary_address_latitude` float default NULL,
`primary_address_longitude` float default NULL,
`first_name_soundex` varchar(30) default NULL,
`last_name_soundex` varchar(30) default NULL,
`primary_address_street_soundex` varchar(30) default NULL,
`campaign_id` varchar(36) default NULL,
`portal_password` varchar(32) default NULL,
`pss_branch` varchar(40) default NULL,
`pss_id` int(12) default NULL,
`source_detail` varchar(100) default NULL,
`source` varchar(100) default NULL,
`pss_region` varchar(30) default NULL,
`source_added` datetime default NULL,
`terminated_user` char(3) default 'off',
`invite_opt_out` char(3) default 'off',
`newsletter_opt_out` char(3) default 'off',
`stream_opt_out` char(3) default 'off',
PRIMARY KEY (`id`),
KEY `idx_contacts_del_last` (`deleted`,`last_name`),
KEY `idx_cont_del_reports` (`deleted`,`reports_to_id`,`last_name`),
KEY `idx_contact_del_team` (`deleted`,`team_id`),
KEY `idx_contact_salesforceid` (`salesforceid`),
KEY `idx_contacts_username` (`username`),
KEY `idx_email_opt_out` (`email_opt_out`),
KEY `idx_primary_address_street` (`primary_address_street`),
KEY `idx_primary_address_city` (`primary_address_city`),
KEY `idx_primary_address_state` (`primary_address_state`),
KEY `idx_primary_address_postalcode` (`primary_address_postalcode`),
KEY `idx_primary_address_country` (`primary_address_country`),
KEY `idx_modified_user_id` (`modified_user_id`),
KEY `idx_assigned_user_id` (`assigned_user_id`),
KEY `idx_created_by` (`created_by`),
KEY `idx_team_id` (`team_id`),
KEY `idx_reports_to_id` (`reports_to_id`),
KEY `idx_contacts_efx_id` (`efx_id`),
KEY `idx_contacts_title1` (`title`,`deleted`),
KEY `idx_contacts_email1` (`email1`),
KEY `idx_contacts_email2` (`email2`),
KEY `idx_contacts_job_category` (`job_category`),
KEY `idx_contacts_first_name_sdx` (`first_name_soundex`),
KEY `idx_contacts_primary_street_sdx` (`primary_address_street_soundex`),
KEY `idx_contacts_last_name_sdx` (`last_name_soundex`),
KEY `idx_contacts_portal_name` (`portal_name`),
KEY `idx_contacts_portal_active` (`portal_active`),
KEY `idx_contacts_del_last_first` (`deleted`,`last_name`,`first_name`),
KEY `idx_contacts_del_first` (`deleted`,`first_name`),
KEY `idx_pss_id` (`pss_id`),
KEY `idx_phone_work_last_name_first_name_deleted` (`phone_work`,`last_name`,`first_name`,`deleted`),
KEY `idx_phone_work_last_name_first_name_deleted_sdx` (`phone_work`,`last_name_soundex`,`first_name_soundex`,`deleted`),
KEY `idx_email1_last_name_first_name_deleted` (`email1`,`last_name`,`first_name`,`deleted`),
KEY `idx_email1_last_name_first_name_deleted_sdx` (`email1`,`last_name_soundex`,`first_name_soundex`,`deleted`),
KEY `idx_phone_fax_last_name_first_name_deleted` (`phone_fax`,`last_name`,`first_name`,`deleted`),
KEY `idx_phone_fax_last_name_first_name_deleted_sdx` (`phone_fax`,`last_name_soundex`,`first_name_soundex`,`deleted`),
KEY `idx_phone_work_last_name_deleted` (`phone_work`,`last_name`,`deleted`),
KEY `idx_phone_work_last_name_deleted_sdx` (`phone_work`,`last_name_soundex`,`deleted`),
KEY `idx_email1_last_name_deleted` (`email1`,`last_name`,`deleted`),
KEY `idx_email1_last_name_deleted_sdx` (`email1`,`last_name_soundex`,`deleted`),
KEY `idx_phone_fax_last_name_deleted` (`phone_fax`,`last_name`,`deleted`),
KEY `idx_phone_fax_last_name_deleted_sdx` (`phone_fax`,`last_name_soundex`,`deleted`),
KEY `idx_email1_first_name_deleted` (`email1`,`first_name`,`deleted`),
KEY `idx_email1_first_name_deleted_sdx` (`email1`,`first_name_soundex`,`deleted`),
KEY `idx_phone_fax_first_name_deleted` (`phone_fax`,`first_name`,`deleted`),
KEY `idx_phone_fax_first_name_deleted_sdx` (`phone_fax`,`first_name_soundex`,`deleted`),
KEY `idx_email1_deleted` (`email1`,`deleted`),
KEY `idx_last_name_first_name_deleted_sdx` (`last_name_soundex`,`first_name_soundex`,`deleted`),
KEY `idx_phone_mobile_deleted` (`phone_mobile`,`deleted`,`id`),
KEY `idx_first_name_bmtn` (`first_name`,`bmtn`),
KEY `idx_first_name_bmtn_email1` (`first_name`,`bmtn`,`email1`),
KEY `idx_bmtn_email1` (`bmtn`,`email1`),
KEY `idx_deleted` (`deleted`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
-
SELECT acc.id, acc.name, con_reports_to.first_name, con_reports_to.last_name
from contacts
left join accounts_contacts a_c on a_c.contact_id = '9802f40d-78bb-8dd4-dfaa-43f1064ccd5e' and a_c.deleted=0
left join accounts acc on a_c.account_id = acc.id and acc.deleted=0
left join contacts con_reports_to on con_reports_to.id = contacts.reports_to_id
where contacts.id = '9802f40d-78bb-8dd4-dfaa-43f1064ccd5e'
I suspect the assertion "whenever we query this table mysql hangs down" is an overbid -- for example, with MyISAM, SELECT COUNT(*) FROM TheTable should be very fast, essentially "no matter what". Sure, some queries will be slow -- especially if the table is not indexed properly for the queries, or if MySQL's alleged optimizer is picking the wrong strategy (but you could give it hints).
Why don't you show us the CREATE TABLE (including indices), a couple of the queries that take too long, ideally a precise measure of how long they take, and the output of EXPLAIN SELECT (&c) for those couple queries -- I bet we could really be of some help then!
Edit: the CREATE TABLE essentially shows that the table is just too "broad" -- far too many columns -- to expect decent performance (even though no queries were shown). The schema needs a redesign, breaking up chunks of this huge monolithic table (e.g., the address-related information) into other auxiliary tables. Exactly how to best do it depends entirely on the queries that are most important to optimize, so, not knowing the queries in question, I'm not even going to attempt the task.
Edit again: so the query has been posted and uses other tables, accounts and account_contacts, as well as the hugely broad contacts one described; the query as posted (trying to make sense of it by formatting &c) is:
SELECT acc.id, acc.name, con_reports_to.first_name, con_reports_to.last_name
FROM contacts
LEFT JOIN accounts_contacts a_c
ON a_c.contact_id = '9802f40d-78bb-8dd4-dfaa-43f1064ccd5e' AND
a_c.deleted=0
LEFT JOIN accounts acc
ON a_c.account_id = acc.id AND
acc.deleted=0
LEFT JOIN contacts con_reports_to
ON con_reports_to.id = contacts.reports_to_id
WHERE contacts.id = '9802f40d-78bb-8dd4-dfaa-43f1064ccd5e'
Why the LEFT JOINs here instead of normal INNER joins? Is it possible in each case that there's no corresponding row on the right-hand-side table? For example, if there's no line in a_c with the given values for contact_id and deleted, then all the fields of a_c in the first LEFT JOIN will be NULL, so there can be no correspondence for acc either: is it important to emit NULL, NULL as the first two columns in this case? Moreover the JOIN conditions for a_c an acc make no reference at all to contacts, so this will be a cartesian product: every line selected from acc, if any, will pair up with every line selected from con_reports_to. So the a_c/acc query could be entirely separated from the one on contacts and con_reports, presumably ligthtening the query considerably (the two logically separate results could of course easily be put together again in the client).
What does EXPLAIN SELECT say for this complex query and what does it say for the two lighter-weight separate ones I'm suggesting? What indices are on the accounts and account_contact tables?
horizontal splitting? though i guess 1.2 million records are not that much to introduce horizontal splitting.. try to locate the bottom neck... also the problem may lie with your hardware as well for example harddisk almost full etc.