MySQL Query is slow (Joins/counts) - mysql

I have this query:
SELECT `assemblies`.`id`,
`assemblies`.`type`,
`assemblies`.`champion`,
`assemblies`.`name`,
`assemblies`.`author`,
`assemblies`.`githublastmod`,
( assemblies.forum IS NOT NULL ) AS forumExists,
Count(votes.id) AS votesCount,
Count(install_clicks.id) AS installCount,
Count(github_clicks.id) AS githubCount,
Count(forum_clicks.id) AS forumCount
FROM `assemblies`
INNER JOIN `votes`
ON `votes`.`assembly` = `assemblies`.`id`
INNER JOIN `install_clicks`
ON `install_clicks`.`assembly` = `assemblies`.`id`
INNER JOIN `github_clicks`
ON `github_clicks`.`assembly` = `assemblies`.`id`
INNER JOIN `forum_clicks`
ON `forum_clicks`.`assembly` = `assemblies`.`id`
WHERE `assemblies`.`type` = 'utility'
AND Unix_timestamp(Date(assemblies.githublastmod)) > '1419536536'
GROUP BY `assemblies`.`id`
ORDER BY `votescount` DESC,
`githublastmod` DESC
For some reason this query is very slow, I'm using the database engine MyISAM. I hope someone can help me out here :)
Explain command:

I believe this is a case where making the subqueries for the counts will make it run a lot faster (and the values will be correct).
The problem with the original query is the explosion of the number of intermediate rows: For each 'assembly', there were n1 votes, n2 installs, etc. That led to n1*n2*... rows per assembly.
SELECT `assemblies`.`id`, `assemblies`.`type`, `assemblies`.`champion`,
`assemblies`.`name`, `assemblies`.`author`, `assemblies`.`githublastmod`,
( assemblies.forum IS NOT NULL ) AS forumExists,
( SELECT Count(*)
FROM votes
WHERE `assembly` = `assemblies`.`id`
) AS votesCount,
( SELECT Count(*)
FROM install_clicks
WHERE `assembly` = `assemblies`.`id`
) AS installCount,
( SELECT Count(*)
FROM github_clicks
WHERE `assembly` = `assemblies`.`id`
) AS githubCount,
( SELECT Count(*)
FROM forum_clicks.id
WHERE `assembly` = `assemblies`.`id`
) AS forumCount
FROM `assemblies`
WHERE `assemblies`.`type` = 'utility'
AND Unix_timestamp(Date(assemblies.githublastmod)) > '1419536536'
ORDER BY `votescount` DESC, `githublastmod` DESC
Each secondary table needs an INDEX starting with assembly.

Your problem should be fixed using the right indices:
CREATE INDEX index_name_1 ON `votes`(`assembly`);
CREATE INDEX index_name_2 ON `install_clicks`(`assembly`);
CREATE INDEX index_name_3 ON `github_clicks`(`assembly`);
CREATE INDEX index_name_4 ON `forum_clicks`(`assembly`);
Try your query again after creating these indices and it should be quite faster.

Related

Mysql tekes too much time to excute sql query, based on multiple join

My Sql query takes more time to execute from mysql database server . There are number of tables are joined with sb_tblproperty table. sb_tblproperty is main table that contain more than 1,00,000 rows . most of table contain 50,000 rows.
How to optimize my sql query to fast execution. I have also used indexing.
indexing Explain - query - structure
SELECT `t1`.`propertyId`, `t1`.`projectId`,
`t1`.`furnised`, `t1`.`ownerID`, `t1`.`subType`,
`t1`.`fors`, `t1`.`size`, `t1`.`unit`,
`t1`.`bedrooms`, `t1`.`address`, `t1`.`dateConfirm`,
`t1`.`dateAdded`, `t1`.`floor`, `t1`.`priceAmount`,
`t1`.`priceRate`, `t1`.`allInclusive`, `t1`.`booking`,
`t1`.`bookingRate`, `t1`.`paidPercetage`,
`t1`.`paidAmount`, `t1`.`is_sold`, `t1`.`remarks`,
`t1`.`status`, `t1`.`confirmedStatus`, `t1`.`source`,
`t1`.`companyName` as company, `t1`.`monthly_rent`,
`t1`.`per_sqft`, `t1`.`lease_duration`,
`t1`.`lease_commencement`, `t1`.`lock_in_period`,
`t1`.`security_deposit`, `t1`.`security_amount`,
`t1`.`total_area_leased`, `t1`.`lease_escalation_amount`,
`t1`.`lease_escalation_years`, `t2`.`propertyTypeName` as
propertyTypeName, `t3`.`propertySubTypeName` subType,
`t3`.`propertySubTypeId` subTypeId, `Owner`.`ContactName`
ownerName, `Owner`.`companyName`, `Owner`.`mobile1`,
`Owner`.`otherPhoneNo`, `Owner`.`mobile2`,
`Owner`.`email`, `Owner`.`address` as caddress,
`Owner`.`contactType`, `P`.`projectName` as project,
`P`.`developerName` as developer, `c`.`name` as city,
if(t1.projectId="", group_concat( distinct( L.locality)),
group_concat( distinct(L2.locality))) as locality, `U`.`firstname`
addedBy, `U1`.`firstname` confirmedBy
FROM `sb_tblproperty` as t1
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
JOIN `sb_propertytype` t2 ON `t1`.`propertyType`= `t2`.`propertyTypeId`
JOIN `sb_propertysubtype` t3 ON `t1`.`subType` =`t3`.`propertySubTypeId`
LEFT JOIN `sb_tbluser` U ON `t1`.`addedBy` = `U`.`userId`
LEFT JOIN`sb_tbluser` U1 ON `t1`.`confirmedBy` = `U1`.`userId`
LEFT JOIN `sb_tblproject` P ON `P`.`id` = `t1`.`projectId` LEFT
JOIN `sb_tblpropertylocality` PL ON `t1`.`propertyId` = `PL`.`propertyId`
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
LEFT JOIN `sb_tblprojectlocality` PROL ON `PROL`.`projectId` = `P`.`id`
LEFT JOIN `sa_localitiez` L2 ON `L2`.`id` = `PROL`.`localityId`
LEFT JOIN `sb_tblfloor` F
ON `F`.`floorName` =`t1`.`floor`
WHERE `t1`.`is_sold` != '1' GROUP BY `t1`.`propertyId`
ORDER BY `t1`.`dateConfirm`
DESC LIMIT 1000
Please provide the EXPLAIN.
Meanwhile, try this:
SELECT ...
FROM (
SELECT propertyId
FROM sb_tblproperty
WHERE `is_sold` = 0
ORDER BY `dateConfirm` DESC
LIMIT 1000
) AS x
JOIN `sb_tblproperty` as t1 ON t1.propertyId = x.propertyId
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
...
LEFT JOIN `sb_tblfloor` F ON `F`.`floorName` =`t1`.`floor`
ORDER BY `t1`.`dateConfirm` DESC -- yes, again
Together with
INDEX(is_sold, dateConfirm)
How can t1.projectId="" ? Isn't projectId the PRIMARY KEY? (This is one of many reasons for needing the SHOW CREATE TABLE.)
If my suggestion leads to "duplicate" rows (that is, multiple rows with the same propertyId), don't simply add back the GROUP BY propertyId. Instead figure out why, and avoid the need for the GROUP BY. (That is probably the performance issue.)
A likely case is the GROUP_CONCAT. A common workaround is to change from
GROUP_CONCAT( distinct( L.locality)) AS Localities,
...
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
to
( SELECT GROUP_CONCAT(distinct locality)
FROM sa_localitiez
WHERE id = PL.localityId ) AS Localities
...
# and remove the JOIN

SQL Query behavior

I'm bogged in trying to figure out why query a is returning different records than query b. Both queries have seemingly same purpose yet a is returning 500 and b 3500.
this is query a:
SELECT DISTINCT ODE.OrderBillToID
FROM APTIFY.dbo.vwVwOrderDetailsKGExtended ODE
WHERE ProductID IN (2022, 1393)
AND LTRIM(RTRIM(ODE.OrderStatus)) <> 'Cancelled'
AND LTRIM(RTRIM(ODE.OrderType)) <> 'Cancellation'
AND LTRIM(RTRIM(ODE.cancellationStatus)) <> 'FULLY CANCELLED'
UNION
SELECT DISTINCT ID
FROM APTIFY.dbo.vwPersons WHERE City = 'A'
UNION
SELECT DISTINCT RecordID
FROM APTIFY.dbo.vwTopicCodeLinks WHERE TopicCodeID = 16 AND Value = 'Yes, Please'
query b:
SELECT
APTIFY..vwPersons.ID
FROM
APTIFY..vwPersons
WHERE
( APTIFY..vwPersons.ID IN (
SELECT
vwMeetingRegistrants.ID
FROM
APTIFY.dbo.vwMeetings vwMeetings
INNER JOIN APTIFY.dbo.vwMeetingRegistrants vwMeetingRegistrants
ON vwMeetings.ID=vwMeetingRegistrants.ActualMeetingID WHERE
vwMeetings.ProductID = 2022
)
OR
APTIFY..vwPersons.ID IN (
SELECT
vwMeetingRegistrants.ID
FROM
APTIFY.dbo.vwMeetings vwMeetings
INNER JOIN APTIFY.dbo.vwMeetingRegistrants vwMeetingRegistrants
ON vwMeetings.ID=vwMeetingRegistrants.ActualMeetingID WHERE
vwMeetings.ProductID = 1393
)
OR
APTIFY..vwPersons.City = N'Albany' )
OR
((
APTIFY..vwPersons.ID IN (
SELECT
RecordID
FROM
APTIFY.dbo.vwTopicCodeLinks vwTopicCodeLinks
WHERE
vwTopicCodeLinks.TopicCodeID = 16
)
AND
APTIFY..vwPersons.ID IN (
SELECT
RecordID
FROM
APTIFY.dbo.vwTopicCodeLinks vwTopicCodeLinks
WHERE
vwTopicCodeLinks.Value = N'Yes, Please'
) )
)
vwMeetingsRegistrants from the b query are producing the same records as orderkgdetailsextended from query. I cannot see ANY difference in those queries - which perhaps shows my lack of understanding the query behaviour.
BIG Thanks for any points guys! :)
As it came out, incorrectly structured query is a result of badly configured application, Aptify.

How to rewrite this statement with JOIN, to make it faster?

I have a mySQL query, it is working, but when it comes to 1.6 million executions, it does not meet the performance I need, because of the constant re-executions of NOT EXIST(new query)
INSERT INTO `votes` (`representatives_rID`, `events_eID`, `voteResult`)
SELECT `representatives`.`rID`,`events`.`eID`,?
FROM `representatives`, `events`
WHERE `events`.`eventDateTime` = ?
AND `representatives`.`rID` = ?
AND NOT EXISTS (SELECT * FROM `votes`
WHERE `votes`.`representatives_rID` = `representatives`.`rID`
AND `votes`.`events_eID` = `events`.`eID`);
In schematic:
if (input one is in `representatives` AND input two is in `events` AND there is no row in `votes` that contains ( `votes`.`representatives_rID` = `representatives`.`rid` and `votes`.`events_eID` = `events`.`eID)) {
insert a row into votes containing `eid` from `events` and `rid` from `representatives` and input three;
}
Is there any way of making this faster, possibly with join?
INSERT IGNORE INTO votes
(representatives_rID
,events_eID
,voteResult
)
SELECT r.rID
, e.eID
, ?
FROM representatives r
CROSS
JOIN events e
ON e.eventDateTime = ?
AND r.rID = ?;
?

MYSQL Select As Case when number of rows statement

I'm trying to find the number of notifications for each user, I am having a little problem, I had the query working perfect for what I needed it for, then I changed my table around just a little bit.
Working query:
$numNotifications = mysql_num_rows(mysql_query("
SELECT
N.*,
P.*
FROM
notifications N,
posts P
WHERE
N.userID='$session'
AND
(
N.action='1' OR N.action='2'
)
AND
N.uniqueID=P.id AND P.state='0'"
));
However, uniqueID is now different for some rows. when N.aciton is "1" then N.uniqueID should be compared to P.id, however when N.action is "2" it should compare a row in that table with P.id.
Example query, (that SHOULD work, but doesn't)
$numNotifications = mysql_num_rows(mysql_query("
SELECT
N.*,
P.*,
C.*,
(CASE WHEN (
N.action = 2 AND N.state = 0
)
THEN
C.postID ELSE N.uniqueID END
) AS postId
FROM
notifications N,
posts P,
comments C
WHERE
N.userID='$session'
AND
(
N.action='1' OR N.action='2'
)
AND
postId=P.id AND P.state='0'"
));
diagram of my 3 table structures:
http://i41.tinypic.com/nyzolg.png
here you go :)
SELECT
COUNT(`notif_id`) AS `number_of_new_notifications`
FROM
(
SELECT
`notifications`.`id` AS `notif_id`
FROM
`notifications`
JOIN
`posts`
ON
`notifications`.`uniqueID`=`posts`.`id`
WHERE
`notifications`.`userID`='$session'
AND
`notifications`.`action`=1
AND
`notifications`.`state`=0
AND
`posts`.`state`=0
UNION ALL
SELECT
`notifications`.`id` AS `notif_id`
FROM
`notifications`
JOIN
`comments`
ON
`notifications`.`uniqueID`=`comments`.`id`
JOIN
`posts`
ON
`comments`.`postID`=`posts`.`id`
WHERE
`notifications`.`userID`='$session'
AND
`notifications`.`action`=2
AND
`notifications`.`state`=0
AND
`comments`.`state`=0
AND
`posts`.`state`=0
) AS notification_ids;

Finding unmatched records in MYSQL in a m2m scenario

I have 3 tables in a m2m relationship in a mysql database as follows:
tbltest (test_id, testdescription)
tblprofile (profile_id, profiledescription)
tbltestprofile(testprofile_id,test_id,profile_id)
I need to display test_id and description from the test table where I have no matching records in tbltestprofile and where the profile_id = x
I've tried various combinations using NOT IN but without the desired results. Can anyone assist?
SELECT test_id
, testdescription
FROM tbltest AS t
WHERE NOT EXISTS
( SELECT *
FROM tbltestprofile tp
WHERE t.test_id = tp.test_id
AND tp.profile_id = X
)
Sidenote. It would help if you kept table (and field) names simple and not add the tbl prefix, like test, profile, testprofile. A simple 3-table join you have probably used:
SELECT tbltest.test_id
, tbltest.testdescription
, tblprofile.profile_id
, tblprofile.profiledescription
FROM tbltest
JOIN tbltestprofile
ON tbltest.test_id = tbltestprofile.test_id
JOIN tblprofile
ON tblprofile.profile_id = tbltestprofile.profile_id
ORDER BY tblprofile.profiledescription
makes me rather dizzy. Wouldn't it be better like this? Even without aliases:
SELECT test.id AS test_id
, test.description AS test
, profile.id AS profile_id
, profile.description AS profile
FROM test
JOIN testprofile
ON test.id = testprofile.test_id
JOIN profile
ON profile.id = testprofile.profile_id
ORDER BY profile.description
SELECT test_id, testdescription
FROM tbltest
LEFT JOIN tbltestprofile ON tbltest.test_id = tbltestprofile.test_id
WHERE tbltestprofile.test_id IS NULL;