i am writing a mysql query as below
SELECT `user_master`.`first_name`,
`city_name`,
`user_master`.`last_name`,
`user_master`.`user_master_id`,
`account_management_master`.`account_name`,
`donation_receipt_info`.`receipt_temple_id`,
date(dt) AS dt,
SUM(`donation_receipt_info`.`amount`) AS amount
FROM (`donation_receipt_info`)
JOIN `donation_receipt_master` ON donation_receipt_master`.`receipt_id`=`donation_receipt_info`.`receipt_id`
JOIN `account_management_master` ON `account_management_master`.`account_id`=`donation_receipt_info`.`account_id`
JOIN `user_master` ON `user_master`.`user_master_id`=`donation_receipt_master`.`user_master_id`
JOIN `user_address_info` ON `user_address_info`.`user_master_id`=`user_master`.`user_master_id`
JOIN `city_master` ON `city_master`.`city_id`=`user_address_info`.`city_id`
WHERE `donation_receipt_info`.`temple_id` = '1'
GROUP BY `donation_receipt_info`.`receipt_id`,
`donation_receipt_info`.`account_id`
the table donation_receipt_info and master have approx 42k results the query is taking way to much time of about 5 to 6 minutes to execute in mysql itself.
can someone please help me optimize the query, any help or suggestion would be very helpful
Thanks.
First, your query is impossible to read. You should format it and learn to use table aliases:
SELECT um.first_name, city_name, um.last_name, um.user_master_id, amm.account_name,
dri.receipt_temple_id, date(dt) AS dt, SUM(dri.amount) AS amount
FROM donation_receipt_info dri JOIN
donation_receipt_master drm
ON drm.receipt_id = dri.receipt_id JOIN
account_management_master amm
ON amm.account_id = dri.account_id JOIN
user_master um
ON um.user_master_id = drm.user_master_id JOIN
user_address_info uai
ON uai.user_master_id = um.user_master_id JOIN
city_master cm
ON cm.city_id = uai.city_id
WHERE dri.temple_id = '1'
GROUP BY dri.receipt_id, dri.account_id;
Next. Do all the tables have the obvious indexes? That is, each table appears to have an id and these should be declared as keys (primary keys preferably). For instance, city_master(city_id).
Next, there should be an index on donation_receipt_info(temple_id, receipt_id, account_id). This should help with the where. Note: if temple_id is really an integer, the where clause should be expressed as WHERE dri.temple_id = 1 -- no quotes. You don't want MySQL to get confused and decide not to use the index.
These changes will probably help. 5-6 minutes seems like a long time for such a query.
Related
Now I have a Query that runs 50 minutes on Mysql database and I can't accept that...
I want this process can running under 15 minutes....
insert into appianData.IC_DeletedRecords
(sourcetableid,
concatkey,
sourcetablecode)
select mstid,
concatkey,
'icmstlocationheader' as sourcetablecode
from appianData.IC_MST_LocationHeader
where concatkey not in(select concatkey
from appianData.IC_PURGE_LocationHeader)
The "sourcetableid" and "mstid" are unique.
So what is the best way to add INDEX or optimize on this?
Thank you
I would write the select as:
select mstid, concatkey, 'icmstlocationheader' as sourcetablecode
from appianData.IC_MST_LocationHeader lh
where not exists (select 1
from appianData.IC_PURGE_LocationHeader lhp
where lhp.concatkey = lh.concatkey
);
Then you want an index on IC_PURGE_LocationHeader(concatkey).
Since it is a NOT IN condition you should be able to use a "LEFT JOIN ... WHERE rightTable has no match" without concern for multiple matches inflating the results.
INSERT INTO appianData.IC_DeletedRecords (sourcetableid, concatkey, sourcetablecode)
SELECT m.mstid, m.concatkey, 'icmstlocationheader' as sourcetablecode
FROM appianData.IC_MST_LocationHeader AS m
LEFT JOIN appianData.IC_PURGE_LocationHeader AS p
ON m.concatkey = p.concatkey
WHERE p.concatkey IS NULL
;
With this version query, or the one you presented in the question, indexes on concatkey in both source tables should help significantly.
I have a SQL query that links 2 tables to provide the data if a horse is in both the tables:
SELECT ProformSystem.TheDate as racedate,
ProformSystem.Course as course,
ProformSystem.TheTime as thetime,
ProformSystem.Horse as horse,
ATRSpeedRatings.rank as rank
FROM ATRSpeedRatings
INNER JOIN ProformSystem ON (ATRSpeedRatings.Horse = trim(ProformSystem.Horse)) AND (ATRSpeedRatings.TheDate = ProformSystem.TheDate) order by ProformSystem.TheTime;"
Is it possible that if the horse in ProformSystem.Horse is NOT in ATRSpeedRatings.Horse then I just make rank = 0 as a default value, or would I need to run a separate query?
This way I can display all horses from ProformSystem even if they don't have a rank in ATRSpeedRatings.
I think you want a left join and coalesce():
SELECT ps.TheDate as racedate, ps.Course as course, ps.TheTime as thetime,
ps.Horse as horse, COALESCE(sr.rank, 0) as rank
FROM ProformSystem ps LEFT JOIN
ATRSpeedRatings sr
ON sr.Horse = TRIM(ps.Horse) AND sr.TheDate = ps.TheDate)
ORDER BY ps.TheTime;
Note that this query uses table aliases. These make the query easier to write and to read.
Also, the JOIN condition sr.Horse = trim(ps.Horse) is highly suspect. You should fix the data so there are no spaces in ProformSystem. Fixing the data is more efficient and it will prevent problems on future queries.
I am working on project management tool and while generating reports through admin end the loading time for the data from the database is very much > 5 minutes. There are few points which I know would help me to increase the performance but right now I need to have the help in SELECT query
SELECT timesheet_client.organisation as client_name, timesheet_project.title as project_name,
timesheet_task.name as task, CONCAT(timesheet_user.first_name, ' ', timesheet_user.last_name) as resource_name,
timesheet_user.bill_factor, timesheet_client.client_type, sum(spent) as spent, sum(delivered_hours) as delivered_hours,
sum(billable_hours) as billable_hours, comments, color, lock_color, updated_by, updated_by_date, timesheet_user_psdb.grp_id,
timesheet_user_psdb.client_id, timesheet_user_psdb.proj_id, timesheet_user_psdb.task_id, timesheet_user_psdb.uid,
timesheet_user_grp.grp_name
FROM timesheet_user_psdb,timesheet_user, timesheet_client,
timesheet_project,timesheet_task,timesheet_user_grp
WHERE timesheet_user.username=timesheet_user_psdb.uid and
timesheet_client.client_id=timesheet_user_psdb.client_id and timesheet_project.proj_id=timesheet_user_psdb.proj_id and
timesheet_task.task_id = timesheet_user_psdb.task_id and timesheet_user_grp.grp_id=timesheet_user_psdb.grp_id and month =3
AND year = 2017 and month!='' and timesheet_user_psdb.client_id=326
GROUP BY timesheet_user_psdb.task_id,timesheet_user_psdb.uid
ORDER BY timesheet_client.client_type desc,timesheet_client.organisation,timesheet_user_psdb.proj_id,
timesheet_user_psdb.task_id,timesheet_user.uid,timesheet_user_psdb.task_id;
I have already used an index on all the primary keys.
EXPLAIN Output:
Help for this would be highly appreciable.
Give this a try:
SELECT
TC.ORGANISATION AS CLIENT_NAME,
TP.TITLE AS PROJECT_NAME,
TT.NAME AS TASK,
CONCAT(TU.FIRST_NAME, ' ', TU.LAST_NAME) AS RESOURCE_NAME,
TU.BILL_FACTOR,
TC.CLIENT_TYPE, SUM(SPENT) AS SPENT, -- You should specify which table this comes from
SUM(DELIVERED_HOURS) AS DELIVERED_HOURS, -- You should specify which table this comes from
SUM(BILLABLE_HOURS) AS BILLABLE_HOURS, -- You should specify which table this comes from
COMMENTS, -- You should specify which table this comes from
COLOR, -- You should specify which table this comes from
LOCK_COLOR, -- You should specify which table this comes from
UPDATED_BY, -- You should specify which table this comes from
UPDATED_BY_DATE, -- You should specify which table this comes from
TUP.GRP_ID,
TUP.CLIENT_ID,
TUP.PROJ_ID,
TUP.TASK_ID,
TUP.UID,
TUG.GRP_NAME
FROM
TIMESHEET_USER AS TU
LEFT OUTER JOIN
TIMESHEET_USER_PSDB AS TUP
ON TU.USERNAME = TUP.UID
LEFT OUTER JOIN
TIMESHEET_USER_GRP AS TUG
ON TUP.GRP_ID = TUG.GRP_ID
LEFT OUTER JOIN
TIMESHEET_CLIENT AS TC
ON TUP.CLIENT_ID = TC.CLIENT_ID
LEFT OUTER JOIN
TIMESHEET_PROJECT AS TP
ON TUP.PROJ_ID = TP.PROJ_ID
LEFT OUTER JOIN
TIMESHEET_TASK TT
ON TUP.TASK_ID = TT.TASK_ID
WHERE
MONTH = 3 AND -- You should specify which table this comes from
YEAR = 2017 AND -- You should specify which table this comes from
MONTH != '' AND -- You should specify which table this comes from
TUP.CLIENT_ID = 326
GROUP BY
TUP.TASK_ID,
TUP.UID
ORDER BY
TC.CLIENT_TYPE DESC,
TC.ORGANISATION,
TUP.PROJ_ID,
TUP.TASK_ID,
TU.UID,
TUP.TASK_ID;
Doing the EXPLAIN should shed some light on it.
You might see a performance gain by doing the table joins explicitly in the FROM clause instead of inside the WHERE (https://dev.mysql.com/doc/refman/5.7/en/join.html). By doing explicit joins (e.g. LEFT OUTER, etc...) you will not only improve the readability of the query, but may be able to use less expensive joins where needed. This also affects how the query is executed as each clause is executed in a specific order (MySQL query / clause execution order).
I have four tables, user, user_billingprofile, user_shippingprofile, and user_address.
user: userId, dateCreated
user_billingprofile: userId, address
user_shippingprofile: userId, address
user_address: random address crap
Here is the query I have to get a users billing and shipping profiles in one shot.
SELECT * FROM `user`
JOIN `user_billingprofile` ON `user`.`userId` = `user_billingprofile`.`userId`
JOIN `user_address` ON `user_billingprofile`.`currentAddress` = `user_address`.`addressId`
JOIN `user_shippingprofile` ON `user_shippingprofile`.`currentAddress` = `user_address`.`addressId`
JOIN `user_address` ON `user_shippingprofile`.`currentAddress` = `user_address`.`addressId`
I get the error: #1066 - Not unique table/alias: 'user_address'.
Is there a way to take a simple join where a table is accessed twice in the same query, and separate the two results? Preferably with some kind of table prefix...
I'm a bit lost here. I know I could do this in two sepparate queries quite easily, but i'd like to learn how to do stuff like this in one shot.
Any help/suggestions/direction is greatly appreciated, thank you!.
Can you post the structure of your tables? Based on your query I'd say you need to consider changing it up a bit.
That said you can fix your current query by adding a table alias like so:
SELECT * FROM `user`
JOIN `user_billingprofile` ON `user`.`userId` = `user_billingprofile`.`userId`
JOIN `user_address` AS user_billing_address ON `user_billingprofile`.`currentAddress` = `user_address`.`addressId`
JOIN `user_shippingprofile` ON `user_shippingprofile`.`currentAddress` = `user_address`.`addressId`
JOIN `user_address` AS user_shipping_address ON `user_shippingprofile`.`currentAddress` = `user_address`.`addressId`
Note the AS clause I added. You'll probably need to alias the columns too (instead of SELECT * you likely will need SELECT user_shipping_address.address AS user_shipping_address_value, user_billing_address.address AS user_billing_address_value ... )
Hope that helps!
I'm in over my head with a big mysql query (mysql 5.0), and i'm hoping somebody here can help.
Earlier I asked how to get distinct values from a joined query
mysql count only for distinct values in joined query
The response I got worked (using a subquery with join as)
select *
from media m
inner join
( select uid
from users_tbl
limit 0,30) map
on map.uid = m.uid
inner join users_tbl u
on u.uid = m.uid
unfortunately, my query has grown more unruly, and though I have it running, joining into a derived table is taking too long because there is no indexes available to the derived query.
my query now looks like this
SELECT mdate.bid, mdate.fid, mdate.date, mdate.time, mdate.title, mdate.name,
mdate.address, mdate.rank, mdate.city, mdate.state, mdate.lat, mdate.`long`,
ext.link,
ext.source, ext.pre, meta, mdate.img
FROM ext
RIGHT OUTER JOIN (
SELECT media.bid,
media.date, media.time, media.title, users.name, users.img, users.rank, media.address,
media.city, media.state, media.lat, media.`long`,
GROUP_CONCAT(tags.tagname SEPARATOR ' | ') AS meta
FROM media
JOIN users ON media.bid = users.bid
LEFT JOIN tags ON users.bid=tags.bid
WHERE `long` BETWEEN -122.52224684058 AND -121.79760915942
AND lat BETWEEN 37.07500915942 AND 37.79964684058
AND date = '2009-02-23'
GROUP BY media.bid, media.date
ORDER BY media.date, users.rank DESC
LIMIT 0, 30
) mdate ON (mdate.bid = ext.bid AND mdate.date = ext.date)
phew!
SO, as you can see, if I understand my problem correctly, i have two derivative tables without indexes (and i don't deny that I may have screwed up the Join statements somehow, but I kept messing with different types, is this ended up giving me the result I wanted).
What's the best way to create a query similar to this which will allow me to take advantage of the indexes?
Dare I say, I actually have one more table to add into the mix at a later date.
Currently, my query is taking .8 seconds to complete, but I'm sure if I could take advantage of the indexes, this could be significantly faster.
First, check for indices on ext(bid, date), users(bid) and tags(bid), you should really have them.
It seems, though, that it's LONG and LAT that cause you most problems. You should try keeping your LONG and LAT as a (coordinate POINT), create a SPATIAL INDEX on this column and query like that:
WHERE MBRContains(#MySquare, coordinate)
If you can't change your schema for some reason, you can try creating additional indices that include date as a first field:
CREATE INDEX ix_date_long ON media (date, `long`)
CREATE INDEX ix_date_lat ON media (date, lat)
These indices will be more efficient for you query, as you use exact search on date combined with a ranged search on axes.
Starting fresh:
Question - why are you grouping by both media.bid and media.date? Can a bid have records for more than one date?
Here's a simpler version to try:
SELECT
mdate.bid,
mdate.fid,
mdate.date,
mdate.time,
mdate.title,
mdate.name,
mdate.address,
mdate.rank,
mdate.city,
mdate.state,
mdate.lat,
mdate.`long`,
ext.link,
ext.source,
ext.pre,
meta,
mdate.img,
( SELECT GROUP_CONCAT(tags.tagname SEPARATOR ' | ')
FROM tags
WHERE ext.bid = tags.bid
ORDER BY tags.bid GROUP BY tags.bid
) AS meta
FROM
ext
LEFT JOIN
media ON ext.bid = media.bid AND ext.date = media.date
JOIN
users ON ext.bid = users.bid
WHERE
`long` BETWEEN -122.52224684058 AND -121.79760915942
AND lat BETWEEN 37.07500915942 AND 37.79964684058
AND ext.date = '2009-02-23'
AND users.userid IN
(
SELECT userid FROM users ORDER BY rank DESC LIMIT 30
)
ORDER BY
media.date,
users.rank DESC
LIMIT 0, 30
You might want to compare your perforamnces against using a temp table for each selection, and joining those tables together.
create table #whatever
create table #whatever2
insert into #whatever select...
insert into #whatever2 select...
select from #whatever join #whatever 2
....
drop table #whatever
drop table #whatever2
If your system has enough memory to hold full tables this might work out much faster. It depends on how big your database is.