my current query is look like this:
SELECT distinct pg.id,pg.email,pg.theme,op.PaymentMode,od.DeliveryMode,uv1.value AS FirstName, uv2.value AS LastName, uv3.value AS Mobile, uv4.value AS City, uv5.value AS State, uv6.value AS Address
FROM userdb.user u,userdb.paymentgatewaytracker pg,
oms.deliverymode od,oms.paymentmode op,
userdb.uservar uv1
JOIN userdb.uservar uv2 ON uv1.user_id = uv2.user_id
AND uv2.name = 'billingLastName'
LEFT JOIN userdb.uservar uv3 ON uv1.user_id = uv3.user_id
AND uv3.name = 'billingMobile'
JOIN userdb.uservar uv4 ON uv1.user_id = uv4.user_id
AND uv4.name = 'billingCity'
JOIN userdb.uservar uv5 ON uv1.user_id = uv5.user_id
AND uv5.name = 'billingState'
JOIN userdb.uservar uv6 ON uv1.user_id = uv6.user_id
AND uv6.name = 'billingAddress'
where uv1.name = 'billingFirstName'
AND u.id = uv1.user_id
AND u.email=pg.email
and op.PaymentModeId=pg.paymethod
and od.DeliveryModeId=pg.deliveryOption
ORDER BY pg.id
this is giving me the correct result while i am checking in my localhost. but i want to display this in my browser. while i am trying to do this it is showing Fatal error: Maximum execution time of 30 seconds exceeded. but for small query i am getting result in my browser. can any one please optimize my query which i have given above.
You join on user_id and name again and again. You might be missing an appropriate index
CREATE INDEX uservar_user_id_name ON uservar (user_id, name);
But you would get much better performance if you create a table with the columns user_id, billingLastName, billingMobile, billingCity, billingState, billingAddress and billingFirstName instead of having them in separate rows in the uservar table.
You could put an index on userdb.uservar on the id and name columns since most of the joins are on these
Indexes might be of great help here if not already used.
You need an index on "uservar" (user_id, name)
You appear to be doing a potentially large crossjoin which could slow things down quite a bit.
"userdb.user u,userdb.paymentgatewaytracker pg,
oms.deliverymode od,oms.paymentmode op"
I suggest you post the "Explain Select ..." for the join.
Related
My mysql query is too slow and i don't know how to optimize it. My webapp cant load this query because take too much time to run and the webserver have a limit time to get the result.
SELECT rc.trial_id,
rc.created,
rc.date_registration,
rc.agemin_value,
rc.agemin_unit,
rc.agemax_value,
rc.agemax_unit,
rc.exclusion_criteria,
rc.study_design,
rc.expanded_access_program,
rc.number_of_arms,
rc.enrollment_start_actual,
rc.target_sample_size,
(select name from repository_institution where id = rc.primary_sponsor_id) as
primary_sponsor,
(select label from vocabulary_studytype where id = rc.study_type_id) as study_type,
(select label from vocabulary_interventionassigment where id =
rc.intervention_assignment_id) as intervention_assignment,
(select label from vocabulary_studypurpose where id = rc.purpose_id) as study_purpose,
(select label from vocabulary_studymasking where id = rc.masking_id) as study_mask,
(select label from vocabulary_studyallocation where id = rc.allocation_id) as
study_allocation,
(select label from vocabulary_studyphase where id = rc.phase_id) as phase,
(select label from vocabulary_recruitmentstatus where id = rc.recruitment_status_id) as
recruitment_status,
GROUP_CONCAT(vi.label)
FROM
repository_clinicaltrial rc
inner JOIN repository_clinicaltrial_i_code rcic ON rcic.clinicaltrial_id = rc.id JOIN
vocabulary_interventioncode vi ON vi.id = rcic.interventioncode_id
GROUP BY rc.id;
Using inner join instead join could be a solution?
Changing to JOINs vs continuous selects per every row will definitely improve. Also, since you are using MySQL, using the keyword "STRAIGHT_JOIN" tells MySQL to do the query in the order I provided. Since your "rc" table is the primary and all the others are lookups, this will make MySQL use it in that context rather than hoping some other lookup table be the basis of the rest of the joins.
SELECT STRAIGHT_JOIN
rc.trial_id,
rc.created,
rc.date_registration,
rc.agemin_value,
rc.agemin_unit,
rc.agemax_value,
rc.agemax_unit,
rc.exclusion_criteria,
rc.study_design,
rc.expanded_access_program,
rc.number_of_arms,
rc.enrollment_start_actual,
rc.target_sample_size,
ri.name primary_sponsor,
st.label study_type,
via.label intervention_assignment,
vsp.label study_purpose,
vsm.label study_mask,
vsa.label study_allocation,
vsph.label phase,
vrs.label recruitment_status,
GROUP_CONCAT(vi.label)
FROM
repository_clinicaltrial rc
JOIN repository_clinicaltrial_i_code rcic
ON rc.id = rcic.clinicaltrial_id
JOIN vocabulary_interventioncode vi
ON rcic.interventioncode_id = vi.id
JOIN repository_institution ri
on rc.primary_sponsor_id = ri.id
JOIN vocabulary_studytype st
on rc.study_type_id = st.id
JOIN vocabulary_interventionassigment via
on rc.intervention_assignment_id = via.id
JOIN vocabulary_studypurpose vsp
ON rc.purpose_id = vsp.id
JOIN vocabulary_studymasking vsm
ON rc.masking_id = vsm.id
JOIN vocabulary_studyallocation vsa
ON rc.allocation_id = vsa.id
JOIN vocabulary_studyphase vsph
ON rc.phase_id = vsph.id
JOIN vocabulary_recruitmentstatus vrs
ON rc.recruitment_status_id = vrs.id
GROUP BY
rc.id;
One final note. You are using a GROUP BY and applying to the GROUP_CONCAT() which is ok. However, proper group by says you need to group by all non-aggregate columns, which in this case is every other column in the list. You may know this, and the fact the lookups will be the same based on the "rc" associated columns, but its not good practice to do so.
Your joins and subqueries are probably not the problem. Assuming you have correct indexes on the tables, then these are fast. "Correct indexes" means that the id column is the primary key -- a very reasonable assumption.
My guess is that the GROUP BY is the performance issue. So, I would suggest structuring the query with no `GROUP BY:
select . . .
(select group_concat(vi.label)
from repository_clinicaltrial_i_code rcic
vocabulary_interventioncode vi
on vi.id = rcic.interventioncode_id
where rcic.clinicaltrial_id = rc.id
)
from repository_clinicaltrial rc ;
For this, you want indexes on:
repository_clinicaltrial_i_code(clinicaltrial_id, interventioncode_id)
vocabulary_interventioncode(id, label)
My MySQL query code likes as shown below, and there are about several thousands of records in the table, by now below SQL executes about 5 minutes and more. I am looking for ways to optimize it so that it takes less time to execute. Thank you!
SELECT `m`.`id`,
`m`.`id`,
`tr`.`name`,
`m`.`m_date`,
`t1`.`t_name` AS home,
`t2`.`t_name` AS away,
`m`.`score1`,
`m`.`score2`,
`cw1`.`tid` AS tid1,
`cw2`.`tid` AS tid2,
`o1`.`odds` AS odds1,
`o2`.`odds` AS odds2,
`m`.`m_time`
FROM `jos_bl_match` AS `m`
LEFT JOIN `jos_bl_matchday` AS `md` ON (`md`.`id` = `m`.`m_id`)
LEFT JOIN `jos_bl_seasons` AS `s` ON (`s`.`s_id` = `md`.`s_id`)
LEFT JOIN `jos_bl_tournament` AS `tr` ON (`tr`.`id` = `s`.`t_id`)
LEFT JOIN `jos_bl_teams` AS `t1` ON (`m`.`team1_id` = `t1`.`id`)
LEFT JOIN `jos_bl_teams` AS `t2` ON (`m`.`team2_id` = `t2`.`id`)
LEFT JOIN `jos_vuvuzelaodds_odds` AS `o1` ON (`o1`.`m_id` = `m`.`id`)
AND `o1`.`market_id` = 1
AND `o1`.`bookmaker_id` = 1
LEFT JOIN `jos_vuvuzelaodds_odds` AS `o2` ON (`o2`.`m_id` = `m`.`id`)
AND `o2`.`market_id` = 1
AND `o2`.`bookmaker_id` = 2
LEFT JOIN `jos_cwtags_tags` AS `cw1` ON (`cw1`.`item_id` = `o1`.`m_id`)
LEFT JOIN `jos_cwtags_tags` AS `cw2` ON (`cw2`.`item_id` = `o2`.`m_id`)
WHERE `m`.`published` = 1
AND `s`.`published` = '1'
AND `tr`.`published` = '1'
AND `s`.`s_id` = 869
AND `m`.`m_played` = '1'
AND `m`.`m_date` > 2013-01-01
AND `o1`.`odds` != ''
AND `o2`.`odds` != ''
AND `cw1`.`cat_id` = 19
AND `cw2`.`cat_id` = 21
ORDER BY `m`.`m_date`,
`md`.`id`,
`s`.`s_id`,
`tr`.`id` DESC LIMIT 0, 15
"Normalize, but don't over-normalize."
Some composite indexes you may be missing...
jos_bl_match: INDEX(m_played, published, m_date)
The columns need to be in that order. That will more quickly start the filtering.
The following should speed up their JOINs:
jos_vuvuzelaodds_odds: INDEX(market_id, bookmaker_id, m_id)
jos_cwtags_tags: INDEX(cat_id, item_id)
It seems like those last two indexes could (should) be the PRIMARY KEY. Are they?
Some (perhaps all) of the LEFT JOINs may as well be INNER JOINs; did you consider that?
Please provide EXPLAIN SELECT.
Without having access to the database it is a little hard to tell. This seems to be a lot of data to only pull 15 records. Are you sure you need to pull the data this way?
Probably the best route:
Optimize your database as below.
Make a flat database view of all the games that has the fields that you need. A static table will be much faster but you would need to set up updates with triggers which is beyond the scope of this answer but the process is similar
Write queries against the
view.
You are not selecting any fields from the seasons table in your select. I used the field from the jos_bl_matchday table.
This should get you started. You can also use conditionals in your select statements
(IF value = 1, table.field, null) as yadda
instead of joining one table over and over again but you would have to experiment.
CREATE VIEW allTheGames AS SELECT
`m`.`id` as id,
`md`.`s_id` as seasonId,
`tr`.`name` as name,
`m`.`m_date` as m_date,
`t1`.`t_name` AS home,
`t2`.`t_name` AS away,
`m`.`score1` as score1,
`m`.`score2` as score2,
`cw1`.`tid` AS tid1,
`cw2`.`tid` AS tid2,
`o1`.`odds` AS odds1,
`o2`.`odds` AS odds2,
`m`.`m_time` as m_time
FROM `jos_bl_match` AS `m`
LEFT JOIN `jos_bl_matchday` AS `md` ON (`md`.`id` = `m`.`m_id`)
LEFT JOIN `jos_bl_tournament` AS `tr` ON (`tr`.`id` = `s`.`t_id`)
LEFT JOIN `jos_bl_teams` AS `t1` ON (`m`.`team1_id` = `t1`.`id`)
LEFT JOIN `jos_bl_teams` AS `t2` ON (`m`.`team2_id` = `t2`.`id`)
LEFT JOIN `jos_vuvuzelaodds_odds` AS `o1` ON (`o1`.`m_id` = `m`.`id`) AND `o1`.`market_id` = 1 AND `o1`.`bookmaker_id` = 1
LEFT JOIN `jos_vuvuzelaodds_odds` AS `o2` ON (`o2`.`m_id` = `m`.`id`) AND `o2`.`market_id` = 1 AND `o2`.`bookmaker_id` = 2
LEFT JOIN `jos_cwtags_tags` AS `cw1` ON (`cw1`.`item_id` = `o1`.`m_id`)
LEFT JOIN `jos_cwtags_tags` AS `cw2` ON (`cw2`.`item_id` = `o2`.`m_id`)
WHERE `m`.`published` = 1 AND `s`.`published` = '1' AND `tr`.`published` = '1'
AND `m`.`m_played` = '1'
AND `o1`.`odds1` != '' AND `o2`.`odds2` != ''
Then query it with:
select * from allTheGames
WHERE season_id = 869 AND m_date > 2013-01-01 AND tid1 = 19 AND tid2 = 21
Steps to optimize:
Figure out exactly which data you want out of this query and why:
Is this a custom report? A web page? do you need to have all of this data at once or would it make more sense to have the user drill down?
How often is the query run? Once a minute? Once a day?
How many records are in each table? Your view should reflect this "game object"
Your database:
Run "explain" against this query http://dev.mysql.com/doc/refman/5.7/en/explain.html.
It will show all of the work the database is doing, the query execution plan, and how many records it is looking in to do it. This usually happens quickly: it does not actually execute the query.
See if you can put the database on SSD drives or even RAM
Your table structure:
Make sure that you have indexes on all of the fields that you are searching on. There are many ways to optimize this.
Use "explain" to be sure MySQL is able to use indexes.
If there are really only 2 markets in the jos_vuvuzelaodds_odds table consider making 2 fields.
Good luck!
i am writing a mysql query as below
SELECT `user_master`.`first_name`,
`city_name`,
`user_master`.`last_name`,
`user_master`.`user_master_id`,
`account_management_master`.`account_name`,
`donation_receipt_info`.`receipt_temple_id`,
date(dt) AS dt,
SUM(`donation_receipt_info`.`amount`) AS amount
FROM (`donation_receipt_info`)
JOIN `donation_receipt_master` ON donation_receipt_master`.`receipt_id`=`donation_receipt_info`.`receipt_id`
JOIN `account_management_master` ON `account_management_master`.`account_id`=`donation_receipt_info`.`account_id`
JOIN `user_master` ON `user_master`.`user_master_id`=`donation_receipt_master`.`user_master_id`
JOIN `user_address_info` ON `user_address_info`.`user_master_id`=`user_master`.`user_master_id`
JOIN `city_master` ON `city_master`.`city_id`=`user_address_info`.`city_id`
WHERE `donation_receipt_info`.`temple_id` = '1'
GROUP BY `donation_receipt_info`.`receipt_id`,
`donation_receipt_info`.`account_id`
the table donation_receipt_info and master have approx 42k results the query is taking way to much time of about 5 to 6 minutes to execute in mysql itself.
can someone please help me optimize the query, any help or suggestion would be very helpful
Thanks.
First, your query is impossible to read. You should format it and learn to use table aliases:
SELECT um.first_name, city_name, um.last_name, um.user_master_id, amm.account_name,
dri.receipt_temple_id, date(dt) AS dt, SUM(dri.amount) AS amount
FROM donation_receipt_info dri JOIN
donation_receipt_master drm
ON drm.receipt_id = dri.receipt_id JOIN
account_management_master amm
ON amm.account_id = dri.account_id JOIN
user_master um
ON um.user_master_id = drm.user_master_id JOIN
user_address_info uai
ON uai.user_master_id = um.user_master_id JOIN
city_master cm
ON cm.city_id = uai.city_id
WHERE dri.temple_id = '1'
GROUP BY dri.receipt_id, dri.account_id;
Next. Do all the tables have the obvious indexes? That is, each table appears to have an id and these should be declared as keys (primary keys preferably). For instance, city_master(city_id).
Next, there should be an index on donation_receipt_info(temple_id, receipt_id, account_id). This should help with the where. Note: if temple_id is really an integer, the where clause should be expressed as WHERE dri.temple_id = 1 -- no quotes. You don't want MySQL to get confused and decide not to use the index.
These changes will probably help. 5-6 minutes seems like a long time for such a query.
So, this query is currently used in a webshop to retrieve technical data about articles.
It has served its purpose fine except the amount of products shown have increased lately resulting in unacceptable long loading times for some categories.
For one of the worst pages this (and some other queries) get requested about 80 times.
I only recently learned that MySQL does not optimize sub-queries that don't have a depending parameter to only run once.
So if someone could help me with one of the queries and explain how you can replace the in's and exists's to joins, i will probably be able to change the other ones myself.
select distinct criteria.cri_id, des_texts.tex_text, article_criteria.acr_value, article_criteria.acr_kv_des_id
from article_criteria, designations, des_texts, criteria, articles
where article_criteria.acr_cri_id = criteria.cri_id
and article_criteria.acr_art_id = articles.art_id
and articles.art_deliverystatus = 1
and criteria.cri_des_id = designations.des_id
and designations.des_lng_id = 9
and designations.des_tex_id = des_texts.tex_id
and criteria.cri_id = 328
and article_criteria.acr_art_id IN (Select distinct link_art.la_art_id
from link_art, link_la_typ
where link_art.la_id = link_la_typ.lat_la_id
and link_la_typ.lat_typ_id = 17484
and link_art.la_ga_id IN (Select distinct link_ga_str.lgs_ga_id
from link_ga_str, search_tree
where link_ga_str.lgs_str_id = search_tree.str_id
and search_tree.str_type = 1
and search_tree.str_id = 10132
and EXISTS (Select *
from link_la_typ
where link_la_typ.lat_typ_id = 17484
and link_ga_str.lgs_ga_id = link_la_typ.lat_ga_id)))
order by article_criteria.acr_value
I think this one is the main badguy with sub-sub-sub-queries
I just noticed i can remove the last exist and still get the same results but with no increase in speed, not part of the question though ;) i'll figure out myself whether i still need that part.
Any help or pointers are appreciated, if i left out some useful information tell me as well.
I think this is equivalent:
SELECT DISTINCT c.cri_id, dt.tex_text, ac.acr_value, ac.acr_kv_des_id
FROM article_criteria AS ac
JOIN criteria AS c ON ac.acr_cri_id = c.cri_id
JOIN articles AS a ON ac.acr_art_id = a.art_id
JOIN designations AS d ON c.cri_des_id = d.des_id
JOIN des_texts AS dt ON dt.tex_id = d.des_tex_id
JOIN (SELECT distinct la.la_art_id
FROM link_art AS la
JOIN link_la_typ AS llt ON la.la_id = llt.lat_la_id
JOIN (SELECT DISTINCT lgs.lgs_ga_id
FROM link_ga_str AS lgs
JOIN search_tree AS st ON lgs.lgs_str_id = st.str_id
JOIN link_la_typ AS llt ON lgs.lgs_ga_id = llt.lat_ga_id
WHERE st.str_type = 1
AND st.str_id = 10132
AND llt.lat_typ_id = 17484) AS lgs
ON la.la_ga_id = lgs.lgs_ga_id
WHERE llt.lat_typ_id = 17484) AS la
ON ac.acr_art_id = la.la_art_id
WHERE a.art_deliverystatus = 1
AND d.des_lng_id = 9
AND c.cri_id = 328
ORDER BY ac.acr_value
All the IN <subquery> clauses can be replaced with JOIN <subquery>, where you then JOIN on the column being tested equaling the column returned by the subquery. And the EXISTS test is converted to a join with the table, moving the comparison in the subquery's WHERE clause into the ON clause of the JOIN.
It's probably possible to flatten the whole thing, instead of joining with subqueries. But I suspect performance will be poor, because this won't reduce the temporary tables using DISTINCT. So you'll get combinatorial explosion in the resulting cross product, which will then have to be reduced at the end with the DISTINCT at the top.
I've converted all the implicit joins to ANSI JOIN clauses, to make the structure clearer, and added table aliases to make things more readable.
In general, you can convert a FROM tab1 WHERE ... val IN (SELECT blah) to a join like this.
FROM tab1
JOIN (
SELECT tab1_id
FROM tab2
JOIN tab3 ON whatever = whatever
WHERE whatever
) AS sub1 ON tab1.id = sub1.tab1_id
The JOIN (an inner join) will drop the rows that don't match the ON condition from your query.
If your tab1_id values can come up duplicate from your inner query, use SELECT DISTINCT. But don't use SELECT DISTINCT unless you need to; it is costly to evaluate.
I have four tables, user, user_billingprofile, user_shippingprofile, and user_address.
user: userId, dateCreated
user_billingprofile: userId, address
user_shippingprofile: userId, address
user_address: random address crap
Here is the query I have to get a users billing and shipping profiles in one shot.
SELECT * FROM `user`
JOIN `user_billingprofile` ON `user`.`userId` = `user_billingprofile`.`userId`
JOIN `user_address` ON `user_billingprofile`.`currentAddress` = `user_address`.`addressId`
JOIN `user_shippingprofile` ON `user_shippingprofile`.`currentAddress` = `user_address`.`addressId`
JOIN `user_address` ON `user_shippingprofile`.`currentAddress` = `user_address`.`addressId`
I get the error: #1066 - Not unique table/alias: 'user_address'.
Is there a way to take a simple join where a table is accessed twice in the same query, and separate the two results? Preferably with some kind of table prefix...
I'm a bit lost here. I know I could do this in two sepparate queries quite easily, but i'd like to learn how to do stuff like this in one shot.
Any help/suggestions/direction is greatly appreciated, thank you!.
Can you post the structure of your tables? Based on your query I'd say you need to consider changing it up a bit.
That said you can fix your current query by adding a table alias like so:
SELECT * FROM `user`
JOIN `user_billingprofile` ON `user`.`userId` = `user_billingprofile`.`userId`
JOIN `user_address` AS user_billing_address ON `user_billingprofile`.`currentAddress` = `user_address`.`addressId`
JOIN `user_shippingprofile` ON `user_shippingprofile`.`currentAddress` = `user_address`.`addressId`
JOIN `user_address` AS user_shipping_address ON `user_shippingprofile`.`currentAddress` = `user_address`.`addressId`
Note the AS clause I added. You'll probably need to alias the columns too (instead of SELECT * you likely will need SELECT user_shipping_address.address AS user_shipping_address_value, user_billing_address.address AS user_billing_address_value ... )
Hope that helps!