I've recently migrated a shop from shared server to VPS. Copied all files to the new server and imported the database. Completed the initial set ups. Now the homepage of the site is loading, but all other pages showing database exception. It is showing a long query. Can someone suggest me something. My mysql connect timeout is 60 secs. The query is so long that posting here freezes the browser. I'm posting an exrept of that sql query. Can someone suggest me something over this ?
SELECT product_shop.id_product, MAX(product_attribute_shop.id_product_attribute) id_product_attribute
FROM `awfps_product` p
INNER JOIN awfps_product_shop product_shop
ON (product_shop.id_product = p.id_product AND product_shop.id_shop = 1)
LEFT JOIN `awfps_product_attribute` pa ON (product_shop.id_product = pa.id_product)
LEFT JOIN awfps_product_attribute_shop product_attribute_shop
ON (product_attribute_shop.id_product_attribute = pa.id_product_attribute AND product_attribute_shop.id_shop = 1 AND product_attribute_shop.default_on = 1)
WHERE product_shop.`active` = 1
AND (( product_shop.`id_product` = 38) OR( product_shop.`id_product` = 40) OR( product_shop.`id_product` = 41)................all products
AND p.`id_product` IN (
SELECT cp.`id_product`
FROM `awfps_category_group` cg
LEFT JOIN `awfps_category_product` cp ON (cp.`id_category` = cg.`id_category`)
WHERE cg.`id_group` IN (3)
)
AND product_shop.`visibility` IN ("both", "catalog")
GROUP BY product_shop.id_product
ORDER BY RAND() LIMIT 1
I got the solution. I searched for the bug in debugging tool and found some clues. The error was for special blocks module. When you have a lot of products in the special category or in other words if you have applied catalog price discount to a lot of products then the special block module retrieves all products with the same category and chooses a particular to show in the special block. Disabled it and it's working. This part of the code may need to be modified. I've tried to create an issue but didn't succeeded when writing this post. May create that later.
Related
We are in the process of migrating from MySQL 5 to 8 and I found a query that will work in 5 but not 8.
Senario: I'm trying to find records that have been flagged as not having breast screening related data, but actually do have this data in the system. My main record is a Cycle record (t_cycle table). Each of the other related records have a link to the cycle table. In this simplified version of the query I'm looking for cycles that do not have a mammogram record but do have either an office visit or an MRI. But I'm only interested if the office visit had a procedure done with a specific result. I'm getting the list of acceptable results by querying another result table. In the actual query I have multiple other tables joined in but this demonstrates the problem.
Here is the query that works in 5 but not version 8:
SELECT
cyc.f_enroll_id,
cyc.f_cycle_number,
ov.f_cbe_result,
mri.f_uid as 'MRI'
FROM
t_cycle cyc LEFT JOIN
t_mam_rpt mam ON cyc.f_uid = mam.f_cycle_uid LEFT JOIN
t_office_visit ov ON cyc.f_uid = ov.f_cycle_uid AND ov.f_cbe_result IN (SELECT f_code FROM t_lk_cbe_result WHERE f_mde_code IN (1,2)) LEFT JOIN
t_mri mri ON cyc.f_uid = mri.f_cycle_uid
WHERE
cyc.f_mam_indication = 5 AND
mam.f_uid IS NULL AND
(ov.f_uid IS NOT NULL OR
mri.f_uid IS NOT NULL);
In version 5 I get about 20 records back. In version 8 I get over 7000 and most of those have office visits where the CBE result is not in the list of acceptable results.
There are 2 variations that appear to work in 8:
/* appears to work (removed the OR mri.f_uid IS NOT NULL) */
SELECT
cyc.f_enroll_id,
cyc.f_cycle_number,
ov.f_cbe_result,
mri.f_uid as 'MRI'
FROM
t_cycle cyc LEFT JOIN
t_mam_rpt mam ON cyc.f_uid = mam.f_cycle_uid LEFT JOIN
t_office_visit ov ON cyc.f_uid = ov.f_cycle_uid AND ov.f_cbe_result IN (SELECT f_code FROM t_lk_cbe_result WHERE f_mde_code IN (1,2)) LEFT JOIN
t_mri mri ON cyc.f_uid = mri.f_cycle_uid
WHERE
cyc.f_mam_indication = 5 AND
mam.f_uid IS NULL AND
ov.f_uid IS NOT NULL ;
In the above version I modify the WHERE clause to not have the 'OR mri.f_uid IS NOT NULL)' but in the actual use, this is not an option.
This also seems to work:
/* appears to work - replaced the SELECT in the JOIN with hard coded values that are returned in the select) */
SELECT
cyc.f_enroll_id,
cyc.f_cycle_number,
ov.f_cbe_result,
mri.f_uid as 'MRI'
FROM
t_cycle cyc LEFT JOIN
t_mam_rpt mam ON cyc.f_uid = mam.f_cycle_uid LEFT JOIN
t_office_visit ov ON cyc.f_uid = ov.f_cycle_uid AND ov.f_cbe_result IN (2,30,31,33,34,35,38,39,40,42,43,44,45,46,47) LEFT JOIN
t_mri mri ON cyc.f_uid = mri.f_cycle_uid
WHERE
cyc.f_mam_indication = 5 AND
mam.f_uid IS NULL AND
(ov.f_uid IS NOT NULL OR
mri.f_uid IS NOT NULL);
In the above case I replaced by sub-SELECT in the office visit join with a list of acceptable results. In reality, I cannot do this because that list is going to vary.
I guess my question boils down to why are office visit records being included in the result set even when they don't have the correct result code?
I don't post often here, so hopefully I've provided enough information.
TIA for any help!
-Carolyn
Additional Information: 8/4/21
I tried to get a small subset of data to recreate the problem but it only seems to show up with a large set of data in the office visit table (the one that has the join that includes a subquery).
If I remove the subquery from the join and instead left join my list of codes and then adjust the WHERE clause to do the check there, it works correctly. This is what the modified code looks like:
SELECT
cyc.f_enroll_id,
cyc.f_cycle_number,
ov.f_cbe_result,
mri.f_uid as 'MRI'
FROM
t_cycle cyc LEFT JOIN
t_office_visit ov ON cyc.f_uid = ov.f_cycle_uid LEFT JOIN
t_lk_cbe_result lk ON ov.f_cbe_result = lk.f_code LEFT JOIN
t_mam_rpt mam ON cyc.f_uid = mam.f_cycle_uid LEFT JOIN
t_mri mri ON cyc.f_uid = mri.f_cycle_uid
WHERE
cyc.f_mam_indication = 5 AND
mam.f_uid IS NULL AND
((ov.f_uid IS NOT NULL AND lk.f_mde_code IN (1,2)) OR
mri.f_uid IS NOT NULL)
We are using version 8.0.18 so it could very likely be a bug that has already been fixed as mentioned in the comments. We are going to upgrade again and see if that takes care of it. Thanks to all who looked at this and feel free to continue to reply if you know this had been a known issue. If there is a place to find known issues with mySQL I'd appreciate that link as well.
My MySQL query code likes as shown below, and there are about several thousands of records in the table, by now below SQL executes about 5 minutes and more. I am looking for ways to optimize it so that it takes less time to execute. Thank you!
SELECT `m`.`id`,
`m`.`id`,
`tr`.`name`,
`m`.`m_date`,
`t1`.`t_name` AS home,
`t2`.`t_name` AS away,
`m`.`score1`,
`m`.`score2`,
`cw1`.`tid` AS tid1,
`cw2`.`tid` AS tid2,
`o1`.`odds` AS odds1,
`o2`.`odds` AS odds2,
`m`.`m_time`
FROM `jos_bl_match` AS `m`
LEFT JOIN `jos_bl_matchday` AS `md` ON (`md`.`id` = `m`.`m_id`)
LEFT JOIN `jos_bl_seasons` AS `s` ON (`s`.`s_id` = `md`.`s_id`)
LEFT JOIN `jos_bl_tournament` AS `tr` ON (`tr`.`id` = `s`.`t_id`)
LEFT JOIN `jos_bl_teams` AS `t1` ON (`m`.`team1_id` = `t1`.`id`)
LEFT JOIN `jos_bl_teams` AS `t2` ON (`m`.`team2_id` = `t2`.`id`)
LEFT JOIN `jos_vuvuzelaodds_odds` AS `o1` ON (`o1`.`m_id` = `m`.`id`)
AND `o1`.`market_id` = 1
AND `o1`.`bookmaker_id` = 1
LEFT JOIN `jos_vuvuzelaodds_odds` AS `o2` ON (`o2`.`m_id` = `m`.`id`)
AND `o2`.`market_id` = 1
AND `o2`.`bookmaker_id` = 2
LEFT JOIN `jos_cwtags_tags` AS `cw1` ON (`cw1`.`item_id` = `o1`.`m_id`)
LEFT JOIN `jos_cwtags_tags` AS `cw2` ON (`cw2`.`item_id` = `o2`.`m_id`)
WHERE `m`.`published` = 1
AND `s`.`published` = '1'
AND `tr`.`published` = '1'
AND `s`.`s_id` = 869
AND `m`.`m_played` = '1'
AND `m`.`m_date` > 2013-01-01
AND `o1`.`odds` != ''
AND `o2`.`odds` != ''
AND `cw1`.`cat_id` = 19
AND `cw2`.`cat_id` = 21
ORDER BY `m`.`m_date`,
`md`.`id`,
`s`.`s_id`,
`tr`.`id` DESC LIMIT 0, 15
"Normalize, but don't over-normalize."
Some composite indexes you may be missing...
jos_bl_match: INDEX(m_played, published, m_date)
The columns need to be in that order. That will more quickly start the filtering.
The following should speed up their JOINs:
jos_vuvuzelaodds_odds: INDEX(market_id, bookmaker_id, m_id)
jos_cwtags_tags: INDEX(cat_id, item_id)
It seems like those last two indexes could (should) be the PRIMARY KEY. Are they?
Some (perhaps all) of the LEFT JOINs may as well be INNER JOINs; did you consider that?
Please provide EXPLAIN SELECT.
Without having access to the database it is a little hard to tell. This seems to be a lot of data to only pull 15 records. Are you sure you need to pull the data this way?
Probably the best route:
Optimize your database as below.
Make a flat database view of all the games that has the fields that you need. A static table will be much faster but you would need to set up updates with triggers which is beyond the scope of this answer but the process is similar
Write queries against the
view.
You are not selecting any fields from the seasons table in your select. I used the field from the jos_bl_matchday table.
This should get you started. You can also use conditionals in your select statements
(IF value = 1, table.field, null) as yadda
instead of joining one table over and over again but you would have to experiment.
CREATE VIEW allTheGames AS SELECT
`m`.`id` as id,
`md`.`s_id` as seasonId,
`tr`.`name` as name,
`m`.`m_date` as m_date,
`t1`.`t_name` AS home,
`t2`.`t_name` AS away,
`m`.`score1` as score1,
`m`.`score2` as score2,
`cw1`.`tid` AS tid1,
`cw2`.`tid` AS tid2,
`o1`.`odds` AS odds1,
`o2`.`odds` AS odds2,
`m`.`m_time` as m_time
FROM `jos_bl_match` AS `m`
LEFT JOIN `jos_bl_matchday` AS `md` ON (`md`.`id` = `m`.`m_id`)
LEFT JOIN `jos_bl_tournament` AS `tr` ON (`tr`.`id` = `s`.`t_id`)
LEFT JOIN `jos_bl_teams` AS `t1` ON (`m`.`team1_id` = `t1`.`id`)
LEFT JOIN `jos_bl_teams` AS `t2` ON (`m`.`team2_id` = `t2`.`id`)
LEFT JOIN `jos_vuvuzelaodds_odds` AS `o1` ON (`o1`.`m_id` = `m`.`id`) AND `o1`.`market_id` = 1 AND `o1`.`bookmaker_id` = 1
LEFT JOIN `jos_vuvuzelaodds_odds` AS `o2` ON (`o2`.`m_id` = `m`.`id`) AND `o2`.`market_id` = 1 AND `o2`.`bookmaker_id` = 2
LEFT JOIN `jos_cwtags_tags` AS `cw1` ON (`cw1`.`item_id` = `o1`.`m_id`)
LEFT JOIN `jos_cwtags_tags` AS `cw2` ON (`cw2`.`item_id` = `o2`.`m_id`)
WHERE `m`.`published` = 1 AND `s`.`published` = '1' AND `tr`.`published` = '1'
AND `m`.`m_played` = '1'
AND `o1`.`odds1` != '' AND `o2`.`odds2` != ''
Then query it with:
select * from allTheGames
WHERE season_id = 869 AND m_date > 2013-01-01 AND tid1 = 19 AND tid2 = 21
Steps to optimize:
Figure out exactly which data you want out of this query and why:
Is this a custom report? A web page? do you need to have all of this data at once or would it make more sense to have the user drill down?
How often is the query run? Once a minute? Once a day?
How many records are in each table? Your view should reflect this "game object"
Your database:
Run "explain" against this query http://dev.mysql.com/doc/refman/5.7/en/explain.html.
It will show all of the work the database is doing, the query execution plan, and how many records it is looking in to do it. This usually happens quickly: it does not actually execute the query.
See if you can put the database on SSD drives or even RAM
Your table structure:
Make sure that you have indexes on all of the fields that you are searching on. There are many ways to optimize this.
Use "explain" to be sure MySQL is able to use indexes.
If there are really only 2 markets in the jos_vuvuzelaodds_odds table consider making 2 fields.
Good luck!
I am stuck with this query below. It takes forever to complete the query.
I have been using describe to help me find best way to pull this data. But this is what I can come up with so far. But this take forever to pull just 1000 records . I would like to get some recommendations / suggestions on how to approve this issue ? I'm stuck.
I have read about "EXISTS" that I can use to improve subquery , but I am not quite understand how it would work in this situation.
I have already used primary keys as much as I could ( id , contact_id , flag_if ) => those ending with _id are all primary keys.
Thanks.
SELECT ac.id,
acc.category,
ac.value,
fg.*
FROM contact AS ac
JOIN application AS app ON app.id = ac.id
JOIN category AS acc USING (contact_id)
LEFT JOIN (SELECT af.name_short
,att.value
,att.flag_id
FROM attribute AS att
LEFT JOIN attribute_flag AS af ON af.flag_id = att.flag_id
LEFT JOIN application app on
( (att.level = 'customer' AND att.related_id = app.customer_id)
OR (att.level = 'application' and att.related_id = app.id)
)
WHERE app.id IN (001,002,003)
) fg on ac.value = fg.value
WHERE ac.id IN (001,002,003)
== UPDATE ==
here is the result from explain query'
I am running a SQL query to obtain results with the column applicationstatus that aren't S to tell which position is still open. There are 3 application status U=Unsuccessful, O = ongoing and S = successful. This works fine. Below is the code I am running.
SELECT DISTINCT position.position_ID, Title, EmployerName, Industry
FROM position
JOIN application ON position.position_id = application.position_id
JOIN employer ON position.employer_id = employer.employer_id
WHERE applicationstatus != 'S'
The problem I am facing is with the above query code, let's assume position_ID 3212 has received 3 applications, with one successful application; I get 2 results for the mentioned position(Excluding the successful one) like this.
Is there a way to filter it so that if a position already has a successful application, then the rows with the same position ID will be ignored?
Add a NOT EXISTS condition to exclude positions with successful applications:
SELECT DISTINCT
p.position_ID,
Title,
EmployerName,
Industry
FROM position p
JOIN application a
ON p.position_id = a.position_id
JOIN employer e
ON p.employer_id = e.employer_id
WHERE
applicationstatus != 'S'
AND NOT EXISTS(
SELECT 1
FROM application
WHERE
position_id = a.position_id
AND applicationstatus = 'S'
)
Note that I've rewritten your query to use meaningful aliases. You should also do this to improve readability and maintainability of your code.
I'm trying to get all articles that are the current/latest articles in Mediawiki 1.16.0. I need to do this in phpMyadmin and make a dump from those results.
my SQL:
SELECT
page.page_title, page.page_latest
, revision.rev_id, revision.rev_text_id
, text.old_id, text.old_text
FROM page, revision, text
WHERE rev_id = page_latest AND rev_text_id = old_id
I get the image names also but not a problem. I feel that this SQL above is not getting the latest version of the articles.
If there is a way to not show image names and redirects in the results it would also help.
First of all please don't use that ugly implicit join syntax. It's confusing and error-prone.
Change it to this:
SELECT
page.page_title, page.page_latest
, revision.rev_id, revision.rev_text_id
, text.old_id, text.old_text
FROM page
INNER JOIN revision ON (rev_id = page_latest)
INNER JOIN text ON (rev_text_id = old_id)
Now you can see why: it's getting all pages. There is no where clause, there are just join clauses.
This is the DB layout: http://upload.wikimedia.org/wikipedia/commons/b/b7/MediaWiki_database_schema_1-17_%28r82044%29.png
And here are the description of the fields in the various tables:
http://www.mediawiki.org/wiki/Manual:Database_layout
Revised query
SELECT
p.page_title, p.page_latest
, MAX(revision.rev_id) as rev_id, revision.rev_text_id
, text.old_id, text.old_text
FROM page p
INNER JOIN revision r ON (r.rev_id = p.page_latest)
INNER JOIN `text` t ON (r.rev_text_id = t.old_id)
WHERE p.page_is_redirect = 0 AND p.page_namespace <> 6 /*NS_IMAGE = 6*/
GROUP BY p.page_latest
ORDER BY p.page_title
This filters out the redirects and excludes the pages where namespace = ns_image.
I'm not 100% sure though 'cause I don't have MediaWiki to test.