I have a query running fine on my current PC that is my developpement PC but not in production environment.
The two environement have the same material configuration.
The setting of the database are the same two.
On dev the query execute in less than 30 sec, but when I try it on production even after 2 hours I don't get result.
This is the only running query on the server at the moment of execution.
Do you have any idea of why this is happening ?
SELECT td.td_date,
td.td_number,
td.td_status,
td.td_open_datetime,
td.td_update_datetime,
Min(ack.ackup) ackUpdate
FROM t_ticketdossier_td td
INNER JOIN (SELECT a.td_number,
a.td_update_datetime ackUp
FROM t_ticketdossier_td a
WHERE a.td_status = 'Acknowledged') ack
ON td.td_number = ack.td_number
AND td.td_update_datetime <= ack.ackup
WHERE td.td_number IN (SELECT td_number
FROM t_ticketdossier_td base
WHERE Date(base.td_date) = #date
AND base.td_status = 'Acknowledged'
AND base.td_scope IS NULL
AND base.td_subcategory NOT LIKE '%RDV%'
AND base.td_product_type NOT LIKE '%RDV%'
GROUP BY td_number)
AND td.td_status = 'Affected'
GROUP BY td.td_date,
td.td_number,
td.td_status,
td.td_update_datetime
Explain from the dev server.
Explain from the prod server.
Sorry for the links, I don't have enougth reputation to post the image directly.
Cheers,
Maxime.
From what I see in the EXPLAIN statements, it seems that your production database misses quite a few indexes.
For example:
TD_STATUS_IDX,TD_NUMBER_IDX
Try adding these and run the EXPLAIN again.
Related
We have recently migrated a PHP web application to a different hosting. Along with that, the database was also successfully copied over to the new mysql server. The earlier version of database server was mysql5.6 and the new hosting has mariadb 10 installed. Below is one example query which is on old hosting is taking around 0.5s and on the new hosting with mariadb, it is taking around 40s which is very long. This long time is chocking up the hosting server.
SELECT
IB.IndicatorId,
IB.IndicatorTitle,
IB.IndicatorCode,
IC.IndicatorCategoryName,
AA.Answer,
GROUP_CONCAT(AI.`Answer`) AS CombinedAnswers
FROM answer AS A
LEFT JOIN answers_answer AS AA ON AA.AnswerId = A.AnswerId
LEFT JOIN activity_sections_indicators AS ASI ON ASI.SectionIndicatorId = AA.SectionIndicatorId
LEFT JOIN indicators_bank AS IB ON IB.IndicatorId = ASI.IndicatorId
LEFT JOIN indicator_categories AS IC ON IC.IndicatorCategoryId = IB.IndicatorCategoryId
LEFT JOIN answers_items AS AI ON AI.AnswerAnswerId = AA.AnswerAnswerId
WHERE A.ProgramActivityId IN (103,104,105)
AND A.Code = 'Newsas263'
GROUP BY IB.IndicatorId
Further more the number of records in tables is as follows
answer : 5355
answers_answer : 845209
activity_sections_indicators : 2866
indicators_bank : 1175
indicator_categories : 17
answers_items : 934347
My understanding is that there is something missing in mariadb server configuration though the query can be optimized as well. I have VPS and can change configuration for mariadb but I am not sure what to do from here.
Let me know if some more detail is needed. I really appreciate your help in this regard.
MySQL and MariaDB diverged somewhat significantly with 5.6. It would be hard to spot the particular thing that is different.
The EXPLAIN SELECT ... output might help.
These composite and/or covering indexes may help on both servers:
IB: INDEX(IndicatorId, IndicatorTitle, IndicatorCode, IndicatorCategoryId)
IC: INDEX(IndicatorCategoryId, IndicatorCategoryName)
AA: INDEX(AnswerId, Answer, SectionIndicatorId, AnswerAnswerId)
AI: INDEX(AnswerAnswerId, Answer)
A: INDEX(Code, ProgramActivityId, AnswerId)
ASI: INDEX(SectionIndicatorId, IndicatorId)
When adding a composite index, DROP index(es) with the same leading columns.
That is, when you have both INDEX(a) and INDEX(a,b), toss the former.
The index on "A" may be the most important.
Are any of the tables "many-to-many"?
Which version of MariaDB? There is already 10.0 through 10.7.
I was able to resolve the issue.
Thanks for the recommendation/suggestion by "Ergest Basha", "Markus Zeller" and "Rick James".
I ran the "Explain Select ..." on the query and the "Execution Plan" has some differences. I was able to mitigate those differences and the query on new server now works like normal.
Though I was not clearly able to find out the differences but I just re copied the whole database and the issue was resolved.
I have a query that works when I troubleshoot this within my code on my local pc, it works perfectly but the moment I put this up on the live server the query fails for some reason, I think, but I'm not sure, that this boils down to the date fields somehow but honestly I have no idea why it's failing on the server. I have tried the sql functions Date, date_to_string and my dates are in the format '2019-03-08' please assist, my query is as per below.
select posted_ads.id, posted_ads.title,
posted_ads.description, posted_ads.price,
posted_ads.datePosted,
posted_ads.adCategory,
posted_ads.adSuspended, posted_ads.suspensionEmailSent,
posted_ads.adSuspensionReason, posted_ads_images.id, posted_ads_images.imageName,
posted_ads_images.type, posted_ads_images.adID
from posted_ads inner join
posted_ads_images
on posted_ads.id = posted_ads_images.adID
where title like '%mechanic%' and
posted_ads.adExpired = 0 and
posted_ads.datePosted between '2018-12-09' and '2019-03-09'
group by posted_ads.id
limit 6 offset 6
I tried to run the following query in my Cloud VPS cPanel phpMyadmin
SELECT bankcode, bankname
FROM newbankdetails
WHERE
type='DB' AND
bankcode IN ( SELECT drawingbankname
FROM dd1
WHERE entrydate BETWEEN '01/04/2014' AND '30/04/2014'
AND paymentmode='DD'
AND state='TAMIL NADU' )
It takes very very very long time nearly 5 to 6 hours and says out of time or keeps running for days and don't show any error also don't show results.
Whereas it works perfectly in my local machine xampp
It happens only when working on large size tables like 12GB and around like that
How to speed it up and display the result as it displays instantly in localhost xampp
I think, replacing IN with a LEFT JOIN can make more sense, like this:
SELECT
bankcode, bankname
FROM
newbankdetails nd
LEFT JOIN
dd1 ON nd.bankcode = dd1.drawingbankname
WHERE
nd.type='DB'
AND
dd1.entrydate BETWEEN '01/04/2014' AND '30/04/2014'
AND
dd1.paymentmode = 'DD'
AND
dd1.state = 'TAMIL NADU'
I can't test it, But I think with dd1.paymentmode = 'DD' automatically removes rows of dd1.paymentmode IS NULL because of using LEFT JOIN.
First you should use EXPLAIN QUERY (https://dev.mysql.com/doc/refman/5.0/en/using-explain.html)
That will give you information about what is slow.
Your problem seems to be related to the size of the table. On your localhost you may have a small table, but in production it is very large. Also put an index on the first table on nd.bankcode.
You should try to remove the subselect, and use a JOIN instead.
Also, you should put indexes on the columns you use for search.
Put an index on drawingbankname, entrydate, paymentmode, state. Put also an index on nd.backcode .
Column drawingbankname is varchar, you should convert it to fixed char.
Remove the subselect and use JOIN.
Use EXPLAIN after the changes to see progress.
I'm running multiple queries on both MySQL and SQLServer (same queries on both servers, same db). Almost all of them run fine. I have a problem with this one:
SELECT
`Extent1`.`IdGosc`,
`Extent2`.`Imie`,
`Extent2`.`Nazwisko`
FROM `TGosc` AS `Extent1`
INNER JOIN `TOsoba` AS `Extent2` ON `Extent1`.`IdGosc` = `Extent2`.`IdOsoba`
WHERE EXISTS(
SELECT 1 AS `C1`
FROM (
SELECT `Extent3`.`IdRezerwacja`
FROM `TRezerwacja` AS `Extent3`
(here!) WHERE `Extent1`.`IdGosc` = `Extent3`.`IdGosc`) AS `Project1`
)
It runs on SQL Server just fine, returns correct results, but MySQL says:
Error Code: 1054. Unknown column 'Extent1.IdGosc' in 'where clause'.
Why so? :|
Are there any limitations about MySQL nested queries?
(Please don't offer queries that return the same and work, I can do that as well, but it's not my point)
I have seen this problem on MySQL.
SELECT `Extent1`.`IdGosc`, `Extent2`.`Imie`, `Extent2`.`Nazwisko`
FROM `TGosc` `Extent1` INNER JOIN
`TOsoba` `Extent2`
ON `Extent1`.`IdGosc` = `Extent2`.`IdOsoba`
WHERE EXISTS (SELECT `Extent3`.`IdRezerwacja`
FROM `TRezerwacja` AS `Extent3`
(here!) WHERE `Extent1`.`IdGosc` = `Extent3`.`IdGosc`
)
Fortunately, in this case, you can just eliminate the middle subquery.
I too have faced this sort of error in mysql.What I have done at tht tym is :
mysql remember only current table so try to do it may b it would work
replace
FROM `TRezerwacja` AS `Extent3
with
FROM `TRezerwacja` AS `Extent3`,`TGosc` AS `Extent1`
Ok. It turns out to be the problem with MySQL.
I'm using Entity Framework's query, that later turns into db specific SQL. I this case MySQL. So, the query in EF is:
var query3a = from TGosc gosc in context.TGosc
where gosc.TRezerwacja
.Any(x => x.TPlatnosc
.Any(y => y.Kwota > 100000))
select new { gosc.IdGosc, gosc.TOsoba.Imie, gosc.TOsoba.Nazwisko };
Now, the provider in my app is Connector NET 6.7.4. It includes MySQL.Data and MySQL.Data.Entities, both in version 6.7.4.
However, I also installed MySQL for Visual Studio 1.0.2 to be able to use more GUI than code in Visual Studio. But this thing comes with same dlls, just in different (older) versions 6.6.5. And these took precedence over the newer ones when application was running. (It's weird in the first place that in the same MySQL Installer there are two somehow conflicting versions of the same dlls.)
Anyway, I removed MySQL for Visual Studio 1.0.2, which left me with the newer dlls and see what happens to the very same LINQ to Entities query, when it's being translated to db sql:
--old 6.6.5
SELECT
Extent1.IdGosc,
Extent2.Imie,
Extent2.Nazwisko
FROM TGosc AS Extent1
INNER JOIN TOsoba AS Extent2 ON Extent1.IdGosc = Extent2.IdOsoba
WHERE EXISTS(
SELECT 1 AS C1
FROM (
SELECT Extent3.IdRezerwacja
FROM TRezerwacja AS Extent3
WHERE Extent1.IdGosc = Extent3.IdGosc) AS Project1
WHERE EXISTS(
SELECT 1 AS C1
FROM TPlatnosc AS Extent4
WHERE (Project1.IdRezerwacja = Extent4.IdRezerwacja)
AND (Extent4.Kwota > 100000)))
vs
-- new 6.7.4
SELECT
Extent1.IdGosc,
Extent2.Imie,
Extent2.Nazwisko
FROM TGosc AS Extent1
INNER JOIN TOsoba AS Extent2 ON Extent1.IdGosc = Extent2.IdOsoba
WHERE EXISTS(
SELECT 1 AS C1
FROM TRezerwacja AS Project1
WHERE EXISTS(
SELECT 1 AS C1
FROM TPlatnosc AS Extent4
WHERE (Project1.IdRezerwacja = Extent4.IdRezerwacja)
AND (Extent4.Kwota > 100000))
AND Extent1.IdGosc = Project1.IdGosc)
It's similar to what Gordon Linoff answered in this post. The middle subquery dissapears.
And of course the new query works fine!
Summing up, I guess the MySQL provider for .NET got better over these versions. I still have some queries that cause similar problems but now I think I know why that is - provider. I'm ok with that.
The annoying thing is that there are two different versions of dlls, one overriding another, in MySQL Installer. I'm using mysql-installer-community-5.6.13.0.
I found that code in the bottom executes more than 10 minutes on my server. The server itself is quite good and I cannot find any suitable explanation for that.
I am using Joomla 1.6.3 the data is migrated from Joomla 1.5.23 using jUpgrade and MySQL client version is 5.1.45.
SELECT a.*,COUNT(DISTINCT m1.id) AS count_published,COUNT(DISTINCT m2.id) AS count_unpublished,COUNT(DISTINCT m3.id) AS count_trashed
FROM `j16_menu_types` AS a
LEFT JOIN `j16_menu` AS m1 ON m1.menutype = a.menutype AND m1.published = 1
LEFT JOIN `j16_menu` AS m2 ON m2.menutype = a.menutype AND m2.published = 0
LEFT JOIN `j16_menu` AS m3 ON m3.menutype = a.menutype AND m3.published = -2
GROUP BY a.id
ORDER BY a.id asc;
I would be very pleased if someone could help me since I am in big trouble :)
P.s. I have downloaded db and checked it on my computer - still the same, execution time is terrible. Is there any way to solve this problem? Or maybe to remove this sql part without significant changes in administration of joomla?
Well it was done quite faster on my PC but still the result is far from one which would satisfy my.
EDIT
Well, I've found a reported bug of this problem. http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=24868
And the solution is:
CREATE INDEX idx_menu_published ON j16_menu (published);
However, I am not sure how this will affect administration. I there is anyone who could briefly tell how this part works and should I edit Joomla core code or just use the above code on mysql once. I am wondering if I should index table everytime when I edit menu.
A database / table Index is just a kind of yellow pages if you don't mind the comparison. It updates automatically. If this Index fixes your problem there is nothing else you need to do or do again. You can't break anything either - so just give it a try.