MySQL Different results from same query/data - mysql

I have two servers running MySQL. Both are on windows. One, is my local machime (Windows 7, MySQL 5.6.25, 32bit) and the other is my production webserver (Windows 2012, MySQL 5.7.11-log, 64bit (that's what show variables showed me).
The data is identical between the two. I backed the data up from the windows 7 (using MySQL Workbench) and restored it on the 2012 machine.
I am running a query on both machine but I am getting different results. I have two tables, projects and projectsnotes with a 1:m relationship between them related on projects.id to projectsnotes.idProject. Each note is marked with a date (dComment). The goal of the query is to retrieve project information and the latest comment only.
Here's the query:
select space(1) as cAction,
p.id,
p.iNum,
p.cStatus,
p.cName,
p.cDesc,
ifnull(pl.cNickName, 'UNASSIGNED') as cProjectLeader,
IFNULL(concat (
date_format(pn.dComment, '%Y-%m-%d'),
': ',
pn.cComment
), '') as cComment,
date_format(p.dRequested, '%Y-%m-%d') as dRequested,
date_format(p.dRequired, '%Y-%m-%d') as dRequired,
format(p.nPercentComplete, 2) as nPercentComplete,
p.tLastUpdated,
p.bCompleted,
p.idProjectLeader
from projects p
left outer join projectleaders pl on p.idProjectLeader = pl.id
left outer join (
select idProject,
dComment,
cComment
from projectnotes
order by dComment desc,
tLastUpdated desc
) pn on p.id = pn.idProject
where p.cInstallCode = 'ITM'
and cStatus in ('Pending', 'Active', 'On Hold', 'Completed', 'Cancelled')
and bCompleted = 0
group by iNum
order by iNum;
Now, here's the weird part. When I run this on my Windows 7 machine, I get the right value for cComment. Specifically:
2017-03-28: Text from note replace
That is the latest note. When I run it on the 2012 server:
2016-05-17: Text from this note replaced too
If I run the subquery alone on the 2012 server, I get the right values (namely, a list of all the notes in the reverse order.
Oh, and this note is neither the first nor the last in the notes for this project.
So I am honestly wondering what is going on. Any thoughts on this would be greatly appreciated.
Thanks in advance.

This is expected behavior.
select ...
from projects p
left outer join projectleaders pl on p.idProjectLeader = pl.id
left outer join (...) pn on p.id = pn.idProject
where ...
group by iNum
order by iNum;
Due to MySQL's peculiar handling of GROUP BY, it will not report an error on this query. However, you must keep in mind that, since you use no aggregates, and the GROUP BY will eliminate lots of rows, the rows that are kept in the final result set are determined by rather obscure criteria...
For example:
SELECT a,b FROM t GROUP BY a
Which b will be returned? In some MySQL versions, this will be the first value of b that is found in table t. If table t is ordered in a certain way, this can be exploited. But I would definitely not trust that behavior to stay unchanged between versions... Also, remember MySQL is free to change your join order...

OK. I think I have a solution to this. Instead of doing it with a join I wrote a function that returned the value I needed as follows:
DROP FUNCTION if exists f_lastprojectnote;
DELIMITER $$
CREATE FUNCTION f_lastprojectnote(tidProject varchar(36))
RETURNS varchar(1000) DETERMINISTIC
BEGIN
DECLARE cRetVal VARCHAR(1000);
SELECT concat(date_format(pn.dComment, '%Y-%m-%d'), ': ', pn.cComment) INTO cRetVal
FROM projectnotes pn
WHERE idProject = tidProject
ORDER BY dComment DESC, tLastUpdated DESC
LIMIT 1;
RETURN cRetVal;
END$$
DELIMITER ;
It works...

Related

SQL: Something wrong with inheriting variables for NULL next-row values

I'm trying to inherit value from previous row (based on correct subscription_id + checking for IS NULL subscription_status), but something goes wrong and I get incorrect value.
Take a look at screenshot.
If I'm not mistaken it also called last non-null puzzle, but examples of possible solution for other DB provide window function with IGNORE NULLS.
But, I'm using MySQL 8.x and it doesn't support this function.
I'm sorry, but SQL fiddle doesn't provide correct text-value for variables in my code :(
https://www.db-fiddle.com/f/wHanqoSCHKJHus5u6BU4DB/4
Or, you can see mistakes here:
SET #history_subscription_status = NULL;
SET #history_subscription_id = 0;
SELECT
c.date,
c.user_id,
c.subscription_id,
sd.subscription_status,
(#history_subscription_id := c.subscription_id) as 'historical_sub_id',
(#history_subscription_status := CASE
WHEN #history_subscription_id = c.subscription_id AND sd.subscription_status IS NULL
THEN #history_subscription_status
ELSE
sd.subscription_status
END
) as 'historical'
FROM
calendar c
LEFT JOIN
subscription_data sd ON sd.date = c.date AND sd.user_id = c.user_id AND sd.subscription_id = c.subscription_id
ORDER BY
c.user_id,
c.subscription_id,
c.date
I expect to get results for this query in this way:
IMPORTANT: I'm going to use this code for a lot of data (about 1 mln rows), so it very important for me to avoid additional select or subquery that can slow down the execution of the query.

MySQL DISTINCT returning not so distinct results

Good day,
I have a small issue with MySQL Distinct.
Trying the following query in my system :
SELECT DISTINCT `booking_id`, `booking_ticket`, `booking_price`, `bookingcomment_id`, `bookingcomment_message` FROM `mysystem_booking`
LEFT JOIN `mysystem_bookingcomment` ON `mysystem_booking`.`booking_id` = `mysystem_bookingcomment`.`bookingcomment_link`
WHERE `booking_id` = 29791
The point is that there are bookings like 29791 that have many comments added.
Let's say 10. Then when running the above query I see 10 results instead of one.
And that's not the way DISTINCT supposes to work.
I simply want to know if there are any comments. If the comment ID is not 0 then there is a comment. Of course I can add COUNT(blabla) as comment_number but that's a whole different story. For me now I'd like just to have this syntax right.
You may try aggregating here, to find which bookings have at least a single comment associated with them:
SELECT
b.booking_id,
b.booking_ticket,
b.booking_price
FROM mysystem_booking b
LEFT JOIN mysystem_bookingcomment bc
ON b.booking_id = bc.bookingcomment_link
WHERE
b.booking_id = 29791
GROUP BY
b.booking_id
HAVING
COUNT(bc.bookingcomment_link) > 0;
Note that depending on your MySQL server mode, you might have to also add the booking_ticket and booking_price columns to the GROUP BY clause to get the above query to run.
You can try below - using a case when expression
SELECT DISTINCT `booking_id`, `booking_ticket`, `booking_price`, `bookingcomment_id`,
case when `bookingcomment_message`<>'0' then 'No' else 'Yes' end as comments
FROM `mysystem_booking`
LEFT JOIN `mysystem_bookingcomment` ON `mysystem_booking`.`booking_id` = `mysystem_bookingcomment`.`bookingcomment_link`
WHERE `booking_id` = 29791

How to increase performance of complicated MySQL select query?

I am working on project management tool and while generating reports through admin end the loading time for the data from the database is very much > 5 minutes. There are few points which I know would help me to increase the performance but right now I need to have the help in SELECT query
SELECT timesheet_client.organisation as client_name, timesheet_project.title as project_name,
timesheet_task.name as task, CONCAT(timesheet_user.first_name, ' ', timesheet_user.last_name) as resource_name,
timesheet_user.bill_factor, timesheet_client.client_type, sum(spent) as spent, sum(delivered_hours) as delivered_hours,
sum(billable_hours) as billable_hours, comments, color, lock_color, updated_by, updated_by_date, timesheet_user_psdb.grp_id,
timesheet_user_psdb.client_id, timesheet_user_psdb.proj_id, timesheet_user_psdb.task_id, timesheet_user_psdb.uid,
timesheet_user_grp.grp_name
FROM timesheet_user_psdb,timesheet_user, timesheet_client,
timesheet_project,timesheet_task,timesheet_user_grp
WHERE timesheet_user.username=timesheet_user_psdb.uid and
timesheet_client.client_id=timesheet_user_psdb.client_id and timesheet_project.proj_id=timesheet_user_psdb.proj_id and
timesheet_task.task_id = timesheet_user_psdb.task_id and timesheet_user_grp.grp_id=timesheet_user_psdb.grp_id and month =3
AND year = 2017 and month!='' and timesheet_user_psdb.client_id=326
GROUP BY timesheet_user_psdb.task_id,timesheet_user_psdb.uid
ORDER BY timesheet_client.client_type desc,timesheet_client.organisation,timesheet_user_psdb.proj_id,
timesheet_user_psdb.task_id,timesheet_user.uid,timesheet_user_psdb.task_id;
I have already used an index on all the primary keys.
EXPLAIN Output:
Help for this would be highly appreciable.
Give this a try:
SELECT
TC.ORGANISATION AS CLIENT_NAME,
TP.TITLE AS PROJECT_NAME,
TT.NAME AS TASK,
CONCAT(TU.FIRST_NAME, ' ', TU.LAST_NAME) AS RESOURCE_NAME,
TU.BILL_FACTOR,
TC.CLIENT_TYPE, SUM(SPENT) AS SPENT, -- You should specify which table this comes from
SUM(DELIVERED_HOURS) AS DELIVERED_HOURS, -- You should specify which table this comes from
SUM(BILLABLE_HOURS) AS BILLABLE_HOURS, -- You should specify which table this comes from
COMMENTS, -- You should specify which table this comes from
COLOR, -- You should specify which table this comes from
LOCK_COLOR, -- You should specify which table this comes from
UPDATED_BY, -- You should specify which table this comes from
UPDATED_BY_DATE, -- You should specify which table this comes from
TUP.GRP_ID,
TUP.CLIENT_ID,
TUP.PROJ_ID,
TUP.TASK_ID,
TUP.UID,
TUG.GRP_NAME
FROM
TIMESHEET_USER AS TU
LEFT OUTER JOIN
TIMESHEET_USER_PSDB AS TUP
ON TU.USERNAME = TUP.UID
LEFT OUTER JOIN
TIMESHEET_USER_GRP AS TUG
ON TUP.GRP_ID = TUG.GRP_ID
LEFT OUTER JOIN
TIMESHEET_CLIENT AS TC
ON TUP.CLIENT_ID = TC.CLIENT_ID
LEFT OUTER JOIN
TIMESHEET_PROJECT AS TP
ON TUP.PROJ_ID = TP.PROJ_ID
LEFT OUTER JOIN
TIMESHEET_TASK TT
ON TUP.TASK_ID = TT.TASK_ID
WHERE
MONTH = 3 AND -- You should specify which table this comes from
YEAR = 2017 AND -- You should specify which table this comes from
MONTH != '' AND -- You should specify which table this comes from
TUP.CLIENT_ID = 326
GROUP BY
TUP.TASK_ID,
TUP.UID
ORDER BY
TC.CLIENT_TYPE DESC,
TC.ORGANISATION,
TUP.PROJ_ID,
TUP.TASK_ID,
TU.UID,
TUP.TASK_ID;
Doing the EXPLAIN should shed some light on it.
You might see a performance gain by doing the table joins explicitly in the FROM clause instead of inside the WHERE (https://dev.mysql.com/doc/refman/5.7/en/join.html). By doing explicit joins (e.g. LEFT OUTER, etc...) you will not only improve the readability of the query, but may be able to use less expensive joins where needed. This also affects how the query is executed as each clause is executed in a specific order (MySQL query / clause execution order).

SQL Server LEFT JOIN fails to match rows without JOIN hint

I have what appears to be a corrupt index?
Here is what is happening. I have two table-functions which the first is a set of cases and the second is a set of aware dates. These two sets have a 1 (case) to 0 or 1 (aware date) relationship. Normally I query them like;
SELECT c.CaseID, a.AwareDate
FROM Cases(#date) AS c
LEFT JOIN AwareDates(#date) AS a ON c.CaseID = a.CaseID;
The trouble is that not all of the rows from AwareDates which match seem to be JOIN'd. If I add a join hint, they then do. say;
SELECT c.CaseID, a.AwareDate
FROM Cases(#date) AS c
LEFT MERGE JOIN AwareDates(#date) AS a ON c.CaseID = a.CaseID;
What I notice from the query plan is that adding the join hint adds a sort of the AwareDate data before the join which is not there otherwise. Also, the query planner flips the join to a RIGHT OUTER JOIN when there is no hint, and of course keeps the LEFT JOIN where the hint is present.
I've done the following with no errors detected;
DBCC UPDATEUSAGE (0) WITH INFO_MESSAGES, COUNT_ROWS;
EXECUTE sp_updatestats 'resample';
DBCC CHECKDB (0) WITH ALL_ERRORMSGS, EXTENDED_LOGICAL_CHECKS;
I'm stumped... any ideas?
Here are the UDF definitions
ALTER FUNCTION dbo.Cases( #day date ) RETURNS TABLE
WITH SCHEMABINDING
AS RETURN (
SELECT
CaseID -- other 42 columns ommitted
FROM (
SELECT
ROW_NUMBER() OVER (PARTITION BY CaseID ORDER BY UpdateDate DESC, UpdateNumber DESC) AS RecordAge,
CaseID,
Action
FROM
dbo.CaseAudit
WHERE
convert(date,UpdateDate) <= #day
) AS History
WHERE
RecordAge = 1 -- only the most current record version
AND isnull(Action,'') != N'DEL' -- only include cases that have not been deleted
)
ALTER FUNCTION dbo.AwareDates( #day date ) RETURNS TABLE
WITH SCHEMABINDING
AS RETURN (
WITH
History AS (
SELECT row_number() OVER (PARTITION BY CaseID, ContactID ORDER BY UpdateDate DESC, UpdateNumber DESC) AS RecordAge,
CaseID, InfoReceived, ReceiveDate, ResetClock, Action
FROM dbo.ContactLogAudit WITH (NOLOCK)
WHERE convert(date,UpdateDate) <= #day
),
Notes AS (
SELECT
CaseID,
convert(date,ReceiveDate,112) AS ReceiveDate,
ResetClock
FROM History
WHERE RecordAge = 1 -- only the most current record version
AND isnull(Action,'') != N'DEL' -- only include notes that have not been deleted
AND InfoReceived = N'Y' -- only include notes that have Info Rec'd checked
AND len(ReceiveDate) = 8 AND isnumeric(ReceiveDate) = 1 AND isdate(ReceiveDate) = 1 -- only include those with a valid aware date
),
Initials AS (
SELECT CaseID, min(ReceiveDate) AS ReceiveDate
FROM Notes
GROUP BY CaseID
),
Resets AS (
SELECT CaseID, max(ReceiveDate) AS ReceiveDate
FROM Notes
WHERE ResetClock = N'Y'
GROUP BY CaseID
)
SELECT
i.CaseID AS CaseID,
i.ReceiveDate AS InitialAwareDate, -- the oldest valid aware date value (must have AE Info Reveived checked and a received date)
coalesce(r.ReceiveDate,i.ReceiveDate) AS AwareDate -- either the newest valid aware date value with the Reset Clock checked, otherwise the initial aware date value
FROM Initials AS i
LEFT JOIN Resets AS r
ON i.CaseID = r.CaseID
);
I have further found that if I drop the "WITH (NOLOCK)" table hint, I get correct results. Also if add a join hint to the AwareDates UTF or even add a COLLATE Latin1_General_BIN on the LEFT JOIN relation between Initials and Resets.
Query plan row counts -- without join hint (broken)
Cases { Actual: 25,891, Estimate: 19,071.9 }
AwareDates { Actual: 24,693, Estimated: 1,463.09 }
Initials { Actual: 24,693, Estimated: 1,463.09 }
Rests { Actual: 985, Estimated: 33.2671 }
AwareDates matches 8,108 of the Cases rows in the join'd result-set
Query plan row counts -- with join hint (working)
Cases { Actual: 25,891, Estimate: 19,071.9 }
AwareDates { Actual: 24,673, Estimated: 1,837.67 }
Initials { Actual: 24,673, Estimated: 1,837.67 }
Rests { Actual: 982, Estimated: 42.6238 }
AwareDates matches 24,673 of the Cases rows in the join'd result-set
I have further whittled down the scope of the issue. I can;
SELECT * FROM AwareDate(#date);
and
SELECT * FROM AwareDate(#date) ORDER BY CaseID;
With different row counts.
You don't specify the specific version of SQL (##version), but this seems suspiciously like a bug that was fixed in Cumulative Update 6 for SQL 2008 R2 (apparently it also applies to SQL 2008).
KB 2433265
FIX: You may receive an incorrect result when you run a query that uses the
ROW_NUMBER function together with a left outer join in SQL Server 2008
The example in the article specifies DISTINCT. The article, however, is worded ambiguously -- it's not clear whether you NEED a distinct or if DISTINCT is one of the triggers.
Your example doesn't have a distinct like the article, but it appears modified for the sake of asking the question(i.e. 42 columns missing). Is there a distinct? Also in the AwareDates udf by the time i get down to the Initials CTE you do a GROUP BY which could have the same effect as a DISTINCT.
UPDATE
#Dennis from your comment I still can't tell if you're using SQL 20080 or 2008 R2.
If you're running 2008, the KB article says "The fix for this issue was first released in Cumulative Update 11 for SQL Server 2008 Service Pack 1." So, post SP1.
On the other hand, if you're using SQL 2008 R2, you are correct that this was fixed in CU 6, which was part of SP1. But this bug appears to have resurfaced. Look at Cumulative update package 4 for SQL Server 2008 R2 Service Pack 1 -- released post SP1.
970198 FIX: You receive an incorrect result when you run a
query that uses the row_number function in SQL Server 2008
or in SQL Server 2008 R2
In the associated KB article MS dropped the reference to distinct:
Consider the following scenario. You run a query against a table that has a
clustered index in Microsoft SQL Server 2008 or in Microsoft SQL Server 2008
R2. In the query, you use the row_number function. In this scenario, you
receive an incorrect result when a parallel execution plan is used for the
query. If you run the query many times, you may receive different results.
This seems to confirm my earlier reading of KB 2433265 -- the phrasing suggests distinct is just one of many conditions that can cause the behavior. It seems that a parallel execution plan is the culprit this time around.

MySQL COUNT() causing empty array() return

MySQL Server Version: Server version: 4.1.14
MySQL client version: 3.23.49
Tables under discussion: ads_list and ads_cate.
Table Relationship: ads_cate has many ads_list.
Keyed by: ads_cate.id = ads_list.Category.
I am not sure what is going on here, but I am trying to use COUNT() in a simple agreggate query, and I get blank output.
Here is a simple example, this returns expected results:
$queryCats = "SELECT id, cateName FROM ads_cate ORDER BY cateName";
But if I modify it to add the COUNT() and the other query data I get no array return w/ print_r() (no results)?
$queryCats = "SELECT ads_cate.cateName, ads_list.COUNT(ads_cate.id),
FROM ads_cate INNER JOIN ads_list
ON ads_cate.id = ads_list.category
GROUP BY cateName ORDER BY cateName";
Ultimately, I am trying to get a count of ad_list items in each category.
Is there a MySQL version conflict on what I am trying to do here?
NOTE: I spent some time breaking this down, item by item and the COUNT() seems to cause the array() to disappear. And the the JOIN seemed to do the same thing... It does not help I am developing this on a Yahoo server with no access to the php or mysql error settings.
I think your COUNT syntax is wrong. It should be:
COUNT(ads_cate.id)
or
COUNT(ads_list.id)
depending on what you are counting.
Count is an aggregate. means ever return result set at least one
here you be try count ads_list.id not null but that wrong. how say Myke Count(ads_cate.id) or Count(ads_list.id) is better approach
you have inner join ads_cate.id = ads_list.category so Count(ads_cate.id) or COUNT(ads_list.id) is not necessary just count(*)
now if you dont want null add having
only match
SELECT ads_cate.cateName, COUNT(*),
FROM ads_cate INNER JOIN ads_list
ON ads_cate.id = ads_list.category
GROUP BY cateName
having not count(*) is null
ORDER BY cateName
all
SELECT ads_cate.cateName, IFNULL(COUNT(*),0),
FROM ads_cate LEFT JOIN ads_list
ON ads_cate.id = ads_list.category
GROUP BY cateName
ORDER BY cateName
Did you try:
$queryCats = "SELECT ads_cate.cateName, COUNT(ads_cate.id)
FROM ads_cate
JOIN ads_list ON ads_cate.id = ads_list.category
GROUP BY ads_cate.cateName";
I am guessing that you need the category to be in the list, in that case the query here should work. Try it without the ORDER BY first.
You were probably getting errors. Check your server logs.
Also, see what happens when you try this:
SELECT COUNT(*), category
FROM ads_list
GROUP BY category
Your array is empty or disappear because your query has errors:
there should be no comma before the FROM
the "ads_list." prefix before COUNT is incorrect
Please try running that query directly in MySQL and you'll see the errors. Or try echoing the output using mysql_error().
Now, some other points related to your query:
there is no need to do ORDER BY because GROUP BY by default sorts on the grouped column
you are doing a count on the wrong column that will always give you 1
Perhaps you are trying to retrieve the count of ads_list per ads_cate? This might be your query then:
SELECT `ads_cate`.`cateName`, COUNT(`ads_list`.`category`) `cnt_ads_list`
FROM `ads_cate`
INNER JOIN `ads_list` ON `ads_cate`.`id` = `ads_list`.`category`
GROUP BY `cateName`;
Hope it helps?