Okay so i didn't quite know what to call this post i might update the title later.
I have the following problem:
Say for instance you have the following calculating nested sql:
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`#`localhost`
SQL SECURITY DEFINER
VIEW `v_user_category_stat` AS
SELECT
(SELECT
SUM(MS.score) + (SELECT
SUM(UHMS.score)
FROM
User_has_module_score UHMS
RIGHT OUTER JOIN
module M ON M.id = UHMS.module_id
JOIN
Category C ON C.id = M.category_id
WHERE
UHMS.user_id = U.id)
FROM
Module_score MS
RIGHT OUTER JOIN
Module M ON M.id = MS.module_id
JOIN
Category C ON C.id = M.category_id
WHERE
MS.user_id = U.id) AS total_score,
U.id
FROM
User U
This gives me the wanted result which is:
# total_score, id
NULL, '2'
NULL, '7'
NULL, '8'
NULL, '9'
NULL, '10'
NULL, '11'
NULL, '12'
NULL, '13'
'13', '14'
Now i thought to my self i want to prettyfi this by making sure that if the value is null then it should be 0 (instead of null)
My question is how do you make a CASE that says if null THEN 0 ELSE normal?
Check the COALESCE function. In your case you will do this:
COALESCE(normal, 0)
Related
I have a MySQL database table that's quite large with 2.5million rows and growing. To speed up queries I've added an index to one of columns. When I set the index manually, through PHPMyAdmin for example, it's cardinality is around 1500 which seems about right and my queries run without issue.
The problem then comes after a few queries, (especially on an INSERT but not limited to), have been run, the cardinality of that index drops to 17 or 18, and queries run extremely slow. Sometimes it seems to work it's way back to about 1500 or I have to do it through PHPMyAdmin again.
Is there any way to stop this cardinality drop from happening?
CREATE TABLE IF NOT EXISTS `probe_results` (
`probe_result_id` int(11) NOT NULL AUTO_INCREMENT,
`date` date NOT NULL,
`month` int(11) NOT NULL,
`year` int(11) NOT NULL,
`time` time NOT NULL,
`type` varchar(11) NOT NULL,
`probe_id` varchar(50) NOT NULL,
`status` varchar(11) NOT NULL,
`temp_1` decimal(11,0) NOT NULL,
`temp_2` decimal(11,0) NOT NULL,
`crc` varchar(11) NOT NULL,
`raw_data` text NOT NULL,
`txt_file` text NOT NULL,
PRIMARY KEY (`probe_result_id`),
KEY `probe_id` (`probe_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=2527300 ;
The 'probe_result_id' column is the primary key, the probe_id is the column with the index in question.
Example query:
SELECT IF(b.reactive_total IS NULL, 0, b.reactive_total) AS reactive_total, a.* FROM (SELECT COUNT(CASE WHEN asset_testing_results.asset_testing_year = '2016' AND asset_testing_results.asset_testing_month = '7' AND asset_testing_results.asset_stopped = '0' AND asset_testing_results.asset_testing_completed = '0' THEN 1 END) AS due_total, (COUNT(CASE WHEN asset_testing_results.asset_testing_year = '2016' AND asset_testing_results.asset_stopped = '0' AND asset_testing_results.asset_testing_completed = '1' AND asset_testing_results.asset_testing_satisfactory = '1' AND asset_testing_results.asset_testing_actioned = '0' THEN 1 END)+(IF(probes_passed_total IS NULL, 0, probes_passed_total))) AS passed_total, (COUNT(CASE WHEN asset_testing_results.asset_testing_year = '2016' AND asset_testing_results.asset_stopped = '0' AND asset_testing_results.asset_testing_completed = '1' AND asset_testing_results.asset_testing_satisfactory = '0' AND asset_testing_results.asset_testing_actioned = '0' THEN 1 END)+(IF(probes_failed_total IS NULL, 0, probes_failed_total))) AS failed_total, COUNT(CASE WHEN asset_testing_results.asset_testing_year = '2016' AND asset_testing_results.asset_stopped = '0' AND asset_testing_results.asset_testing_completed = '1' AND asset_testing_results.asset_testing_actioned = '1' THEN 1 END) AS actioned_total, COUNT(CASE WHEN asset_testing_results.asset_testing_year = '2016' AND asset_testing_results.asset_testing_month < '7' AND asset_testing_results.asset_testing_completed = '0' AND asset_testing_results.asset_testing_satisfactory = '0' AND asset_testing_results.asset_stopped = '0' THEN 1 END) AS missed_total, site.site_key, site.site_name FROM site LEFT JOIN location ON location.site_key = site.site_key LEFT JOIN sub_location ON sub_location.location_key = location.location_key LEFT JOIN asset ON asset.sub_location_key = sub_location.sub_location_key AND asset.stopped = '0' LEFT JOIN asset_testing ON asset_testing.asset_type_key = asset.asset_type_key AND asset_testing.probe_assessed = '0' LEFT JOIN asset_testing_results ON asset_testing_results.asset_testing_key = asset_testing.asset_testing_key AND asset_testing_results.asset_key = asset.asset_key LEFT JOIN (SELECT site.site_key, COUNT(CASE WHEN p.probe_id IS NOT NULL AND p.asset_testing_key IS NOT NULL THEN 1 END) AS probes_passed_total, COUNT(CASE WHEN p.probe_id IS NOT NULL AND p.asset_testing_key IS NULL AND p.temp_1 IS NOT NULL THEN 1 END) AS probes_failed_total FROM assetsvs_probes LEFT JOIN (SELECT q.probe_id, q.month, q.year, IF(r.temp_1 IS NULL, q.temp_1, r.temp_1) as temp_1, r.asset_testing_key FROM (SELECT DISTINCT probe_results.probe_id, probe_results.month, probe_results.year, probe_results.temp_1 FROM probe_results LEFT JOIN assetsvs_probes ON assetsvs_probes.probe_id = probe_results.probe_id LEFT JOIN asset ON asset.asset_key = assetsvs_probes.asset_key LEFT JOIN sub_location ON sub_location.sub_location_key = asset.sub_location_key LEFT JOIN location ON location.location_key = sub_location.location_key LEFT JOIN site ON site.site_key = location.site_key WHERE site.client_key = '25')q LEFT JOIN (SELECT probe_results.month, probe_results.year, probe_results.probe_id, temp_1, asset_testing.asset_testing_key FROM probe_results LEFT JOIN assetsvs_probes ON assetsvs_probes.probe_id = probe_results.probe_id LEFT JOIN asset_testing ON asset_testing.asset_testing_key = assetsvs_probes.asset_testing_key LEFT JOIN asset ON asset.asset_key = assetsvs_probes.asset_key LEFT JOIN sub_location ON sub_location.sub_location_key = asset.sub_location_key LEFT JOIN location ON location.location_key = sub_location.location_key LEFT JOIN site ON site.site_key = location.site_key WHERE temp_1 != 'invalid' AND ((temp_1 >= test_min AND test_max = '') OR (temp_1 <= test_max AND test_min = '') OR (temp_1 >= test_min AND temp_1 <= test_max)) AND year = '2016' AND site.client_key = '25' GROUP BY probe_results.month, probe_results.year, probe_results.probe_id)r ON r.probe_id = q.probe_id AND r.month = q.month AND r.year = q.year WHERE q.year = '2016' GROUP BY probe_id, month, year) p ON p.probe_id = assetsvs_probes.probe_id LEFT JOIN asset_testing ON asset_testing.asset_testing_key = assetsvs_probes.asset_testing_key LEFT JOIN asset ON asset.asset_key = assetsvs_probes.asset_key LEFT JOIN sub_location ON sub_location.sub_location_key = asset.sub_location_key LEFT JOIN location ON location.location_key = sub_location.location_key LEFT JOIN site ON site.site_key = location.site_key GROUP BY site.site_key) probe_results ON probe_results.site_key = site.site_key WHERE site.client_key = '25' GROUP BY site.site_key)a LEFT JOIN (SELECT COUNT(CASE WHEN jobs.status = '3' THEN 1 END) AS reactive_total, site.site_key FROM jobs LEFT JOIN jobs_meta ON jobs_meta.job_id = jobs.job_id AND jobs_meta.meta_key = 'start_date' LEFT JOIN site ON site.site_key = jobs.site_key WHERE site.client_key = '25' AND jobs_meta.meta_value LIKE '%/2016 %' GROUP BY site.site_key)b ON b.site_key = a.site_key
Thanks
Cardinality (along with other statistics) is calculated and updated by MySQL automatically, so you do not have direct means to prevent it from dropping.
However, you can take a few steps to make this less likely to happen or correct the behaviour.
First of all, MySQL updates index statistics for all supported table engines if you run analyze table command.
For innodb table engine MySQL provides a set of configuration settings that can influence the behaviour of the sampling. The settings and their effect are described in the MySQL documentation:
non-persistent statistics
persistent statistics
The main setting is innodb_stats_transient_sample_pages:
• Small values like 1 or 2 can result in inaccurate estimates of
cardinality.
• Increasing the innodb_stats_transient_sample_pages value might
require more disk reads. Values much larger than 8 (say, 100), can
cause a significant slowdown in the time it takes to open a table or
execute SHOW TABLE STATUS.
• The optimizer might choose very different query plans based on
different estimates of index selectivity
.
For myisam MySQL dos not provide such a variety of settings. myisam_stats_method setting is described in the general index statistics documentation
I have the following nested query:
SELECT `messages`.*,
IF((select `status` from messages_status
where messages_status.message_id = messages.id and user_id = 149) IS NULL,
'unread', messages_status.status) as `status`
FROM `messages`
What I would like to do, if there is no messages_status.status set (i.e. if it is NULL), it should return 'unread'. However, if it is set, is should return its value.
Currently, it returns an error: Unknown column 'messages_status.status' in 'field list'
Do you have any idea how to fix this?
You can't refer to the result of the if's condition in other places. You could repeat the query:
IF((select `status` from messages_status
where messages_status.message_id = messages.id and user_id = 149) IS NULL,
'unread',
(select `status` from messages_status
where messages_status.message_id = messages.id and user_id = 149))
as status
But it's probably clearer to use a join instead:
select m.*
, coalesce(ms.status, 'unknown')
from messages m
left join
messages_status ms
on ms.message_id = m.id
and ms.user_id = 149
The coalesce function produces the first of its arguments that is not null.
SELECT
(
`members`.`id`
SELECT COUNT(`members`.`id`) FROM `members` WHERE `gender` = 0 AS `Unknown`
SELECT COUNT(`members`.`id`) FROM `members` WHERE `gender` = 1 AS `Female`
SELECT COUNT(`members`.`id`) FROM `members` WHERE `gender` = 2 AS `Male`
) FROM `members` INNER JOIN `mapMember`
ON `mapMember`.`id` = `members`.`id`
WHERE `mapMember`.`mapper_id` = 3
My expected result:
Unknown Female Male
0 1 3
However I get SYNTAX error. Cant' figure out what's wrong.
I also tried:
SELECT COUNT(`members`.id) AS `members`, `gender`
FROM `members` INNER JOIN `mapMember`
ON `mapMember`.`id` = `members`.`id`
WHERE `mapMember`.`mapper_id` = 3 GROUP BY `gender` ORDER BY `gender` ASC
Which gives me almost the result I want to have, the only difference is If the there are no members with the given gender, there won't be a 0 result back. (no row that is) I always expect three rows back.
SELECT
sum(if (`gender` = 0, 1,0)) as `Unknown`,
sum(if (`gender` = 1, 1,0)) as `Female`,
sum(if (`gender` = 2, 1,0)) as `Male`
FROM `members` INNER JOIN `mapMember`
ON `mapMember`.`id` = `members`.`id`
WHERE `mapMember`.`mapper_id` = 3
SELECT * FROM
(
SELECT `members`.`id`,COUNT(`members`.`id`) AS `Unknown` FROM `members` WHERE `gender` = 0
UNION
SELECT `members`.`id`,COUNT(`members`.`id`) AS `Female` FROM `members` WHERE `gender` = 1
UNION
SELECT `members`.`id`,COUNT(`members`.`id`) AS `Male` FROM `members` WHERE `gender` = 2
) Z INNER JOIN `mapMember`
ON `mapMember`.`id` = `Z`.`id`
WHERE `mapMember`.`mapper_id` = 3
Others have given you solutions, so I mainly tell you where you went wrong with your own statement.
COUNT(column_name) simply counts records where column_name is not null. members.id is not null, so you simply count all records from members. You need a where clause instead limiting the counted records to the member id in question.
Sub queries must be in parentheses.
Here is your statement re-written. It is not good though, because you query the same table again and again. I just wanted to use your statement and only correct errors:
SELECT
`members`.`id`,
(SELECT COUNT(*) FROM `members` u WHERE `gender` = 0 AND u.id = members.id) AS `Unknown`
(SELECT COUNT(*) FROM `members` f WHERE `gender` = 1 AND f.id = members.id) AS `Female`
(SELECT COUNT(*) FROM `members` m WHERE `gender` = 2 AND m.id = members.id) AS `Male`
FROM `members` INNER JOIN `mapMember`
ON `mapMember`.`id` = `members`.`id`
WHERE `mapMember`.`mapper_id` = 3;
Now it's syntactically correct. However, as a member record has exactly one gender, you will always get records with 0-0-1 or 0-1-0 or 1-0-0. So you don't really want to select members and have the counts per member.
Here is a better statement querying the tables just once, counting over all records rather than per member and providing better readabilty by using an IN clause for mapmember. (You can as well replace the IN clause with an EXISTS clause, which is sometimes faster.)
select
sum( case when gender = 0 then 1 else 0 end ) as unknown,
sum( case when gender = 1 then 1 else 0 end ) as female,
sum( case when gender = 2 then 1 else 0 end ) as male
from members
where id in (select id from mapmember where mapper_id = 3);
(BTW: Is the mapmember id really a members id? It looks strange to have a table with a column named id and this not being the id of the table itself but the id of another table actually.)
EDIT: I just notice you use MySQL. There you have a boolean data type you can use:
select
sum( gender = 0 ) as unknown, sum( gender = 1 ) as female, sum( gender = 2 ) as male
from members
where id in (select id from mapmember where mapper_id = 3);
This is no longer standard SQL, because it uses an enhancement from MySQL.
I have the following query, that I use to filter rows based on software_id and level.
I've put the conditions in the ON-Clause since I still want rows returned, where there are no corresponding rows in the JobadvertsSoftware Table.
SELECT `Jobadvert`.`id` FROM `jobadverts` AS `Jobadvert`
LEFT JOIN `users` AS `User` ON (`Jobadvert`.`user_id` = `User`.`id`)
LEFT JOIN `jobadverts_softwares` AS `JobadvertsSoftware_0` ON
(`Jobadvert`.`id` = 'JobadvertsSoftware_0.jobadvert_id' AND
(`JobadvertsSoftware_0`.`software_id` = '32' AND
`JobadvertsSoftware_0`.`level` IN ('1', 4)))
WHERE `Jobadvert`.`active` = 1 AND `User`.`premium` = '1' AND
Jobadvert`.`department_id` = (5)
GROUP BY `Jobadvert`.`id`
The problem is that it also returns JobadvertsSoftware-rows where level is e.g. 2
Again, if I put that in the WHERE clause it will filter out the rows where there are not JobadvertsSoftware which it shouldn't do.
How can I tell MySQL to return all rows of Jobadvert, where the given software_id AND the level matches or are NULL?
Try this:
SELECT `Jobadvert`.`id`, `JobadvertsSoftware_0`.`level`
FROM `jobadverts` AS `Jobadvert`
LEFT JOIN `users` AS `User` ON (`Jobadvert`.`user_id` = `User`.`id`)
INNER JOIN `jobadverts_softwares` AS `JobadvertsSoftware_0` ON
(`Jobadvert`.`id` = 'JobadvertsSoftware_0.jobadvert_id' AND
(`JobadvertsSoftware_0`.`software_id` = '32' AND
`JobadvertsSoftware_0`.`level` IN ('1', 4)))
WHERE `Jobadvert`.`active` = 1 AND `User`.`premium` = '1' AND
Jobadvert`.`department_id` = (5)
GROUP BY `Jobadvert`.`id`
Saludos!
Try this( it's a bit unclear if some fields are numeric on string, it might be corrected):
SELECT distinct(`Jobadvert`.`id`) FROM `jobadverts` AS `Jobadvert`
LEFT JOIN `users` AS `User` ON (`Jobadvert`.`user_id` = `User`.`id`)
LEFT JOIN `jobadverts_softwares` AS `JobadvertsSoftware_0`
ON `Jobadvert`.`id` = `JobadvertsSoftware_0.jobadvert_id`
WHERE
`Jobadvert`.`active` = 1
AND `User`.`premium` = '1'
AND Jobadvert`.`department_id` = (5)
AND JobadvertsSoftware_0`.`software_id` = '32'
AND (`JobadvertsSoftware_0`.`level` IN (1, 4) OR `JobadvertsSoftware_0`.`level` is NULL)
Assuming the level parameters in your ON clause is not needed for the join you can do a nested SELECT on your Software table to clear out the data you do not need first:
SELECT * FROM jobadverts_softwares
WHERE
(`software_id` = 32 OR `software_id` IS NULL) --Select all software_id that are 32 or null
AND
`level` IN (1, 4)
Then you can incorporate this as a nested statement in your main SQL query so you only join on the data which is filtered in your LEFT JOIN but keep any null values that you needed:
SELECT `Jobadvert`.`id`
FROM `jobadverts` AS `Jobadvert`
LEFT JOIN `users` AS `User`
ON `Jobadvert`.`user_id` = `User`.`id`
LEFT JOIN
( --Software Subquery
SELECT `jobadvert_id`, `level` FROM jobadverts_softwares
WHERE
(`software_id` = 32 OR `software_id` IS NULL) --Select all software_id that are 32 or null
AND
`level` IN (1, 4)
) AS `software_subquery`
ON `Jobadvert`.`id` = `software_subquery`.`jobadvert_id`
WHERE
`Jobadvert`.`active` = 1
AND
`User`.`premium` = '1'
AND
`Jobadvert`.`department_id` = 5
ORDER BY `Jobadvert`.`id` --Changed GROUP BY to ORDER BY as not needed
This is untested but try it out and see if this will help.
Try this:
SELECT j.id
FROM jobadverts j
LEFT JOIN User u ON (j.user_id = u.id)
LEFT JOIN jobadverts_softwares AS js ON
(j.id = js.jobadvert_id)
WHERE j.active = 1
AND u.premium = '1'
AND j.department_id = (5)
AND js.software_id` = '32'
AND js.level IN ('1', 4)))
You won't need a GROUP BY unless summing data in some way.
I am trying to get results that i know are there but just can't seem to get the query right. I am using the following:
SELECT
*
FROM
carpets AS c
INNER JOIN carpet_relations AS r ON c.id = r.carpet_id
WHERE
c.active = '1'
AND **((r.relation_type = '5')
AND (r.related_id = '1' ))**
*AND* ((r.relation_type = '4')
AND (r.related_id = '1'))
AND (c.width_feet BETWEEN '0' AND '17')
AND (c.width_inches BETWEEN '0' AND '11')
AND (c.height_feet BETWEEN '0' AND '29')
AND (c.height_inches BETWEEN '0' AND '11')
ORDER BY
c.item_no
as you can see i am trying get results that have two matching fields in a second table if i change the and that is in italics to an or i get results but it is results for both i need the results that are in the set prior to the and which is bold
so it would be something like:
list that has relation type 5 and 4
keep in mind that related id could be different for both there are three colums carpet_id,related_id,relation_type
thanks for any help
c_categories
Column Type Null Default Comments
id int(11) No
title varchar(250) No
active int(11) No
weight int(11) No
template_id int(11) No
c_sizes
Column Type Null Default Comments
id int(11) No
title varchar(250) No
active int(11) No
weight int(11) No
template_id int(11) No
carpet_relations
Column Type Null Default Comments
carpet_id int(11) No <------ signifies which carpet the relationship is with
related_id int(11) No <------ the id of the c_size or c_categories to use
relation_type int(11) No <------ signifies which table either c_sizes or c_categories
Change this:
((r.relation_type = '5') AND (r.related_id = '1' )) AND ((r.relation_type = '4') AND (r.related_id = '1'))
to this:
(r.relation_type = '5' OR r.relation_type = '4') AND (r.related_id = '1')
For flexibility's sake:
r.relation_type IN (4, 5)
Now I think I understand your problem - you're after the intersection of results of type 5 and type 4.
This could be done in PHP as you mention by doing two queries and intersecting them, or you can do this in MySQL by joining the carpet_relations table on twice like so:
SELECT
*
FROM
carpets AS c
INNER JOIN carpet_relations AS r1 ON c.id = r1.carpet_id
INNER JOIN carpet_relations AS r2 ON c.id = r2.carpet_id
WHERE
c.active = '1'
AND (r1.relation_type = '5')
AND (r1.related_id = '1' )
AND (r2.relation_type = '4')
AND (r2.related_id = '1')
AND (c.width_feet BETWEEN '0' AND '17')
AND (c.width_inches BETWEEN '0' AND '11')
AND (c.height_feet BETWEEN '0' AND '29')
AND (c.height_inches BETWEEN '0' AND '11')
ORDER BY
c.item_no
AND ((r.relation_type = '5')
AND ((r.relation_type = '4')
you are expecting relation type to be 5 and 4 at the same time.
Would you like to have some "or" in your salad?
edit:
SELECT
*
FROM
carpets AS c
INNER JOIN carpet_relations AS r ON c.id = r.carpet_id
WHERE
c.active = '1'
AND
((r.relation_type = '5') AND (r.related_id = '1' ))
OR
((r.relation_type = '4') AND (r.related_id = '1'))
AND (c.width_feet BETWEEN '0' AND '17')
AND (c.width_inches BETWEEN '0' AND '11')
AND (c.height_feet BETWEEN '0' AND '29')
AND (c.height_inches BETWEEN '0' AND '11')
ORDER BY
c.item_no
solves your problem.