How to improve ColdFusion MySQL query time? - mysql

I have 15 queries that generate data for a table on a page dynamically for the purpose of reports. Each query takes between 250 and 900ms which means a page loading time of 4 to 13 seconds depending on server load. The loading time is causing some users to the think the page is not going to load at all.
I was wondering if there was some way I could streamline the queries to give a more acceptable loading time. Here is one of the queries:
<cfquery datasource="MeetingDB" name="One">
SELECT COUNT( meetingID ) AS countatron
FROM case_meeting
WHERE meetingID
IN (
SELECT DISTINCT a.meetingID
FROM case_meeting a
INNER JOIN meeting b ON a.meetingID = b.meetingID
WHERE b.categoryID = '1'
AND SUBSTRING( meetingCode, 5, 2 )
BETWEEN 12
AND 22
AND SUBSTRING( meetingCode, 7, 2 )
BETWEEN 01
AND 12
AND SUBSTRING( meetingCode, 9, 2 )
BETWEEN 01
AND 31
)
AND caseID
IN (
'1', '2', '3', '28', '29', '30', '39', '40', '45'
)
GROUP BY meetingID
HAVING COUNT( caseID ) > 0 AND COUNT( caseID ) < 2
</cfquery>
<td><cfoutput> #One.recordcount# </cfoutput></td>

Try this query
SELECT COUNT( a.meetingID ) AS countatron
FROM case_meeting a, case_meeting b
WHERE a.meetingID = b.meetingID
AND b.categoryID = '1'
AND SUBSTRING( b.meetingCode, 5, 2 )
BETWEEN 12
AND 22
AND SUBSTRING( b.meetingCode, 7, 2 )
BETWEEN 01
AND 12
AND SUBSTRING( b.meetingCode, 9, 2 )
BETWEEN 01
AND 31
AND b.caseID
IN (
'1', '2', '3', '28', '29', '30', '39', '40', '45'
)
GROUP BY a.meetingID
HAVING COUNT( a.caseID ) = 1

Might be worth trying to do joins on subselects rather than using IN.
Something like this:-
SELECT COUNT( case_meeting.meetingID ) AS countatron
FROM case_meeting
INNER JOIN (
SELECT DISTINCT a.meetingID
FROM case_meeting a
INNER JOIN meeting b ON a.meetingID = b.meetingID
WHERE b.categoryID = '1'
AND SUBSTRING( meetingCode, 5, 2 ) BETWEEN 12 AND 22
AND SUBSTRING( meetingCode, 7, 2 ) BETWEEN 01 AND 12
AND SUBSTRING( meetingCode, 9, 2 ) BETWEEN 01 AND 31
) Sub1
ON case_meeting.meetingID = Sub1.meetingID
INNER JOIN (
SELECT meetingID, COUNT( caseID ) AS MeetingCaseCount
FROM case_meeting
WHERE caseID IN ('1', '2', '3', '28', '29', '30', '39', '40', '45')
GROUP BY meetingID
) Sub2
ON case_meeting.meetingID = Sub2.meetingID
WHERE Sub2.MeetingCaseCount > 0 AND Sub2.MeetingCaseCount < 2
GROUP BY case_meeting.meetingID

If you have a lot of queries that don't depend on each other, then take a look at cfthread. This will allow you to run the queries concurrently.
Make sure you test it thoroughly. I've had one experience where the use of cfthread had adverse effects on a database server.
It's still worth a shot though.

i suppose creating a procedure in the MySql and invoking it with arguments from my sql is most appropriate. you could also create views with dynamic param

Related

How to check values in mysql for preceeding rows and using them to find sum

I have to find the time, that the system that is based on this table has elapsed while having code '100', so firstly i thought that I have to find the newest row of the xID group and after that, check the previous rows if their code is 100, if so i have to proceed with previous previous row till it gets a 200 value, after that it finds the time from the following row of 200 hundred till now (value 100).
ID xID createdDate CODE
1 '1', '2019-07-27 11:52:01', '100'
2 '1', '2019-07-27 11:54:01', '200'
3 '2', '2019-09-03 05:10:02', '200'
4 '2', '2019-09-03 05:12:02', '200'
5 '3', '2019-09-02 05:12:02', '200'
6 '3', '2019-09-02 05:12:02', '100'
7 '3', '2019-09-02 05:12:02', '200'
8 '4', '2019-09-02 05:13:02', '200'
9 '5', '2019-09-03 05:10:03', '200'
10 '6', '2018-12-13 05:03:02', '200'
So this query must for each group of xID find the total time for which the system has been with code 100 until now. Hope I've been clear. And here is the sql so far.
select id, createdDate, code
from wlogs
where id in (
select max(id)
from wlogs
group by xid
)
order by xid;
EDIT:
MYSQL VERSION 8.0
RESULT must be something like this where the column totTimeWithCode100 must show the time in seconds or minutes doesn't matter, for each type of xID.
xID totTimeWithCode100
'1', '500'
'2', '2'
'3', '33'
'4', '200'
'5', '40'
'6', '200'
These rows
Prior to MySQL 8.0, we can use of user-defined variables (in a way that is unsupported) in carefully crafted SQL that takes advantage of behavior that is repeatably observed but not guaranteed. (The MySQL Reference Manual warns specifically about this usage of user-defined variables.)
Something like this:
SELECT s.xid AS `xID`
, IFNULL(SUM(s.secs_in_code100),0) AS `totTimeWithCode100`
FROM (
SELECT IF(#prev_xid = t.xid AND #prev_code = 100, TIMESTAMPDIFF(SECOND,#prev_date,t.createddate),0) AS secs_in_code100
, #prev_xid := t.xid AS xid
, #prev_date := t.createddate AS createddate
, #prev_code := t.code AS code
FROM ( SELECT #prev_xid := ''
, #prev_date := '1970-01-02 03:00'
, #prev_code := ''
) i
CROSS
JOIN wlogs t
ORDER
BY t.xid
, t.createddate
) s
GROUP BY s.xid
ORDER BY s.xid
With MySQL 8.0, we can avoid the user-defined variables by using analytic/window functions.
You can get the result you want by finding all the rows with CODE = 100 for a given xID that are immediately followed (in time) by a row with CODE != 100. This can be done by LEFT JOINing the rows with CODE != 100 to the preceding row if that row has CODE = 100:
SELECT w.xID, COALESCE(SUM(TIMESTAMPDIFF(SECOND, w1.createdDate, w2.createdDate)), 0) AS totTimeWithCode100
FROM (SELECT DISTINCT xID FROM wlogs) w
LEFT JOIN (SELECT *
FROM wlogs
WHERE CODE = 100) w1 ON w1.xID = w.xID
LEFT JOIN wlogs w2 ON w2.xID = w1.xID
AND w2.createdDate = (SELECT MIN(createdDate)
FROM wlogs w3
WHERE w3.xID = w1.xID AND
w3.createdDate > w1.createdDate)
GROUP BY w.xID
ORDER BY w.xID;
Demo on dbfiddle

converting to pivot table

Edit : Raymond Nijland has suggested and issue was at my end . i put extra group by in end
, right syntax is
SELECT
bkg.status,
count( if( bkg.status = '1', bkg.bookingId, 0 ) ) AS 'confirmed',
count( if( bkg.status = '2', bkg.bookingId, 0 ) ) AS 'Tentative'
FROM
tbl_booking bkg
thanks
I have table data for a table like this
i have a query
select bkg.status,
count(bkg.status) from tbl_booking bkg
GROUP BY
bkg.status;
that is giving results like
I want to convert that into pivot format
like
1 | 2 | 3 | 4
21| 57| 7 | 14
I tried
SELECT
bkg.status,
count( if( bkg.status = '1', bkg.bookingId, 0 ) ) AS 'confirmed',
count( if( bkg.status = '2', bkg.bookingId, 0 ) ) AS 'Tentative'
FROM
tbl_booking bkg
GROUP BY
bkg.status;
but that is giving me results like
If I tried to group by diffrent column like booking Id , result gone more different . I am getting confused about which column to be placed in sum and which column to be placed in group by
please suggest

Get sub total of two column for particular criterion and another total as full group by X parameter

I want to group by keyword name, get sum of cf1+cf2 (where bug_status=CLOSED or RESOLVED) and get total sum (irrespective of bug status). Output will have 3 columns like mentioned.
Tried query but no luck:
SELECT keyworddefs.name as keyword, IFNULL(SUM(bugs.cf1 + bugs.cf2),0) as completed, (SELECT IFNULL(SUM(bugs.cf1 + bugs.cf2) ,0) FROM bugs, keywords, keyworddefs WHERE (keywords.bug_id = bugs .bug_id) AND (keyworddefs.id=keywords.keywordid) AND (keyworddefs.name LIKE 'K%')) as total FROM bugs, keywords, keyworddefs WHERE (keywords.bug_id = bugs .bug_id) AND (keyworddefs.id=keywords.keywordid) AND (bugs.bug_status = 'VERIFIED' OR bugs.bug_status = 'CLOSED') GROUP BY keyworddefs.name DESC;
Here's the query formatted.
SELECT keyworddefs.name as keyword,
IFNULL(SUM(bugs.cf1 + bugs.cf2),0) as completed,
(SELECT IFNULL(SUM(bugs.cf1 + bugs.cf2) ,0)
FROM bugs, keywords, keyworddefs
WHERE (keywords.bug_id = bugs .bug_id)
AND (keyworddefs.id=keywords.keywordid)
AND (keyworddefs.name LIKE 'K%')) as total
FROM bugs, keywords, keyworddefs
WHERE (keywords.bug_id = bugs .bug_id)
AND (keyworddefs.id=keywords.keywordid)
AND (bugs.bug_status = 'VERIFIED' OR bugs.bug_status = 'CLOSED')
GROUP BY keyworddefs.name DESC;
SQL Fiddle:
http://sqlfiddle.com/#!9/a11b4/7
Expected:
Matching records:
cf1+cf2 bugid, keyword bug_status
5 (102, 'K1') CLOSED
3 (565, 'K2') CLOSED
3 (1352, 'K1') VERIFIED
4 (13634, 'K1') NEW
# Query output should be:
keyword completed total
K1 8 12
K2 3 3
DDLs:
bugs table1 (master table) :
CREATE TABLE `bugs` (
`bug_id` int(11) NOT NULL,
`bug_date` date NOT NULL,
`cf1` int(11) NOT NULL,
`cf2` int(11) NOT NULL,
`bug_status` varchar(200) NOT NULL)
ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `bugs` (`bug_id`, `bug_date`, `cf1`, `cf2`, `bug_status`) VALUES
(102, '2016-07-19', 2, 1, 'CLOSED'),
(72123, '2016-07-19', 2, 1, 'VERIFIED'),
(57234, '2016-07-19', 2, 1, 'VERIFIED'),
(1352, '2016-07-19', 2, 1, 'VERIFIED'),
(565, '2016-07-19', 2, 1, 'CLOSED'),
(13634, '2016-07-22', 2, 2, 'NEW');
keywords table2 (having keyword ids):
CREATE TABLE `keywords` (
`bug_id` int(11) NOT NULL,
`keywordid` varchar(11) NOT NULL)
ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `keywords` (`bug_id`, `keywordid`) VALUES
(102, '3'),
(565, '4'),
(398, '1'),
(565, '2'),
(1352, '1'),
(57234, '2'),
(1363, '1'),
(72123, '2'),
(13634, '3');
keyworddefs table3 (having keyword names according to keywordid):
CREATE TABLE `keyworddefs` (
`id` int(11) NOT NULL,
`name` varchar(200) NOT NULL,
`description` varchar(200) NOT NULL)
ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `keyworddefs` (`id`, `name`, `description`) VALUES
(1, 'J1', 'My J1 item'),
(2, 'J2', 'My J2 item'),
(3, 'K1', 'My K1 item'),
(4, 'K2', 'My K2 item');
How can I get output like expected?
It looks to me like you're making this too complicated.
For one thing, you should join your keywords and keyworddefs tables ON keywords.keywordid = keyworddefs.name. You're using keyworddefs.id. That's a number. So, your old-timey early 1990s comma join yields no results.
For another thing, you don't need to join the keyworddefs table to get your result.
SUM() operations rarely yield NULL results. So, you should put your conditionals inside the parentheses of SUM() rather than outside.
Finally, you need a GROUP BY operation with two SUM() aggregates in it. One should be conditioned on the bug_status and the other should not.
http://sqlfiddle.com/#!9/a11b4/11/0
Something like this should work.
SELECT keywords.keywordid,
SUM(CASE WHEN bugs.bug_status IN ('CLOSED', 'RESOLVED')
THEN bugs.cf1 + bugs.cf2
ELSE 0 END) completed,
SUM(bugs.cf1 + bugs.cf2) total
FROM bugs
JOIN keywords ON bugs.bug_id = keywords.bug_id
GROUP BY keywords.keywordid
ORDER BY keywords.keywordid
If you need to filter your results by keywords.keywordid LIKE 'K%', you can just add a where clause.
Extended query from Ollie's comment, it works fine with couple of changes.
Highly appreciated Ollie!
SELECT keyworddefs.name,
SUM(CASE WHEN bugs.bug_status IN ('CLOSED', 'VERIFIED') THEN bugs.cf1 + bugs.cf2 ELSE 0 END) completed,
SUM(bugs.cf1 + bugs.cf2) total
FROM bugs
JOIN keywords ON bugs.bug_id = keywords.bug_id
JOIN keyworddefs ON keyworddefs.id = keywords.keywordid
WHERE keyworddefs.name LIKE 'K%'
GROUP BY keywords.keywordid
ORDER BY keyworddefs.name DESC;

Select change column value if in list

I am trying to query my table to count the number of votes and if the voting method is in list ['C', 'M', 'S', 'L', 'T', 'V', 'B', 'E'] then count it as one and replace the voting_method to 'L'.
Right now I have the following query which returns the right results but doesn't take care of the duplicates.
select `election_lbl`, `voting_method`, count(*) as numVotes
from `gen2014` group by `election_lbl`, `voting_method` order by `election_lbl` asc
election_lbl voting_method numVotes
2014-09-04 M 1
2014-09-05 M 2
2014-09-05 S 1
2014-09-08 C 16
2014-09-08 M 5
2014-09-08 S 9
2014-09-09 10 5
2014-09-09 C 46
2014-09-09 M 4
2014-09-09 S 5
2014-09-10 C 92
2014-0g-10 M 3
2014-09-10 S 7
2014-09-11 C 96
2014-09-11 M 3
2014-09-11 S 2
2014-09-12 C 104
2014-09-12 M 10
2014-09-12 S 3
2014-09-15 C 243
2014-09-15 M 18
2014-09-15 S 3
2014-09-16 10 1
2014-09-16 C 161
2014-09-16 M 4
2014-09-16 S 3
2014-09-17 C 157
2014-09-17 M 5
2014-09-17 S 12
You can see that for 2014-09-05 I have two voting_method M and S both of which is in the list. I want the ideal result to remove the duplicate date field if the values are in the list. So it would be 2014-09-05 'L' 3. I don't want the vote for that date to disappear so the results should count them as one.
Changed the query to this but mysql says wrong syntax.
select `election_lbl`, `voting_method`, count(*) as numVotes from `gen2014`
(case `voting_method` when in ('C', 'M', 'S', 'L', 'T', 'V', 'B', 'E')
then 'L' END) group by `election_lbl`, `voting_method` order by `election_lbl` asc
Table Schema
CREATE TABLE `gen2014` (
`voting_method` varchar(255) DEFAULT NULL,
`election_lbl` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
SELECT election_lbl
, CASE WHEN voting_method IN ('C','M','S','L','T','V','B','E')
THEN 'L'
ELSE voting_method END my_voting_method
, COUNT(*)
FROM my_table
GROUP
BY my_voting_method -- or vice
, election_lbl; -- versa
If you just want the total votes using those methods for each date, listed as method 'L', then do not include method in the group by, and have the SELECT select 'L' as voting_method
select `election_lbl`, 'L' AS `voting_method`, count(*) as numVotes
from `gen2014`
where voting_method IN ('C', 'M', 'S', 'L', 'T', 'V', 'B', 'E')
group by `election_lbl`
order by `election_lbl` asc
select x.`election_lbl`, x.`voting_method`, count(*) as numVotes
from (
select `election_lbl`,
CASE when `voting_method` in ('C', 'M', 'S', 'L', 'T', 'V', 'B', 'E')
then 'L'
else `voting_method`
END as `voting_method`
from `gen2014`) x
group by x.`election_lbl`, x.`voting_method`
order by x.`election_lbl` asc

MySQL multidimensional select from views

I would like to display data re-arranged year by year and one of the possible solution is using views and select from them. The data matrix is something like (of course it's a ficticious demo dataset):
USA 2005 22 156
CAN 2005 14 101
MEX 2005 5 32
USA 2006 24 160
CAN 2006 16 103
USA 2007 26 163
MEX 2007 8 35
The SQL code to create and populate the table is:
DROP TABLE IF EXISTS `tab1`;<br>
CREATE TABLE `tab1` ( <br>
`id1` int(4) unsigned NOT NULL AUTO_INCREMENT,
`iso3` char(3) NOT NULL,
`year` int(4) unsigned NOT NULL,
`aaa` int(10) DEFAULT NULL,
`bbb` int(10) DEFAULT NULL,
PRIMARY KEY (`id1`)
)
INSERT INTO `tab1` VALUES
('1', 'USA', '2005', '22', '156'),
('2', 'CAN', '2005', '14', '101'),
('3', 'MEX', '2005', '5', '32'),
('4', 'USA', '2006', '24', '160'),
('5', 'CAN', '2006', '16', '103'),
('6', 'USA', '2007', '26', '163'),
('7', 'MEX', '2007', '8', '35');
COMMIT;
And now I would like to obtain for parameter 'aaa' a 2D table like this:
country 2005 2006 2007
USA 22 24 26
CAN 14 16
MEX 5 8
However the following SQL code is omitting all the lines with missing data, be it one single value and I only get one line
USA 22 24 26
The SQL code is:
SELECT view2005.Country, view2005.2005, view2006.2006, view2007.2007
FROM view2005, view2006, view2007
WHERE view2005.country = view2006.country
AND view2005.country = view2007.country
Any idea how to do it including lines with missing data? Thanks in advance.
Use left joins, and a view (or table, or inner select like below) which has all distinct countries:
SELECT c.country, view2005.2005, view2006.2006, view2007.2007
FROM (SELECT DISTINCT country FROM tab1) as c
LEFT JOIN view2005 ON view2005.country = c.country
LEFT JOIN view2006 ON view2006.country = c.country
LEFT JOIN view2007 ON view2007.country = c.country
GROUP BY c.country
EDIT:
In a more general context, what you are asking here is to create a pivot of this table, which is a common problem that has common solutions. Here is a nice "How To": http://www.artfulsoftware.com/infotree/queries.php?&bw=1339#78
It's better to use JOIN than implicit JOIN with WHERE. Additional advanatge is that you can convert it to a LEFT JOIN so data for 2005 that don't have a 2006 related row (and are not matched) will still be shown.
Use Galz's solution or search as correctly advised for how to create PIVOT queries.
One such logic to create a pivot query would be:
SELECT iso3 AS Country
, SUM(IF(year=2005, aaa, 0)) AS 2005
, SUM(IF(year=2006, aaa, 0)) AS 2006
, SUM(IF(year=2007, aaa, 0)) AS 2007
FROM tab1 AS t
GROUP BY iso3
If there are years without any data, you will get NULL in that column.
You can use COALESCE() function if you want 0 to be shown and not NULL:
SELECT iso3 AS Country
, COALESCE( SUM( IF(year=2004, aaa, 0) ) , 0) AS "2004"
, COALESCE( SUM( IF(year=2005, aaa, 0) ) , 0) AS "2005"
, COALESCE( SUM( IF(year=2006, aaa, 0) ) , 0) AS "2006"
, COALESCE( SUM( IF(year=2007, aaa, 0) ) , 0) AS "2007"
FROM tab1 AS t
GROUP BY iso3
Thank you Galz for the link to pivots and thank you ypercube for the SQL. It worked after enclosing the years into quotes to make them CHAR.
I was further intrigued by the question what happens if I add a row with no values at all or a row out of range of the years so I have added
INSERT INTO `tab1` VALUES
('7', 'ATA', '2004', '', '')
The result was that I got a mix of NULL and INT zero values. This is not good because zero is a valid number and legitimate data. So I have modified the query to get exactly the result I need:
SELECT iso3 AS countryб
SUM( IF(year=2004, aaa, NULL) ) AS "2004",
SUM( IF(year=2005, aaa, NULL) ) AS "2005",
SUM( IF(year=2006, aaa, NULL) ) AS "2006",
SUM( IF(year=2007, aaa, NULL) ) AS "2007"
FROM tab1
GROUP BY iso3