Hello I have one table in database which contains multiple columns , more than 20 and it contains many records about 2 millions
The table has one primary key : Col1 and it is also the index for my table
, For some reasons , the data type for that column is varchar(200).
I am implementing server side paging ,sorting and later (filtering)
I face a problem in performance , here is my procedure
DELIMITER $$
CREATE DEFINER=`master`#`%` PROCEDURE `spGetData`(
IN DisplayStart int ,
IN SortCol int ,
IN SortDir nvarchar(10),
IN Search nvarchar(255)
)
BEGIN
DECLARE FirstRec int;
DECLARE LastRec int;
SET FirstRec = DisplayStart;
select
col1,col2,col3,col4,col6,col7,col8,col9,col10,col11
col12,col13,col14,col15,col16,col17,col18,col19,col20
col21,
( select count(*) from myTable) as filtered
from myTable
order by
case When (#SortCol = 0 and #SortDir = 'asc') then col1 end asc ,
case When (#SortCol = 0 and #SortDir = 'desc') then col1 end desc ,
case When (#SortCol = 1 and #SortDir = 'asc') then col2 end asc ,
case When (#SortCol = 1 and #SortDir = 'desc') then col2 end desc ,
case When (#SortCol = 2 and #SortDir = 'asc') then col3 end asc ,
case When (#SortCol = 2 and #SortDir = 'desc') then col3 end desc ,
case When (#SortCol = 3 and #SortDir = 'asc') then col4 end asc ,
case When (#SortCol = 3 and #SortDir = 'desc') then col4 end desc ,
case When (#SortCol = 4 and #SortDir = 'asc') then col5 end asc ,
case When (#SortCol = 4 and #SortDir = 'desc') then col5 end desc ,
case When (#SortCol = 5 and #SortDir = 'asc') then col6 end asc ,
case When (#SortCol = 5 and #SortDir = 'desc') then col6 end desc ,
case When (#SortCol = 6 and #SortDir = 'asc') then col7 end asc ,
case When (#SortCol = 6 and #SortDir = 'desc') then col7 end desc ,
case When (#SortCol = 7 and #SortDir = 'asc') then col8 end asc ,
case When (#SortCol = 7 and #SortDir = 'desc') then col8 end desc ,
case When (#SortCol = 8 and #SortDir = 'asc') then col9 end asc ,
case When (#SortCol = 8 and #SortDir = 'desc') then col9 end desc ,
case When (#SortCol = 9 and #SortDir = 'asc') then col10 end asc ,
case When (#SortCol = 9 and #SortDir = 'desc') then col10 end desc ,
case When (#SortCol = 10 and #SortDir = 'asc') then col11 end asc ,
case When (#SortCol = 10 and #SortDir = 'desc') then col11 end desc ,
case When (#SortCol = 11 and #SortDir = 'asc') then col12 end asc ,
case When (#SortCol = 11 and #SortDir = 'desc') then col12 end desc ,
case When (#SortCol = 12 and #SortDir = 'asc') then col13 end asc ,
case When (#SortCol = 12 and #SortDir = 'desc') then col13 end desc ,
case When (#SortCol = 13 and #SortDir = 'asc') then col14 end asc ,
case When (#SortCol = 13 and #SortDir = 'desc') then col14 end desc ,
case When (#SortCol = 14 and #SortDir = 'asc') then col15 end asc ,
case When (#SortCol = 14 and #SortDir = 'desc') then col15 end desc ,
case When (#SortCol = 15 and #SortDir = 'asc') then col16 end asc ,
case When (#SortCol = 15 and #SortDir = 'desc') then col16 end desc ,
case When (#SortCol = 16 and #SortDir = 'asc') then col17 end asc ,
case When (#SortCol = 16 and #SortDir = 'desc') then col17 end desc ,
case When (#SortCol = 17 and #SortDir = 'asc') then col18 end asc ,
case When (#SortCol = 17 and #SortDir = 'desc') then col18 end desc ,
case When (#SortCol = 18 and #SortDir = 'asc') then col19 end asc ,
case When (#SortCol = 18 and #SortDir = 'desc') then col19 end desc ,
case When (#SortCol = 19 and #SortDir = 'asc') then col20 end asc ,
case When (#SortCol = 19 and #SortDir = 'desc') then col20 end desc ,
case When (#SortCol = 20 and #SortDir = 'asc') then col21 end asc ,
case When (#SortCol = 20 and #SortDir = 'desc') then col21 end desc ,
limit FirstRec,10;
END
The query is very slow and it gives a buffer size problem.
If i remove the order by clause , it becomes very fast.
So my questions are
1- How can i enhance this query and make sorting for millions of rows act fast?
2- Later i will apply filtering using a where clause for multiple columns , how can i avoid any performance issue?
The ORDER BY should be constructed (concat, prepare, execute, deallocate) instead of having that huge expression that cannot use any index.
It is possible, but not realistic, to add INDEX(col0), INDEX(col1), ... so that the prepared statement will be fast in all 21 cases. Pick the important ones and don't provide the rest to the end-user.
Don't use OFFSET to paginate; instead 'remember where you left off'.
Where's the filtering? It is likely to interfere with the tips I have already given you.
VARCHAR(200) is usually unwise for the PRIMARY KEY.
Will the user be able to specify sorting on two columns? Filtering on date ranges? Other things? (Please provide the real SHOW CREATE TABLE if you want to get into more details.)
Have you 'normalized' any columns? That is, move large, frequently repeated, values into other tables.
Related
I have a mysql query that is returning the following data:
select Desc,Value from table;
Desc Value
a 4
r 3
e 4
j 7
w 6
p 6
I want to order by Descr but in a specific order (This specific is required for json)
Desc Value
r 3
a 4
e 4
j 7
p 6
w 6
Is this possible?
ORDER BY FIELD(`desc`, 'r', 'a', 'e', 'j', 'p', 'w')
Note: if desc value doesn't match the given values, it will come first, as FIELD() would return 0
It's not pretty, but you can order by (condition) DESC. DESC because if the condition is true it's more than if the condition is false:
SELECT
Desc,
Value
FROM
Foo
ORDER BY
(Desc = 'r') DESC,
(Desc = 'a') DESC,
(Desc = 'e') DESC,
(Desc = 'j') DESC,
(Desc = 'p') DESC,
(Desc = 'w') DESC
You could order by a case expression that specifies the ordering you want:
SELECT `desc`, `value`
FROM my_table
ORDER BY CASE `desc` WHEN 'r' THEN 1
WHEN 'a' THEN 2
WHEN 'e' THEN 3
WHEN 'j' THEN 4
WHEN 'p' THEN 5
WHEN 'w' THEN 6
ELSE 999 --for completeness sake
END CASE ASC
I'm stacked on following query. I need to create an if statement or something similar to distinguish which "CASE" should be added to my select query.
If page_type = 'Region' or 'City' add this to the query:
( CASE When landing_page like concat('%',path_id,'%') and
(LENGTH(landing_page) - LENGTH(REPLACE(landing_page, '/' ,'')) / LENGTH('/')) = 2
Then 'True' Else 'False' End as 'correct lp' , )
Elseif page_type = 'Item' add this to the query:
( CASE When landing_page like concat('%',path_id,'%') and
(LENGTH(landing_page) - LENGTH(REPLACE(landing_page, '/' ,'')) / LENGTH('/')) = 3
Then 'True' Else 'False' End as 'correct lp' , )
Elseif page_type = 'POI (Alle)' add this to the query:
( CASE When landing_page like concat('%',path_id,'%') and
(LENGTH(landing_page) - LENGTH(REPLACE(landing_page, '/' ,'')) / LENGTH('/')) = 3 and
'Hotel' NOT IN landing_page
Then 'True' Else 'False' End as 'correct lp' , )
Case queries should be added to the following query according to if outcome:
SELECT
MONTHNAME(k.crawl_date) As 'Month',
YEAR(k.crawl_date) As 'Year',
WEEKOFYEAR(k.crawl_date) As 'Week',
k.landing_page,
k.pos,
CASE WHEN k.pos = 0 THEN 'No Rank' WHEN k.pos < 4 THEN 'Top 3' WHEN k.pos < 11 THEN 'Page 1'
WHEN k.pos < 21 THEN 'Page 2' WHEN k.pos < 31 THEN 'Page 3' WHEN k.pos < 41 THEN 'Page 4'
WHEN k.pos < 51 THEN 'Page 5' WHEN k.pos < 61 THEN 'Page 6' WHEN k.pos < 71 THEN 'Page 7'
WHEN k.pos < 81 THEN 'Page 8' WHEN k.pos < 91 THEN 'Page 9' WHEN k.pos < 101 THEN 'Page 10'
WHEN k.pos < 111 THEN 'Page 11' END as 'Page',
k.path_id,
k.domain,
x.keyword,
x.engine_id,
l.bucket,
l.page_type,
l.item_label
FROM
xovi_stats k
LEFT JOIN xovi_keywords x ON k.keyword_id = x.id
LEFT JOIN xovi_labels l ON l.id = x.label_id
You can nest CASE clauses like this:
SELECT
CASE
WHEN (page_type IN ('Region', 'City')) THEN
CASE
WHEN landing_page LIKE ... THEN 'True'
ELSE 'False'
END
WHEN (page_type = 'Item') THEN
CASE
...
END
WHEN (page_type = 'POI (Alle)') THEN
CASE
...
END
END AS lp,
MONTHNAME(k.crawl_date) As 'Month', -- other fields
...
Or you can also nest IF(), as you suggest:
SELECT
IF ( (page_type IN ('Region', 'City')),
-- if (page_type IN ('Region', 'City')) then
CASE
WHEN landing_page LIKE ... THEN 'True'
ELSE 'False'
END,
-- else
IF ( (page_type = 'Item'),
-- if (page_type = 'Item') then
CASE
...
END,
-- else
IF ( page_type = 'POI (Alle)',
-- if (page_type = 'POI (Alle)') then
CASE
...
END,
-- else
NULL
)
)
) AS lp,
MONTHNAME(k.crawl_date) As 'Month', -- other fields
...
Off-topic notes:
you can replace your two-branch CASE statements with simple IF():
SELECT CASE
WHEN landing_page LIKE ... THEN 'True'
ELSE 'False'
END AS alias ;
-- strictly equivalent to
SELECT IF(
landing_page LIKE ...,
'True',
'False'
) AS alias ;
you can replace your large existing CASE with:
CASE
WHEN k.pos = 0 THEN 'No Rank'
WHEN k.pos < 4 THEN 'Top 3'
ELSE CONCAT('Page ', (k.pos -1) DIV 10 +1 )
END AS 'Page'
I have a query but I want to order the data as the following:
status (so I have the records with status = 1 on the top and status 2 at the bottom)
if the records have status = 1 then order them as the following
a) CASE WHEN i.assigned_to = '.USER_ID.' THEN 0 ELSE 1 END
b) CASE WHEN (i.approved_by > 0 OR i.approved_on IS NOT NULL) THEN 0 ELSE 1 END
c) i.priority DESC
d) i.created_on ASC
if the records have status = 2 then order the records by completed_on DESC
This is my current syntax but I can't figure out how to split the order
ORDER BY
i.status ASC,
CASE WHEN i.assigned_to = '.USER_ID.' THEN 0 ELSE 1 END,
CASE WHEN (i.approved_by > 0 OR i.approved_on IS NOT NULL) THEN 0 ELSE 1 END,
i.priority DESC,
i.created_on ASC
My query currently order all the records by
a) CASE WHEN i.assigned_to = '.USER_ID.' THEN 0 ELSE 1 END
b) (CASE WHEN (i.approved_by > 0 OR i.approved_on IS NOT NULL) THEN 0 ELSE 1 END)
c) i.priority DESC
d) i.created_on ASC
and I want it to order by those only if the status = 1 otherwise order by completed_on DESC
Try:
ORDER BY
i.status ASC,
CASE WHEN i.assigned_to = '.USER_ID.' and i.status = 1 THEN 0 ELSE 1 END,
CASE WHEN (i.approved_by > 0 OR i.approved_on IS NOT NULL) and i.status = 1
THEN 0 ELSE 1 END,
CASE WHEN i.status = 1 THEN i.priority END DESC,
CASE WHEN i.status = 1 THEN i.created_on END ASC,
CASE WHEN i.status = 2 THEN i.completed_on END DESC
I want to have my table,rcarddet, ordered by "SDNO" (not primary key) in ascending order with the exception of "0". So it should turn out to be like:
1
1
2
.
.
10
0
0
My query now is:
SELECT *
FROM `rcarddet`
WHERE `RDATE` = '2011-05-25'
AND `RCNO` = '1'
AND `PLACE` = 'H'
AND `SDNO` != 0
ORDER BY `rcarddet`.`SDNO` ASC;
The easiest way
SELECT * FROM rcarddet
WHERE RDATE = '2011-05-25' and RCNO = '1'and PLACE = 'H'
ORDER BY CASE
WHEN rcarddet.SDNO = 0 THEN [max_number_for_the_type_of_SDNO]
ELSE rcarddet.SDNO
END ASC
SELECT *
FROM `rcarddet`
WHERE `RDATE` = '2011-05-25'
AND `RCNO` = '1'
AND `PLACE` = 'H'
ORDER BY
`SDNO` = 0,
`SDNO`;
I got a stored procedure that delivers a table of students, and it needs to order by surname, name etc... it also needs to sort ascending, descending, depending on the parameter #orderby...
code:
ORDER BY
CASE
WHEN #orderby = 'studentkey' THEN Studentkey END ASC,
CASE
WHEN #orderby = 'studentkey' and #desc = 1 THEN Studentkey END DESC,
CASE
WHEN #orderby = 'initials' THEN Initials END ASC,
CASE
WHEN #orderby = 'initials' and #desc = 1 THEN Initials END DESC,
CASE
WHEN #orderby = 'firstname' THEN Firstname END ASC,
CASE
WHEN #orderby = 'firstname' and #desc = 1 THEN Firstname END DESC,
CASE
WHEN #orderby = 'nickname' THEN Nickname END ASC,
CASE
WHEN #orderby = 'nickname' and #desc = 1 THEN Nickname END DESC,
CASE
WHEN #orderby = 'insertion' THEN Insertion END ASC,
CASE
WHEN #orderby = 'insertion' and #desc = 1 THEN Insertion END DESC,
CASE
WHEN #orderby = 'surname' THEN Surname END ASC,
CASE
WHEN #orderby = 'surname' and #desc = 1 THEN Surname END DESC
NED
There is a difference in output between #desc = 1 and #desc = 0, but not what i desire...
Does anyone have a solution?
Try this:
CASE
WHEN #orderby = 'studentkey' and #desc = 0 THEN Studentkey END ASC,
CASE
WHEN #orderby = 'studentkey' and #desc = 1 THEN Studentkey END DESC,
...