Why does STRAIGHT_JOIN consume more CPU? - mysql

Why does STRAIGHT_JOIN consume more CPU than a regular join? Do you have any idea?
When i use straight_join on one of my queries, it speeds up the query like from 12 seconds to 3 seconds. But it consumes so much CPU? Might it be about server configuration or something else?
You might want to check the code after this comment / Topic Ids are OK, Getting Devices... /
Before this line there are some code about filling topic_ids to a temp table.
Here is the query:
CREATE PROCEDURE `DevicesByTopic`(IN platform TINYINT, IN application TINYINT, IN topicList TEXT, IN page_no MEDIUMINT UNSIGNED)
BEGIN
DECLARE m_index INT DEFAULT 0;
DECLARE m_topic VARCHAR(255);
DECLARE m_topic_id BIGINT UNSIGNED DEFAULT NULL;
DECLARE m_session_id VARCHAR(40) CHARSET utf8 COLLATE utf8_turkish_ci;
-- Session Id
SET m_session_id = replace(uuid(), '-', '');
-- Temp table
CREATE TEMPORARY TABLE IF NOT EXISTS tmp_topics(
topic_slug VARCHAR(100) COLLATE utf8_turkish_ci
,topic_id BIGINT UNSIGNED
,session_id VARCHAR(40) COLLATE utf8_turkish_ci
,INDEX idx_tmp_topic_session_id (session_id)
,INDEX idx_tmp_topic_id (topic_id)
) CHARSET=utf8 COLLATE=utf8_turkish_ci;
-- Filling topics in a loop
loop_topics: LOOP
SET m_index = m_index + 1;
SET m_topic_id = NULL;
SET m_topic= SPLIT_STR(topicList,',', m_index);
IF m_topic = '' THEN
LEAVE loop_topics;
END IF;
SELECT t.topic_id INTO m_topic_id FROM topic AS t WHERE t.application = application AND (t.slug_hashed = UNHEX(MD5(m_topic)) AND t.slug = m_topic) LIMIT 1;
-- Fill temp table
IF m_topic_id IS NOT NULL AND m_topic_id > 0 THEN
INSERT INTO tmp_topics
(topic_slug, topic_id, session_id)
VALUES
(m_topic, m_topic_id, m_session_id);
END IF;
END LOOP loop_topics;
/* Topic Ids are OK, Getting Devices... */
SELECT
dr.device_id, dr.platform, dr.application, dr.unique_device_id, dr.amazon_arn
FROM
device AS dr
INNER JOIN (
SELECT STRAIGHT_JOIN
DISTINCT
d.device_id
FROM
device AS d
INNER JOIN
device_user AS du ON du.device_id = d.device_id
INNER JOIN
topic_device_user AS tdu ON tdu.device_user_id = du.device_user_id
INNER JOIN
tmp_topics AS tmp_t ON tmp_t.topic_id = tdu.topic_id
WHERE
((platform IS NULL OR d.platform = platform) AND d.application = application)
AND d.page_no = page_no
AND d.status = 1
AND du.status = 1
AND tmp_t.session_id = m_session_id COLLATE utf8_turkish_ci
) dFiltered ON dFiltered.device_id = dr.device_id
WHERE
((platform IS NULL OR dr.platform = platform) AND dr.application = application)
AND dr.page_no = page_no
AND dr.status = 1;
-- Delete rows fFill temp table
DELETE FROM tmp_topics WHERE session_id = m_session_id;
END;
With the STRAIGHT_JOIN this query takes about 3 seconds but consumes so much CPU like 90%, but if i remove the keyword "STRAIGHT_JOIN", it takes 12 seconds but consume 12% CPU.
MySQL 5.6.19a - innodb
What might be the reason?
Best regards.

A STRAIGHT_JOIN is used when you need to override MySQL's optimizer. You are telling it to ignore its own optimized execution path and instead rely on reading the tables in the order you have written them in the query.
99% of the time you don't want to use a straight_join. Just rely on MySQL to do its job and optimize the execution path for you. After all, any RDBMS worth its salt is going to be pretty decent at optimizing.
The few times you should use a straight_join are when you've already tested MySQL's optimization for a given query and found it lacking. In your case with this query, clearly your manual optimization using straight_join is not better than MySQL's baked in optimization.

Related

Why does MySQL not always use index merge here?

Consider this table:
CREATE TABLE `Alarms` (
`AlarmId` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`DeviceId` BINARY(16) NOT NULL,
`Code` BIGINT(20) UNSIGNED NOT NULL,
`Ended` TINYINT(1) NOT NULL DEFAULT '0',
`NaturalEnd` TINYINT(1) NOT NULL DEFAULT '0',
`Pinned` TINYINT(1) NOT NULL DEFAULT '0',
`Acknowledged` TINYINT(1) NOT NULL DEFAULT '0',
`StartedAt` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
`EndedAt` TIMESTAMP NULL DEFAULT NULL,
`MarkedForDeletion` TINYINT(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`AlarmId`),
KEY `Key1` (`Ended`,`Acknowledged`),
KEY `Key2` (`Pinned`),
KEY `Key3` (`DeviceId`,`Pinned`),
KEY `Key4` (`DeviceId`,`StartedAt`,`EndedAt`),
KEY `Key5` (`DeviceId`,`Ended`,`EndedAt`),
KEY `Key6` (`MarkedForDeletion`)
) ENGINE=INNODB;
And, for this test, populate it like so:
-- Populate some dummy data; 500 alarms for each
-- of 1000 one-second periods
SET #testDevice = UNHEX('00030000000000000000000000000000');
DROP PROCEDURE IF EXISTS `injectAlarms`;
DELIMITER ;;
CREATE PROCEDURE injectAlarms()
BEGIN
SET #fromdate = '2018-02-18 00:00:00';
SET #numdates = 1000;
SET #todate = DATE_ADD(#fromdate, INTERVAL #numdates SECOND);
-- Create table of alarm codes to join on
DROP TABLE IF EXISTS `__codes`;
CREATE TEMPORARY TABLE `__codes` (
`Code` BIGINT NOT NULL PRIMARY KEY
);
SET #startcode = 0;
SET #endcode = 499;
REPEAT
INSERT INTO `__codes` VALUES(#startcode);
SET #startcode = #startcode + 1;
UNTIL #startcode > #endcode END REPEAT;
-- Add an alarm for each code, for each second in range
REPEAT
INSERT INTO `Alarms`
(`DeviceId`, `Code`, `Ended`, `NaturalEnd`, `Pinned`, `Acknowledged`, `StartedAt`, `EndedAt`)
SELECT
#testDevice,
`Code`,
TRUE, FALSE, FALSE, FALSE,
#fromdate, #fromdate
FROM `__codes`;
SET #fromdate = DATE_ADD(#fromdate, INTERVAL 1 SECOND);
UNTIL #fromdate > #todate END REPEAT;
END;;
DELIMITER ;
CALL injectAlarms();
Now, for some datasets the following query works quite well:
SELECT * FROM `Alarms`
WHERE
((`Alarms`.`Ended` = FALSE AND `Alarms`.`Acknowledged` = FALSE) OR `Alarms`.`Pinned` = TRUE) AND
`MarkedForDeletion` = FALSE AND
`DeviceId` = #testDevice
;
This is because MariaDB is clever enough to use index merges, e.g.:
id select_type table type possible_keys
1 SIMPLE Alarms index_merge Key1,Key2,Key3,Key4,Key5,Key6
key key_len ref rows Extra
Key1,Key2,Key3 2,1,17 (NULL) 2 Using union(Key1,intersect(Key2,Key3)); Using where
However if I use the dataset as populated by the procedure above, and flip the query around a bit (which is another view I need, but in this case will return many more rows):
SELECT * FROM `Alarms`
WHERE
((`Alarms`.`Ended` = TRUE OR `Alarms`.`Acknowledged` = TRUE) AND `Alarms`.`Pinned` = FALSE) AND
`MarkedForDeletion` = FALSE AND
`DeviceId` = #testDevice
;
… it doesn't:
id select_type table type possible_keys
1 SIMPLE Alarms ref Key1,Key2,Key3,Key4,Key5,Key6
key key_len ref rows Extra
Key2 1 const 144706 Using where
I would rather like the index merges to happen more often. As it is, given the ref=const, this query plan doesn't look too scary … however, the query takes almost a second to run. That in itself isn't the end of the world, but the poorly-scaling nature of my design shows when trying a more exotic query, which takes a very long time:
-- Create a temporary table that we'll join against in a mo
DROP TABLE IF EXISTS `_ranges`;
CREATE TEMPORARY TABLE `_ranges` (
`Start` TIMESTAMP NOT NULL DEFAULT 0,
`End` TIMESTAMP NOT NULL DEFAULT 0,
PRIMARY KEY(`Start`, `End`)
);
-- Populate it (in reality this is performed by my application layer)
SET #endtime = 1518992216;
SET #starttime = #endtime - 86400;
SET #inter = 900;
DROP PROCEDURE IF EXISTS `populateRanges`;
DELIMITER ;;
CREATE PROCEDURE populateRanges()
BEGIN
REPEAT
INSERT IGNORE INTO `_ranges` VALUES(FROM_UNIXTIME(#starttime),FROM_UNIXTIME(#starttime + #inter));
SET #starttime = #starttime + #inter;
UNTIL #starttime > #endtime END REPEAT;
END;;
DELIMITER ;
CALL populateRanges();
-- Actual query
SELECT UNIX_TIMESTAMP(`_ranges`.`Start`) AS `Start_TS`,
COUNT(`Alarms`.`AlarmId`) AS `n`
FROM `_ranges`
LEFT JOIN `Alarms`
ON `Alarms`.`StartedAt` < `_ranges`.`End`
AND (`Alarms`.`EndedAt` IS NULL OR `Alarms`.`EndedAt` >= `_ranges`.`Start`)
AND ((`Alarms`.`EndedAt` IS NULL AND `Alarms`.`Acknowledged` = FALSE) OR `Alarms`.`Pinned` = TRUE)
-- Again, the above condition is sometimes replaced by:
-- AND ((`Alarms`.`EndedAt` IS NOT NULL OR `Alarms`.`Acknowledged` = TRUE) AND `Alarms`.`Pinned` = FALSE)
AND `DeviceId` = #testDevice
AND `MarkedForDeletion` = FALSE
GROUP BY `_ranges`.`Start`
(This query is supposed to gather a list of counts per time slice, each count indicating how many alarms' [StartedAt,EndedAt] range intersects that time slice. The result populates a line graph.)
Again, when I designed these tables and there weren't many rows in them, index merges seemed to make everything whiz along. But now not so: with the dataset as given in injectAlarms(), this takes 40 seconds to complete!
I noticed this when adding the MarkedForDeletion column and performing some of my first large-dataset scale tests. This is why my choice of indexes doesn't make a big deal out of the presence of MarkedForDeletion, though the results described above are the same if I remove AND MarkedForDeletion = FALSE from my queries; however, I've kept the condition in, as ultimately I will need it to be there.
I've tried a few USE INDEX/FORCE INDEX combinations, but it never seems to use index merge as a result.
What indexes can I define to make this table behave quickly in the given cases? Or how can I restructure my queries to achieve the same goal?
(Above query plans obtained on MariaDB 5.5.56/CentOS 7, but solution must also work on MySQL 5.1.73/CentOS 6.)
Wow! That's the most complicated "index merge" I have seen.
Usually (perhaps always), you can make a 'composite' index to replace an index-merge-intersect, and perform better. Change key2 from just (pinned) to (pinned, DeviceId). This may get rid of the 'intersect' and speed it up.
In general, the Optimizer uses index merge only in desperation. (I think this is the answer to the title question.) Any slight changes to the query or the values involved, and the Optimizer will perform the query without index merge.
An improvement on the temp table __codes is to build a permanent table with a large range of values, then use a range of values from that table inside your Proc. If you are using MariaDB, then use the dynamically built "sequence" table. For example the 'table' seq_1_to_100 is effectively a table of one column with numbers 1..100. No need to declare it or populate it.
You can get rid of the other REPEAT loop by computing the time from Code.
Avoiding LOOPs will be the biggest performance benefit.
Get all that done, then I may have other tips.

Speeding up a MySQL insert select into with 10 millions records

I have a few table storing daily orders, customers and salespersons. Yet the schema was not well design as columns have inappropriate data value and type, missing index and partition etc. I re-designed a new schema and populate the new tables with the wrecked tables. I am now stuck on populating the daily orders table (with around 10M records).
Attached data definition and the SQL script to populate the table.
TABLE DEFINITION
CREATE TABLE IF NOT EXISTS `testing`.`Orders` (
`order_ID` INT UNSIGNED NOT NULL AUTO_INCREMENT,
`ord_id` BIGINT UNSIGNED NOT NULL,
`create_time` DATETIME NOT NULL,
`create_date` DATE NOT NULL,
`cust_id` MEDIUMINT UNSIGNED NOT NULL,
`cust_mob` BIGINT UNSIGNED NULL,
`sales_id` MEDIUMINT UNSIGNED NULL,
`sales_mob` BIGINT UNSIGNED NULL,
`sales_flag` TINYINT UNSIGNED NULL,
`comm_flag` TINYINT UNSIGNED NULL,
`extraprice` TINYINT UNSIGNED NULL,
PRIMARY KEY (`order_ID`),
INDEX `Date_cust_id` (`create_date` ASC, `cust_id` ASC),
INDEX `Date_cust_mob` (`create_date` ASC, `cust_mob` ASC),
INDEX `Date_dri_id` (`create_date` ASC, `sales_id` ASC),
INDEX `Date_dri_mob` (`create_date` ASC, `sales_mob` ASC),
INDEX `Date_cust` (`create_date` ASC, `cust_id` ASC, `cstu_mob` ASC),
INDEX `Date_dri` (`create_date` ASC, `sales_id` ASC, `sales_mob` ASC),
INDEX `cust` (`cust_id` ASC, `cust_mob` ASC),
INDEX `dri` (`sales_id` ASC, `sales_mob` ASC),
UNIQUE INDEX `ord_id_UNIQUE` (`ord_id` ASC)
)
ENGINE = InnoDB
DEFAULT CHARACTER SET = utf8;
This script is to populate the table, involving two left join tables: Pag table with 6xx K record and dri table with 3x k record.
SET SQL_SAFE_UPDATES=0;
SET SQL_MODE='';
DROP PROCEDURE IF EXISTS testing.populate_ord1;
DELIMITER $$
CREATE PROCEDURE testing.populate_ord1()
BEGIN
PREPARE stmt
FROM "
INSERT INTO testing.Orders
SELECT
1
,ord_id
,CASE WHEN TRIM(create_time) ='NULL' THEN NULL ELSE STR_TO_DATE(substring(create_time,1,19), '%Y-%m-%d %H:%i:%s') END AS create_time
,CASE WHEN TRIM(create_time) ='NULL' THEN NULL ELSE DATE(STR_TO_DATE(substring(create_time,1,19), '%Y-%m-%d %H:%i:%s')) END AS create_date
,CASE WHEN TRIM(ord.cust_id) = 'NULL' THEN NULL else pag.cust_id END as cust_id
,CASE WHEN TRIM(ord.mob) = 'NULL' THEN NULL else pag.cust_mob END as cust_mob
,CASE WHEN TRIM(ord.sales_id) = 'NULL' THEN NULL else dri.sales_id END as sales_id
,CASE WHEN TRIM(ord.mob1) = 'NULL' THEN NULL else dri.sales_mob END as sales_mob
,CASE WHEN TRIM(sales_flag) ='NULL' THEN NULL ELSE CONVERT(TRIM(sales_flag),UNSIGNED INTEGER) end AS sales_flag
,CASE WHEN TRIM(comm_flag) ='NULL' THEN NULL ELSE CONVERT(TRIM(comm_flag),UNSIGNED INTEGER) end AS comm_flag
,CASE WHEN TRIM(extraprice) ='NULL' THEN NULL ELSE CONVERT(TRIM(extraprice),UNSIGNED INTEGER) end AS extraprice
FROM testing.ord_table ord
LEFT JOIN
(SELECT cust_id,customer_id,cust_mob FROM testing.Passenger) pag
ON TRIM(ord.customer_id) = TRIM(pag.pag_id)
AND TRIM(ord.mob) = TRIM(pag.passenger_mob)
LEFT JOIN
(SELECT sales_id,salesperson_id,sales_mob FROM testing.sales) dri
ON TRIM(ord.salesperson_id) = TRIM(dri.sales_id)
AND TRIM(ord.mob1) = TRIM(dri.sales_mob)
WHERE ord_id != 'NULL' AND create_time IS NOT NULL AND create_time != 'NULL' AND YEAR(create_time) = ? AND MONTH(create_time) = ? AND DAY(create_time) = ?
GROUP BY ord_id
ON DUPLICATE KEY UPDATE ord_id = ord_id
;
";
SET #y = 2014, #m = 9, #d = 1;
WHILE #y<= 2014 DO
WHILE #m<= 12 DO
SET #d = 1;
WHILE #d<= 31 DO
EXECUTE stmt USING #y, #m, #d;
SET #d = #d + 1;
END WHILE;
SET #m = #m + 1;
END WHILE;
SET #y = #y + 1;
SET #m = 1;
END WHILE;
DEALLOCATE PREPARE stmt;
END$$
DELIMITER ;
set autocommit=0;
call testing.populate_ord1();
COMMIT;
I have failed to populate any record to the table. Sometimes it raises lock wait timeout error or data type error or simply takes too long time (2 days) I suspect it is even doing any job.
I searched the web a bit and have added the following settings to my.cnf.
innodb_autoinc_lock_mode = 2
innodb_lock_wait_time_out = 150
innodb_flush_log_at_trx_commit =2
innodb_buffer_pool_size = 14G
Would anyone advise on how I could accomplish the same task efficiently? The code above run without any syntax error. And in case if there is any naming confusion, please let me know if that's critical to get clarified as I am slightly tweaked those variable tables.
Start by performing
UPDATE ... SET
comm_flag = TRIM(comm_flag),
sales_flag = TRIM(sales_flag),
...
That will speed up the subsequent queries some, and simplify them.
Then avoid using LEFT JOIN ( SELECT ... FROM x WHERE ... ). Instead, see if you can turn that into LEFT JOIN x ON ... WHERE .... That is likely to help.
It is usually a bad idea to split a DATE and TIME into two columns. Or do you have a good argument for such? Let's see the queries that touch that pair of columns.
There is no need for STR_TO_DATE() if the string is already a properly formatted DATE or DATETIME. That is, a string works just fine.
Once the TRIM is out of the way, CONVERT(TRIM(comm_flag),UNSIGNED INTEGER) can be simply comm_flag.
Don't loop through things a day at a time -- the way you have it structured, it will be doing a full table scan! About 1000 times !! (This is likely to be the biggest performance issue.)

MySQL / Mariadb Stored Procedure, Prepared Statement, Union, Values from dynamically created tables and column names

I'd like to create reports without having to create a pivot table in excel for every report.
I have survey software that creates a new table for each survey. The columns are named with ID numbers. So, I never know what the columns will be named. The software stores answers in two different tables depending on the 'type' of question. (text, radio button, etc.)
I manually created a table 'survey_answers_lookup' that stores a few key fields but it duplicates the answers. The procedure 'survey_report' works well and produces the required data but there is a challenge.
Since the survey tables are created when someone creates a new survey, I would need a trigger on the schema that creates a second trigger and I don't think that is possible. The second trigger would monitor the survey table and insert the data into the 'survey_answers_lookup' table after someone completes a survey.
I could edit the php software and insert the values into the survey_answers_lookup table but that would create more work when I update the software. (I'd have to update the files and then put my changes back in the files). I also could not determine where they insert the values into the tables.
Can you please help?
Edited. I posted my solution below.
Change some_user to a user who has access to the database.
CREATE DEFINER=`some_user`#`localhost` PROCEDURE `usp_produce_survey_report`(IN survey_id VARCHAR(10), IN lang VARCHAR(2))
SQL SECURITY INVOKER
BEGIN
/*---------------------------------------------------------------------------------
I do not guarantee that this will work for you or that it cannot be hacked with
with SQL injections or other malicious intents.
This stored procedure will produce output that you may use to create a report.
It accepts two arguments; The survey id (745) and the language (en).
It parses the column name in the survey table to get the qid.
It will copy the answers from the survey table to the survey_report
table if the answer is type S or K. It will get the answers from
the answers table for other types. NOTE: Other types might need to
be added to the if statement.
Additionally, the qid and id from the survey table are also copied to
the survey_report table.
Then the questions from the questions table, and answers from the answers
and survey_report tables are combined and displayed.
The data in the survey_report table is deleted after the data is displayed.
The id from the survey table is displayed as the respondent_id which may
be used to combine the questions and answers from a specific respondent.
You may have to change the prefix on the table names.
Example: survey_answers to my_prefix_answers.
Use this to call the procedure.
Syntax: call survey.usp_produce_survey_report('<SURVERY_ID>', '<LANGUAGE>');
Example: call survey.usp_produce_survey_report('457345', 'en');
use this to create the table that stores the data
CREATE TABLE `survey_report` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`qid` int(11) NOT NULL DEFAULT '0',
`survey_row_id` int(11) NOT NULL DEFAULT '0' COMMENT 'id that is in the survey_<id> table',
`answer` mediumtext COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`)
);
*/
DECLARE v_col_name VARCHAR (25);
DECLARE v_qid INT;
DECLARE v_col_count INT DEFAULT 0;
DECLARE done INT DEFAULT false;
DECLARE tname VARCHAR(24) DEFAULT CONCAT('survey_survey_',survey_id);
DECLARE counter INT DEFAULT 0;
DECLARE current_row INT DEFAULT 0;
DECLARE total_rows INT DEFAULT 0;
-- select locate ('X','123457X212X1125', 8); -- use locate to determine location of second X - returns 11
-- select substring('123457X212X1125', 11+1, 7); -- use substring to get the qid - returns 1125
DECLARE cur1 cursor for
SELECT column_name, substring(column_name, 11+1, 7) as qid -- get the qid from the column name. the 7 might need to be higher depending on the id.
FROM information_schema.columns -- this has the column names
WHERE table_name = tname -- table name created form the id that was passed to the stored procedure
AND column_name REGEXP 'X'; -- get the columns that have an X
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
SET done = FALSE;
OPEN cur1;
SET total_rows = (SELECT table_rows -- get the number of rows
FROM INFORMATION_SCHEMA.TABLES
WHERE table_name = tname);
-- SELECT total_rows;
read_loop: LOOP
FETCH cur1 INTO v_col_name, v_qid; -- v_col_name is the original column name and v_qid is the qid that is taken from the column name
IF done THEN
LEAVE read_loop;
END IF;
-- SELECT v_col_name, v_qid;
SET counter = 1; -- use to compare id's
SET current_row = 1; -- used for the while loop
WHILE current_row <= total_rows DO
SET #sql := NULL;
-- SELECT v_col_name, v_qid, counter, x;
-- SELECT counter as id, v_col_name, v_qid as qid, x;
-- SET #sql = CONCAT ('SELECT id ', ',',v_qid, ' as qid ,', v_col_name,' FROM ', tname, ' WHERE id = ', counter );
-- I would have to join the survey table below if I did not add the answer (v_col_name). I assume this is faster than another join.
SET #sql = CONCAT ('INSERT INTO survey_report(qid,survey_row_id,answer) SELECT ',v_qid, ',id,' , v_col_name, ' FROM ', tname, ' WHERE id = ', counter );
-- SELECT #sql;
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
-- SELECT counter, x;
SET current_row = current_row + 1; -- increment counter for while loop
SET counter = counter + 1; -- increment counter for id's
END WHILE;
END LOOP; -- read_loop
CLOSE cur1;
-- SELECT * FROM survey_report
-- ORDER BY id, qid;
SET #counter = 0;
SELECT
#counter:=#counter + 1 AS newindex, -- increment the counter that is in the header
survey_report.id,
survey_report.survey_row_id as respondent_id, -- the id that copied from the survey table
survey_report.qid,
question,
IF(type IN ('S' , 'K'),
(SELECT answer
FROM survey_report
WHERE qid NOT IN (SELECT qid FROM survey_answers)
AND survey_questions.language = lang
AND survey_report.id = #counter),
(SELECT answer
FROM survey_answers
WHERE survey_questions.qid = survey_answers.qid
AND survey_report.qid = survey_questions.qid
AND survey_report.answer = survey_answers.code
AND survey_answers.language = lang
)
) AS answer
FROM survey_questions
JOIN survey_report ON survey_report.qid = survey_questions.qid
WHERE survey_questions.sid = survey_id
ORDER BY survey_report.survey_row_id, survey_report.id;
TRUNCATE TABLE survey_report;
END

Sequel Pro multiple update query with case and random variable

Alright, so I have a table that has a count field that is between 0 and 100. Up to six of these fields are tied to a single id. I need to run an update that will decrease each of these rows by a different random number between 1 and 3.
I know I can get my random value with:
CAST(RAND() * 3 AS UNSIGNED)
And I know I can get my update to work with:
UPDATE Info SET Info.count = CASE WHEN Info.count < 2 THEN 0 ELSE Info.count - 2 END WHERE Info.id = $iid AND Info.type = 'Active';
(This just makes sure I will never dip below 0)
But I cannot combine them for the obvious reason that my random number will be different when evaluated then when it's set...
UPDATE Info SET Info.count = CASE WHEN Info.count < CAST(RAND() * 3 AS UNSIGNED) THEN 0 ELSE Info.count - CAST(RAND() * 3 AS UNSIGNED) END WHERE Info.id = $iid AND Info.type = 'Active';
Now I don't want to save just 1 variable, because I may need up to 6 different numbers...is there a way to do what I want to do in a single query? I know how I can do it in multiple, but I really shouldn't be running up to 6 queries every time I need to update one of these blocks...
The table structure I'm working off of is:
CREATE TABLE `Info` (
`id` int(40) DEFAULT NULL,
`count` int(11) DEFAULT NULL,
`type` varchar(40) DEFAULT NULL,
`AUTOINC` int(11) unsigned NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`AUTOINC`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8;
The AUTOINC field at the moment is just so I can easily verify what's going on during testing.
Thanks!
You're probably best off writing a procedure to do this. So you could write something to the extent of (pseudo code)
create procedure update()
begin
declare id, count, sub int;
declare c cursor for select id, count floor(1+rand()*3) from info
where type='Active';
open c;
loop
fetch c into id, count, sub;
update info set case count - sub < 0 then 0 else count - sub end
where id = id;
end loop;
close c;
end
//
Or you can change the procedure to accept an ID like I had it before and simply use one select and one update statement.

Is recursive looping in MySQL queries possible?

I'm writing myself a forum, and I want to have one of those "you are here" strings along the top ("home > forum > sub forum > topic > etc" kind of thing). Now, the depth the forums can go to is limited to something like 128 by TINYINT in the database, not that this matters.
My question is this: is there a way to select the current forum (using it's ID - easy), but also select everything else it is inside of so I can generate the "you are here" string? Obviously "Home > " is hard coded, but the rest will be titles of forums and sub forums.
I'd need some sort of loop, starting from the deepest level forum I'm currently in and moving up to the top. Is the only way to do it using PHP loops and lots of queries? I'd rather just use one as it's faster.
Thanks,
James
You can do it with a trivially simple query, no joins... if you change your schema to make that information easy to extract. Look up the nested set model.
Well, once you have the initial ID, can't you just quickly use a PHP loop to generate a set of variables that you use to generate a "where" statement for your SQL query?
This is a previous answer of mine which might be of use: Recursively check the parents of a child in a database
It's a non recursive single call from php to db using a stored procedure...
-- TABLES
drop table if exists pages;
create table pages
(
page_id smallint unsigned not null auto_increment primary key,
title varchar(255) not null,
parent_page_id smallint unsigned null,
key (parent_page_id)
)
engine = innodb;
-- TEST DATA
insert into pages (title, parent_page_id) values
('Page 1',null),
('Page 2',null),
('Page 1-2',1),
('Page 1-2-1',3),
('Page 1-2-2',3),
('Page 2-1',2),
('Page 2-2',2);
-- STORED PROCEDURES
drop procedure if exists page_parents;
delimiter #
create procedure page_parents
(
in p_page_id smallint unsigned
)
begin
declare v_done tinyint unsigned default 0;
declare v_depth smallint unsigned default 0;
create temporary table hier(
parent_page_id smallint unsigned,
page_id smallint unsigned,
depth smallint unsigned default 0
)engine = memory;
insert into hier select parent_page_id, page_id, v_depth from pages where page_id = p_page_id;
/* http://dev.mysql.com/doc/refman/5.0/en/temporary-table-problems.html */
create temporary table tmp engine=memory select * from hier;
while not v_done do
if exists( select 1 from pages pg inner join hier on pg.page_id = hier.parent_page_id and hier.depth = v_depth) then
insert into hier
select pg.parent_page_id, pg.page_id, v_depth + 1 from pages pg
inner join tmp on pg.page_id = tmp.parent_page_id and tmp.depth = v_depth;
set v_depth = v_depth + 1;
truncate table tmp;
insert into tmp select * from hier where depth = v_depth;
else
set v_done = 1;
end if;
end while;
select
pg.page_id,
pg.title as page_title,
b.page_id as parent_page_id,
b.title as parent_page_title,
hier.depth
from
hier
inner join pages pg on hier.page_id = pg.page_id
left outer join pages b on hier.parent_page_id = b.page_id
order by
hier.depth, hier.page_id;
drop temporary table if exists hier;
drop temporary table if exists tmp;
end #
delimiter ;
-- TESTING (call this stored procedure from php)
call page_parents(5);
call page_parents(7);
If you assume the user navigated using the physical hierarchy of the forums, just use a lot of left joins as follows:
select current.forum as current,
parent1.forum as history1,
parent2.forum as history2,
parent3.forum as history3,
parent4.forum as history4,
parent5.forum as history5,
parent6.forum as history6
from forum current
left join forum parent1 on parent1.id = current.parentid
left join forum parent2 on parent2.id = parent1.parentid
left join forum parent3 on parent3.id = parent2.parentid
left join forum parent4 on parent4.id = parent3.parentid
left join forum parent5 on parent5.id = parent4.parentid
left join forum parent6 on parent6.id = parent5.parentid
Otherwise, you may want to create a forum breadcrumb table to store the history of locations the user has visited. Update this table with each location the user visits.