I have a database named my_db.
Each table in my_db has a column set_id that has values from 2140 to 2180. I would like to go through all the tables in my_db and delete the rows where set_id is greater than 2170. How do I do that?
I think, that is not a query for that, but you can do something like this
SELECT CONCAT('delete from my_db.',table_name,' where set_id > 270') FROM information_schema.tables where table_schema='my_db';
the result of that are all the queries that you need to run. you can copy and run it.
I like Marco's answer since it's short and provides a nice workaround. This is what I've come up with using Procedures, While-Loops, and Prepared-Statements. No copying is needed.
It retrieves and stores the into a temporary table, then loops through each table-name and performs individual DELETE statements.
DROP PROCEDURE IF EXISTS myPurge;
-- Will be used to signify the end of the procedure
DELIMITER ;;
-- Use a procedure
CREATE PROCEDURE myPurge()
BEGIN
-- Retrieve tables from information_schema and
-- Store them into a temporary table
DROP TABLE IF EXISTS tempTables;
CREATE TEMPORARY TABLE tempTables
SELECT table_name FROM information_schema.tables WHERE table_schema = 'my_db';
-- Initialise variables
SET #i = 0;
SELECT COUNT(*) FROM tempTables INTO #n;
-- Loop from 0 to number of rows
WHILE #i < #n DO
-- Retrieve table from row #i
-- SELECT * FROM tempTables LIMIT #i, 1 INTO #atable; -- doesn't seem to work on MySQL v5
-- Prepare and execute a subsidiary query
SELECT CONCAT("SELECT * FROM tempTables LIMIT ", #i, ",1 INTO #atable") INTO #preLimStmt;
PREPARE limStmt FROM #preLimStmt;
EXECUTE limStmt;
-- Save statement into temporary variable
-- HERE we prepare your PURGE
SELECT CONCAT("DELETE FROM my_db.", #atable, " WHERE set_id > 2170") INTO #preStmt;
-- Prepare and execute the purge statement
PREPARE stmt FROM #preStmt;
EXECUTE stmt;
-- Increment #i
SET #i = #i + 1;
END WHILE;
END;;
-- Call the procedure
CALL myPurge();
-- cleanup
DEALLOCATE PREPARE limStmt;
DEALLOCATE PREPARE stmt;
-- Reset to default
DELIMITER ;
-- Remove the temporary table
DROP TABLE tempTables;
Note: I'm using MySQL version 5.7.23.
Related
I've inherited a project that stores SQL queries inside mySQL tables (long story). I want to run those queries from a stored procedure and automatically convert the resultset to a JSON array of objects with keys equal to the selected columns.
I am trying to avoid a round-trip to the database to run these queries; hence me trying to do this in a stored procedure vs say python or php.
I don't see any way of doing this after searching here and the documentation pretty hard. I was hoping to find some functionality like "row to json" which takes an arbitrary row and returns it as json based on the columns returned. Is that possible?
After a bunch of trial and error, I was able to figure it out myself. Enjoy!
------------------------------------------------------------
-- columns_of_table
--
-- Sets #columns to JSON array of column names.
-- Works with temporary tables.
-- Cannot be a function since it uses dynamic SQL.
------------------------------------------------------------
DELIMITER ;
DROP PROCEDURE IF EXISTS columns_of_table;
DELIMITER $$
CREATE PROCEDURE columns_of_table(table_name VARCHAR(64))
BEGIN
SET #columns = '[]';
SET #query = CONCAT(
'SHOW COLUMNS FROM ', table_name,
' WHERE #columns := JSON_ARRAY_APPEND(#columns, "$", Field)'
);
PREPARE show_cols FROM #query;
EXECUTE show_cols;
DEALLOCATE PREPARE show_cols;
END;
$$
------------------------------------------------------------
-- query_to_json
--
-- Runs given query and converts resultset to JSON array.
--
-- Note: memory engine doesn't work if there are TEXT or
-- BLOB fields in the resultset as temporary tables cannot
-- contain TEXT or BLOB fields for some mystical reason.
--
-- Cannot be a function since it uses dynamic SQL.
--
-- Sets #json_results.
------------------------------------------------------------
DELIMITER ;
DROP PROCEDURE IF EXISTS query_to_json;
DELIMITER $$
CREATE PROCEDURE query_to_json(query TEXT, use_memory_engine BOOLEAN)
BEGIN
DECLARE i INT DEFAULT 0;
DECLARE col VARCHAR(64);
DECLARE object_params TEXT DEFAULT '';
-- Wrap the query such that the results are stored in a temporary table.
DROP TEMPORARY TABLE IF EXISTS temp_results;
SET #query = CONCAT('CREATE TEMPORARY TABLE temp_results ',
IF(use_memory_engine, ' ENGINE=memory ', ''),
query);
PREPARE stmt FROM #query;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
-- Find the columns of the temporary table.
CALL columns_of_table('temp_results');
-- Convert from #columns of '["x", "y"]' to object_params of '"x", x, "y", y'
-- which are interpolated into SQL that calls JSON_OBJECT.
WHILE i < JSON_LENGTH(#columns) DO
SELECT JSON_EXTRACT(#columns, CONCAT('$[', i, ']')) INTO col;
SET object_params = CONCAT(object_params, col, ',', REPLACE(col, '"', ''), ',');
SELECT i + 1 INTO i;
END WHILE;
SET object_params = TRIM("," FROM object_params);
-- Run dynamic query that fetches result rows as JSON objects and concatenates
-- them into a proper JSON array. If you have access to JSON_ARRAYAGG, consider
-- updating this.
SET #query = CONCAT(
'SELECT CONCAT("[", GROUP_CONCAT(JSON_OBJECT(', object_params, ')), "]") ',
'FROM temp_results '
'INTO #json_results'
);
PREPARE stmt FROM #query;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END;
$$
DELIMITER ;
We have a large piece of complex SQL code that constantly falls over when run in HeidiSQL and MySQL WB. If we run the same code with smaller number of records it runs fine.
We are thinking that if we create a loop in an SP where the code is effectively split into 10 to 20 runs it may complete successfully. I think this may involve creating 10 to 20 tables which are union-ed into one table later on.
Tried running on smaller data sets, runs fine. Tried increasing the Server RAM and disk space. Tried adapting the code in many different ways.
I realise this may not be the ideal solution but have been asked to do it this way. If anyone could help me with the code, I would be grateful.
Below is code I built but doesn't work...
DELIMITER $$
DROP PROCEDURE IF EXISTS test_mysql_while_loop$$
CREATE PROCEDURE test_mysql_while_loop()
BEGIN
DECLARE x INT;
SET x = 100;
WHILE x <= 2000 DO
DROP TABLE if EXISTS tablex; /* want to end up with table100-table2000 */
CREATE TABLE tablex AS (
SELECT t1.*
FROM bigtables t1
WHERE urn BETWEEN x AND x+101); /* select records where the URN is between 100 and 201 in 1st instance , 200 and 301 in second etc*/
SET x=x+100;
END WHILE;
END$$
DELIMITER ;
CALL test_mysql_while_loop();
20 tables
Here's a simplified example of how to prepare and execute statements
DROP PROCEDURE IF EXISTS p;
delimiter $$
CREATE PROCEDURE p()
BEGIN
DECLARE x INT;
SET x = 100;
WHILE x <= 300 DO
set #sql= (select concat('DROP TABLE if EXISTS table',x));
select #sql;
prepare sqlstmt from #sql;
execute sqlstmt;
deallocate prepare sqlstmt;
set #sql = (select concat('CREATE TABLE table',x,' AS
SELECT t1.*
FROM users t1;'));
select #sql;
prepare sqlstmt from #sql;
execute sqlstmt;
deallocate prepare sqlstmt;
SET x=x+100;
END WHILE;
END$$
DELIMITER ;
CALL p();
This is the sql i'm using based from this answer:
SET #pattern = '%_movielist';
SELECT concat('TRUNCATE TABLE ', GROUP_CONCAT(concat(TABLE_NAME)), ';')
INTO #truncatelike FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE #pattern;
SELECT #truncatelike;
PREPARE stmt FROM #truncatelike;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
but I get this error Access denied for user 'root'#'%' to database 'information_schema'.
What am I doing wrong? It seems to work for other users
You trying to execute this statement on "information_schema" database. Read more about this database [https://dev.mysql.com/doc/refman/5.7/en/information-schema.html]
You should not be running statements on the information_schema database (unless you REALLY know what you're doing). The database serves as a "meta" repository that dictates how the server operates. Chances are that you have no need to touch it and you'll likely brick your server if you do.
This is already answered here. [#1044 - Access denied for user 'root'#'localhost' to database 'information_schema'
Restriction to above: This query will work only if the no of table returned by the statement is 1 for more than 1 tables, you will require to use it in iteration.
To make this work for all the table matching the pattern we would require to use stored procedure.
Please change the Procedure name
CREATE PROCEDURE `new_procedure`()
BEGIN
-- Pattern to Match
SET #pattern = '%_movielist';
-- Temporary Table to Store the Result of The Select Statement
CREATE TEMPORARY TABLE IF NOT EXISTS Table_ToBeTruncated
(
Id int NOT NULL AUTO_INCREMENT,TableName varchar(100),
PRIMARY KEY (id)
);
-- Insert all the TableName to be Truncated
insert Table_ToBeTruncated(TableName)
SELECT distinct concat('TRUNCATE TABLE `', TABLE_NAME, '`;')
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE #pattern and TABLE_SCHEMA = 'movielist';
-- Declare a variable to count the no of records to be truncated.
SET #count=(Select count(*)from Table_ToBeTruncated);
-- Iterate the list
WHILE #count> 0 DO
-- Pick One table from the Temporary Table List;
SELECT TableName into #truncatelike from Table_ToBeTruncated where ID= #count;
-- Prepare the statement
PREPARE stmt FROM #truncatelike;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
-- Decrease the counter.
set #count = #count- 1;
END WHILE;
drop TEMPORARY TABLE IF EXISTS Table_ToBeTruncated ;
END
I want to get the number of results on my temporary table, so I created a query, something like this:
SELECT * FROM tmp_sales_negative; -- Selected all columns on my temporary table
SET #limit = (SELECT FOUND_ROWS()); -- Set a variable assignment named 'limit'
The above query works fine and I got the correct number of resulting rows, but when I use the variable name in this query:
INSERT INTO tmp_credit_memo_receivables(credit_memo_id, amount)
SELECT id, amount FROM credit ORDER BY id DESC LIMIT #limit;
It prompts a syntax error on #limit part. Am I doing it wrong?
Thanks!
You have to use dynamic sql for this. To use dynamic sql you have to do this in a stored procedure.
Create the procedure like this:
DELIMITER $$
CREATE PROCEDURE sp_limit()
BEGIN
SELECT SQL_CALC_FOUND_ROWS * FROM tmp_sales_negative;
SET #limit = (SELECT FOUND_ROWS());
SET #sql = 'INSERT INTO tmp_credit_memo_receivables(credit_memo_id, amount)
SELECT id, amount FROM credit ORDER BY id DESC LIMIT ?;';
PREPARE stmt FROM #sql;
EXECUTE stmt USING #limit;
DEALLOCATE PREPARE stmt;
END $$
DELIMITER ;
Then execute it with
CALL sp_limit();
I want to create temp tables which have different table names.
The stored procedure takes no input arguments, create a temp table and return
a table name of the table as T_1, T_2, T_3....
How can I implement this in mysql stored procedure?
You can use the TEMPORARY keyword when creating a table. A TEMPORARY table is visible only to the current session, and is dropped automatically when the session is closed. This means that two different sessions can use the same temporary table name without conflicting with each other or with an existing non-TEMPORARY table of the same name. (The existing table is hidden until the temporary table is dropped.) To create temporary tables, you must have the CREATE TEMPORARY TABLES privilege.
Following procedure should help you. But this kind of sequence generation will work in current connection session only. But I hope it is OK as you are expecting it on temporary tables.
delimiter //
drop procedure if exists set_new_temp_table //
create procedure set_new_temp_table()
begin
if( #temp_table_seq_num is null ) then
set #temp_table_seq_num := 1;
end if;
set #temp_table_name := Concat( 'T_', #temp_table_seq_num );
set #sql := concat( 'create temporary table if not exists '
, #temp_table_name
, '( col1 int, col2 varchar(10) )'
);
prepare stmt from #sql;
execute stmt;
drop prepare stmt;
set #temp_table_seq_num := ( cast( #temp_table_seq_num as decimal ) + 1 );
set #sql := null;
end;
//
delimiter ;
select #temp_table_name; -- should return a NULL before first ever call to SP
call set_new_temp_table(); select #temp_table_name;
Demo # MySQL 5.5.32 Fiddle
This is full example
SELECT UUID() INTO #RandomName;
SET #TempTableNameWithSpecialCharectors := CONCAT('TEMP', #RandomName);
SET #TempTableName := REPLACE(#TempTableNameWithSpecialCharectors, '-', '');
SET #tempTable := CONCAT('CREATE TEMPORARY TABLE IF NOT EXISTS ', #TempTableName, ' (InteractionRequestId bigint(20), SendCount int)');
-- SELECT #tempTable;
PREPARE createStmt FROM #tempTable;
EXECUTE createStmt;
DROP PREPARE createStmt;
SET #enqueueDate := '2017-05-28';
SET #insertIntoTable := CONCAT('
INSERT INTO ', #TempTableName, '(InteractionRequestId, SendCount) SELECT InReq.Id, COUNT(*) AS SendCount FROM InteractionRequests AS InReq INNER JOIN InteractionResponses InRes ON InReq.Id = InRes.InteractionRequestId
WHERE InReq.EnqueuedRequest > "', #enqueueDate,
'" GROUP BY InReq.Id');
SELECT #insertIntoTable;
PREPARE insertStmt FROM #insertIntoTable;
EXECUTE insertStmt;
DROP PREPARE insertStmt;
SET #SelctResult := CONCAT('SELECT * FROM ', #TempTableName);
PREPARE selectStmt FROM #SelctResult;
EXECUTE selectStmt;
DROP PREPARE selectStmt;