I'm working on an old database already in use for years and really crappy designed.
There is a table, "Articles", which contains a "code" column that will be our PK.
And many tables like "idXXXXX" where XXXXX is a "code" value with exactly the same structure.
I looked at the application using this database and saw that relations between tables is made there.
I'm not affraid of redesign the database access in the application, but I don't want to lose years of entries in the database.
I want to create a "campain" table which will have an "id" PK and a "id_code" as FK linking "campain" to "articles"
I'm not a SQL master but I know I can get tables names with
SELECT TABLE_NAME FROM INFORMATION_SCHEMA WHERE TABLE_NAME LIKE 'id%'
But I have really no idea about how to deal with the result (which is fine).
So how can I access to every tables named "idXXX" and insert every rows in the "campain" table + set "id_code" column to "XXX"?
Here is the procedure I saved (I didn't add every fields in the INSERT line for testing purpose) :
CREATE PROCEDURE JoinAllTables()
BEGIN
DECLARE done INT default 0;
DECLARE tableName CHAR(9);
DECLARE buffStr CHAR(7);
DECLARE buffId INT default 0;
DECLARE cur1 CURSOR FOR SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE 'id%';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO tableName;
IF done THEN
LEAVE read_loop;
END IF;
SET buffStr = SUBSTRING(tableName, 3);
SET buffId = CAST(buffStr AS SIGNED);
set #sql = CONCAT("INSERT INTO campagnes(id, id_code) SELECT null, bufId FROM ",tableName); # Dynamically building sql statement
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP;
CLOSE cur1;
END;
As u can see, I sub 'idXXXXX' to 'XXXXX' then CAST it AS INTEGER (SIGNED).
But I guess that in the "INSERT INTO" line, second tableName doesn't point to the variable. That's why I'm getting a
"#1446 - Tabble 'bddsoufflage.tablename'doesn't exist" Error :) Any idea ?
Edit: Updated answer
We can't have the tableName dynamically changed inside a prepared statement, so we must go through DynamicSQL to build the query using CONCAT, then compile the SQL with PREPARE, EXECUTE it and DEALLOCATE it.
DELIMITER //
CREATE PROCEDURE JoinAllTables()
BEGIN
DECLARE done INT default 0;
DECLARE tableName CHAR(9);
DECLARE buffStr CHAR(7);
DECLARE buffId INT default 0;
DECLARE cur1 CURSOR FOR SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE 'id%';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO tableName;
IF done THEN
LEAVE read_loop;
END IF;
SET buffStr = SUBSTRING(tableName, 3);
SET buffId = CAST(buffStr AS SIGNED);
set #sql = CONCAT("INSERT INTO campagnes(id, id_code) SELECT null, ", buffId, " FROM ",tableName); # Dynamically building sql statement
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP;
CLOSE cur1;
END; //
See also this answer MySQL Pass table name to cursor select
Old answer
The procedure should look something like this. Thanks Mchl for providing an Insert Into query example, I simply added it to the rest of the procedure.
DELIMITER //
CREATE PROCEDURE JoinAllTables()
BEGIN
DECLARE done INT default 0;
DECLARE tableName CHAR(7); # Variable to contain table names CHAr(7) is assuming id + 5Xs as characters.
DECLARE cur1 CURSOR FOR SELECT TABLE_NAME FROM INFORMATION_SCHEMA WHERE TABLE_NAME LIKE 'id%'; # Create a cursor to iterate over the tables
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO tableName;
IF done THEN
LEAVE read_loop;
END IF;
#Your Insert statement here, using tableName as a field.
INSERT INTO campain (id, id_code, otherfields) SELECT null, tableName, otherfields FROM tableName;
END LOOP;
CLOSE cur1;
END;//
Easiest way would be to run the information_schema query you have within some script (PHP,Python,Perl - whichever suits you best) and use it's results to create queries like:
INSERT INTO
campain (id, id_code, otherfields)
SELECT
null, 'idXXXX', otherfields FROM idXXXX
Related
I wanted to use table names and run a statement with the table name as variable. I used cursor/fetch but when I run a statement with the variable it is not using the value of the variable but just seems to use the variable_name itself. I have seen example with concat where another variable was defined but what if I just wanted to reference the table name in a COMMAND?
DELIMITER $$
DROP PROCEDURE IF EXISTS test $$
CREATE PROCEDURE test()
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE v_table_name TEXT;
DECLARE cur1 CURSOR FOR
SELECT TABLE_NAME FROM information_schema.tables where table_schema = 'rt';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done= TRUE;
OPEN cur1;
myloop: loop
FETCH cur1 INTO v_table_name;
IF done THEN
LEAVE myloop;
END IF;
COMMAND table v_table_name;
END loop;
close cur1;
END $$
If by COMMAND you mean you want to use the value of a variable as an identifier in another SQL statement... you may be able to make use of a prepared statement (in the context of a MySQL stored program).
Reference: https://dev.mysql.com/doc/refman/5.6/en/sql-syntax-prepared-statements.html
As a trivial example of what that might look like:
SET #sql = CONCAT('select * from `',v_table_name,'` limit 1');
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;
SET #sql = NULL;
Note that this approach is not safe from SQL Injection vulnerabilities.
If that's not what you are looking for, I'm at a loss. I don't understand what you are referring to as a COMMAND.
I am trying to find all the tables that has a column name called RecordID and then loop over those tables to see if the RecordID > 5000 or not.
CREATE PROCEDURE check_IDs ()
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE result varchar(50);
DECLARE cur1 CURSOR for SELECT table_name FROM INFORMATION_SCHEMA.COLUMNS
WHERE column_name = 'RecordID' ;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO result;
IF done THEN
LEAVE read_loop;
END IF;
Select * from `result` where RecordID > 5000;
END LOOP;
close cur1;
END;
it says table result not found.
It's got to be
PREPARE stmt FROM CONCAT('SELECT * FROM `', result, '`WHERE RecordID > 5000;)';
EXECUTE stmt;
DEALLOCATE stmt;
instead of
Select * from `result` where RecordID > 5000;
see the examples (last one) of SQL Syntax for Prepared Statements
Your statement will use the hardcoded table name result. It seems, there's no such table in your database.
Seen a lot for dropping tables using a wildcard but not a direct SQL statement except this one:
http://azimyasin.wordpress.com/2007/08/11/mysql-dropping-multiple-tables/
It says:
SHOW TABLES LIKE ‘phpbb_%’;
then DROP TABLES, is there a neat way to combine this all into one SQL Statement?
You could use dynamic SQL to do it, inside a stored procedure. It'd look something like this (untested):
CREATE PROCEDURE drop_like (IN pattern VARCHAR(64))
BEGIN
DECLARE q tinytext;
DECLARE done INT DEFAULT FALSE;
DECLARE cur CURSOR FOR
SELECT CONCAT('DROP TABLE "', table_schema, '"."', table_name, '"')
FROM information_schema.tables WHERE table_type = 'BASE TABLE' AND table_name LIKE pattern;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur;
drop_loop: LOOP
FETCH cur INTO q;
IF done THEN
LEAVE drop_loop;
END IF;
PREPARE stmt FROM #q;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP;
CLOSE cur;
END;
Using dynamic SQL in a query, as per derobert's answer, is the only to do this with pure SQL (no app code).
I wrote a generalized procedure to do this sort of thing (run a query for every table in a database) that you can find here - to use it, you would just need to run this query:
CALL p_run_for_each_table('databasename', 'DROP TABLE `{?database}`.`{?table}`');
It works in essentially the same way as derobert's answer.
However, the writer of that blog post was probably expecting you to write app code to turn the names of tables into a single DROP statement.
To do this, you would iterate over the results of the SHOW TABLE in your code and build a single query like this:
DROP TABLE table1, table2, tablewhatever;
This can be achieved via stored procedure, for example:
CREATE DEFINER=`some_user`#`%` PROCEDURE `drop_tables`()
LANGUAGE SQL
NOT DETERMINISTIC
MODIFIES SQL DATA
SQL SECURITY DEFINER
COMMENT ''
BEGIN
#We need to declare a variable with default 0 to determine weather to continue the loop or exit the loop.
DECLARE done INT DEFAULT 0;
DECLARE archive_table_name VARCHAR(100);
#Select desired tables from `information_schema`
DECLARE cur CURSOR FOR
SELECT t.`TABLE_NAME` FROM information_schema.`TABLES` t WHERE t.`TABLE_NAME` LIKE 'some_table_name%'
AND t.CREATE_TIME BETWEEN DATE_SUB(NOW(), INTERVAL 9 MONTH) AND DATE_SUB(NOW(), INTERVAL 6 MONTH);
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur;
read_loop: LOOP
#Fetch one record from CURSOR and set variable (if not found, then variable `done` will be set to 1 by continue handler)
FETCH cur INTO archive_table_name;
IF done THEN
LEAVE read_loop; #If done is set to 1, then exit the loop, else continue
END IF;
#Do your work
-- Create the truncate query
SET #s = CONCAT('DROP TABLE IF EXISTS ', archive_table_name);
-- Prepare, execute and deallocate the truncate query
PREPARE drop_statement FROM #s;
EXECUTE drop_statement;
DEALLOCATE PREPARE drop_statement;
END LOOP;
CLOSE cur; #Closing the cursor
END
Pay attention to the database user, which is creating/executing the stored routine: it must have appropriate credentials for executing/dropping tables.
I'm performing some database clean up and have noticed that there are a lot of columns that have both empty strings and NULL values in various columns.
Is it possible to write an SQL statement to update the empty strings to NULL for each column of each table in my database, except for the ones that do not allow NULL's?
I've looked at the information_schema.COLUMNS table and think that this might be the place to start.
It's not possible to do this with one simple SQL statement.
But you can do it using one statement for each column.
UPDATE TABLE SET COLUMN = NULL
WHERE LENGTH(COLUMN) = 0
or, if you want to null out the items that also have whitespace:
UPDATE TABLE SET COLUMN = NULL
WHERE LENGTH(TRIM(COLUMN)) = 0
I don't think it's possible within MySQL but certainly with a script language of your choice.
Start by getting all tables SHOW TABLES
Then for each table get the different columns and find out witch ones allow null, either with DESC TABLE, SHOW CREATE TABLE or SELECT * FROM information_schema.COLUMNS, take the one you rather parse
Then for each column that allows null run a normal update that changes "" to null.
Prepare to spend some time waiting :)
I figured out how to do this using a stored procedure. I'd definitely look at using a scripting language next time.
DROP PROCEDURE IF EXISTS settonull;
DELIMITER //
CREATE PROCEDURE settonull()
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE _tablename VARCHAR(255);
DECLARE _columnname VARCHAR(255);
DECLARE cur1 CURSOR FOR SELECT
CONCAT(TABLE_SCHEMA, '.', TABLE_NAME) AS table_name,
COLUMN_NAME AS column_name
FROM information_schema.COLUMNS
WHERE IS_NULLABLE = 'YES'
AND TABLE_SCHEMA IN ('table1', 'table2', 'table3');
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO _tablename, _columnname;
IF done THEN
LEAVE read_loop;
END IF;
SET #s = CONCAT('UPDATE ', _tablename, ' SET ', _columnname, ' = NULL WHERE LENGTH(TRIM(', _columnname, ')) = 0' );
PREPARE stmt FROM #s;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP;
CLOSE cur1;
END//
DELIMITER ;
CALL settonull();
I would like to use LIMIT in a cursor. The cursor should be used and updated several times within a loop, each time with different parameters of LIMIT. Here some code:
DELIMITER $$
CREATE PROCEDURE `updateIt`() READS SQL DATA
BEGIN
declare done int(1) default 0;
declare counter int(10) default 0;
declare xabc int(10) default 0;
declare tab1Cursor cursor for select abc from tab1 limit 100000*counter, 100000;
declare continue handler for not found set done=1;
loopCounter: LOOP
set done = 0;
open tab1Cursor;
igmLoop: loop
fetch tab1Cursor into xabc;
if done = 1 then leave igmLoop; end if;
-- do something
end loop igmLoop;
close tab1Cursor;
if (counter = 1039)
leave loopCounter;
end if;
set counter = counter + 1;
END LOOP loopCounter;
END $$
DELIMITER ;
This, however, does not work (I also tried it with the cursor in the LOOP counterLoop). Can Mysql deal with dynamic cursors?
From the MySQL Manual
a cursor cannot be used for a dynamic statement that is prepared and
executed with PREPARE and EXECUTE. The statement for a cursor is
checked at cursor creation time, so the statement cannot be dynamic.
However there are 2 ways, according to this post in mysql forums:
The first is for cases where absolutely only one user at a time will be running the procedure. A prepare statement can be used to create a view with the dynamic SQL and the cursor can select from this statically-named view. There's almost no performance impact. Unfortunately, these views are also visible to other users (there's no such thing as a temporary view), so this won't work for multiple users.
Analogously, a temporary table can be created in the prepare statement and the cursor can select from the temporary table. Only the current session can see a temporary table, so the multiple user issue is resolved. But this solution can have significant performance impact since a temp table has to be created each time the proc runs.
Bottom line: We still need cursors to be able to be created dynamically!
Here's an example of using a view to pass the table name and column name into a cursor.
DELIMITER //
DROP PROCEDURE IF EXISTS test_prepare//
CREATE PROCEDURE test_prepare(IN tablename varchar(255), columnname varchar(50))
BEGIN
DECLARE cursor_end CONDITION FOR SQLSTATE '02000';
DECLARE v_column_val VARCHAR(50);
DECLARE done INT DEFAULT 0;
DECLARE cur_table CURSOR FOR SELECT * FROM test_prepare_vw;
DECLARE CONTINUE HANDLER FOR cursor_end SET done = 1;
SET #query = CONCAT('CREATE VIEW test_prepare_vw as select ', columnname, ' from ', tablename);
select #query;
PREPARE stmt from #query;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
OPEN cur_table;
FETCH cur_table INTO v_column_val;
WHILE done = 0 DO
SELECT v_column_val;
FETCH cur_table INTO v_column_val;
END WHILE;
CLOSE cur_table;
DROP VIEW test_prepare_vw;
END;
//
DELIMITER ;