Is it possible to show the name of a table in a db where a specific value is present. I have different tables and i want to show only the table names that contains a specific value in any of the fields.
This will return lots of empty result sets, but the non-empty ones correspond to table/column combinations that fit your search. It only works for text, and detects columns that contain the value (as opposed to a full column match.)
DELIMITER |
DROP PROCEDURE IF EXISTS `SearchAllTables`|
CREATE PROCEDURE `SearchAllTables` (
IN _search varchar(256)
)
LANGUAGE SQL
DETERMINISTIC
SQL SECURITY DEFINER
BEGIN
-- declare stuff
declare _tableName varchar(64);
declare _columnName varchar(64);
declare _done tinyint(1) default 0;
-- we will examine every string column in the database
declare _columnCursor cursor for
select TABLE_NAME, COLUMN_NAME
from INFORMATION_SCHEMA.COLUMNS
where TABLE_SCHEMA = database()
and (DATA_TYPE like '%char%'
or DATA_TYPE like 'text');
declare CONTINUE handler for NOT FOUND
SET _done = 1;
OPEN _columnCursor;
LOOP1: LOOP
-- get the next table/column combination
FETCH _columnCursor INTO _tableName,_columnName;
IF _done = 1 THEN
CLOSE _columnCursor;
LEAVE LOOP1;
END IF;
-- query the current column to see if it holds the value
SET #query = concat(
"select '",_tableName,"' as TableName, '",
_columnName,"' as ColumnName
from ",_tableName,"
where ",_columnName," like concat('%',?,'%')
group by 1;"
);
SET #search = _search;
PREPARE _stmt FROM #query;
EXECUTE _stmt USING #search;
DEALLOCATE PREPARE _stmt;
END LOOP LOOP1;
END|
DELIMITER ;
Oh, yeah, and it's ugly... Maybe it'll help you, though!
SELECT TABLE_NAME
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'database_name'
AND COLUMN_NAME = 'column_name'
Related
The goal is to add the missing columns when comparing the schemas of two tables in MySQL 5.5 (engine MyISAM). The argument p_table1 is the model table name from which p_table2 will be compared and "synchronized".
When it is called, nothing happens, no errors, no nothing. I've tried to log some variables but it hasn't worked either.
What could be wrong with the code?
CREATE PROCEDURE synchronize_tables(p_table1 VARCHAR(64), p_table2 VARCHAR(64), p_schema_name VARCHAR(64))
BEGIN
DECLARE v_done INT default false;
DECLARE v_actual_column_name VARCHAR(64);
DECLARE v_does_columns_exist INT default true;
DECLARE v_column_type LONGTEXT;
DECLARE v_column_default LONGTEXT;
DECLARE v_is_nullable VARCHAR(3);
DECLARE v_cur CURSOR FOR
SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = p_table1
AND table_schema = p_schema_name;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET v_done = TRUE;
DECLARE EXIT HANDLER FOR SQLEXCEPTION
BEGIN
OPEN v_cur;
read_loop: LOOP
FETCH v_cur INTO v_actual_column_name;
IF v_done THEN
LEAVE read_loop;
END IF;
SELECT count(*) INTO v_does_columns_exist
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = p_table2
AND table_schema = p_schema_name
AND column_name = v_actual_column_name;
IF NOT v_does_columns_exist THEN
SELECT column_type, COLUMN_DEFAULT, IS_NULLABLE
INTO v_column_type, v_column_default, v_is_nullable
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = p_table1
AND table_schema = p_schema_name
AND column_name = v_actual_column_name;
SET #stmt_text = CONCAT('ALTER TABLE ', p_schema_name, '.', p_table2,
' ADD COLUMN ', v_actual_column_name, ' ', v_column_type, ' ', IF(upper(v_is_nullable) = 'NO', 'NOT NULL', ''),
' DEFAULT ', v_column_default);
prepare v_stmt FROM #stmt_text;
execute v_stmt;
deallocate prepare v_stmt;
END IF;
END LOOP;
CLOSE v_cur;
END;
END
I found a couple of problems.
First is that most of your cursor code is inside an EXIT HANDLER FOR SQLEXCEPTION. This block is run only if an error occurs. So normally this block will never be run.
Second, you CONCAT() columns to form your ALTER TABLE statement, but one or more of the columns can be NULL. When you CONCAT() any string with a null, the result of the whole concat operation is NULL. So you have to make sure NULLs are defaulted to something non-NULL.
In my test, the column default is frequently NULL. We'd want this to become the keyword "NULL" in the ALTER TABLE statement. Also if the default is not NULL, you probably want to quote it, because an ordinary default value may be a string or a date, and you aren't quoting it. The solution: QUOTE() is a builtin function that quotes strings properly, and it even turns a NULL into the keyword "NULL".
Here's what I got to work:
CREATE PROCEDURE synchronize_tables(p_table1 VARCHAR(64),
p_table2 VARCHAR(64), p_schema_name VARCHAR(64))
BEGIN
DECLARE v_done INT default false;
DECLARE v_actual_column_name VARCHAR(64);
DECLARE v_does_columns_exist INT default true;
DECLARE v_column_type LONGTEXT;
DECLARE v_column_default LONGTEXT;
DECLARE v_is_nullable VARCHAR(3);
DECLARE v_cur CURSOR FOR
SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = p_table1
AND table_schema = p_schema_name;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET v_done = TRUE;
OPEN v_cur;
read_loop: LOOP
FETCH v_cur INTO v_actual_column_name;
IF v_done THEN
LEAVE read_loop;
END IF;
SELECT count(*) INTO v_does_columns_exist
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = p_table2
AND table_schema = p_schema_name
AND column_name = v_actual_column_name;
IF NOT v_does_columns_exist THEN
SELECT column_type, COLUMN_DEFAULT, IS_NULLABLE
INTO v_column_type, v_column_default, v_is_nullable
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = p_table1
AND table_schema = p_schema_name
AND column_name = v_actual_column_name;
SET #stmt_text = CONCAT('ALTER TABLE ',
p_schema_name, '.', p_table2,
' ADD COLUMN ', v_actual_column_name, ' ',
v_column_type, ' ',
IF(upper(v_is_nullable) = 'NO', 'NOT NULL', ''),
' DEFAULT ', QUOTE(v_column_default));
PREPARE v_stmt FROM #stmt_text;
EXECUTE v_stmt;
DEALLOCATE prepare v_stmt;
END IF;
END LOOP;
CLOSE v_cur;
END
There are other problems with this approach:
Identifiers are not delimited.
It generates one ALTER TABLE for each missing column, even though one can add multiple columns in one ALTER.
It doesn't do the right thing when a column is NOT NULL but has no default.
However, I wouldn't do this with cursors anyway. There's an easier way:
CREATE PROCEDURE synchronize_tables(p_table1 VARCHAR(64),
p_table2 VARCHAR(64), p_schema_name VARCHAR(64))
BEGIN
SELECT CONCAT(
'ALTER TABLE `', C1.TABLE_SCHEMA, '`.`', p_table2, '` ',
GROUP_CONCAT(CONCAT(
'ADD COLUMN `', C1.COLUMN_NAME, '` ', C1.COLUMN_TYPE,
IF(C1.IS_NULLABLE='NO', ' NOT NULL ', ''),
IF(C1.COLUMN_DEFAULT IS NULL, '',
CONCAT(' DEFAULT ', QUOTE(C1.COLUMN_DEFAULT)))
) SEPARATOR ', '
)
) INTO #stmt_text
FROM INFORMATION_SCHEMA.COLUMNS AS C1
LEFT OUTER JOIN INFORMATION_SCHEMA.COLUMNS AS C2
ON C1.TABLE_SCHEMA=C2.TABLE_SCHEMA
AND C2.TABLE_NAME=p_table2
AND C1.COLUMN_NAME=C2.COLUMN_NAME
WHERE C1.TABLE_SCHEMA=p_schema_name AND C1.TABLE_NAME=p_table1
AND C2.TABLE_SCHEMA IS NULL;
PREPARE v_stmt FROM #stmt_text;
EXECUTE v_stmt;
DEALLOCATE prepare v_stmt;
END
This makes one ALTER TABLE to add all columns. It delimits identifiers. It handles NOT NULL better.
But even my solution still has problems:
Causes an error if you try to add a NOT NULL column with no DEFAULT to a populated table2.
Doesn't notice extra columns that exist in the second table but not the first table.
Doesn't synchronize constraints, indexes, triggers, procedures, or views.
It's a very complex task to sync database structure completely. I suggest that trying to use MySQL's stored procedure language for this is making a hard task even harder.
MySQL's implementation of stored procedures sucks.
Poor documentation.
No support for packages.
No standard procedures or rich function library.
No real debugger exists (some tools try, but they're faking it).
No support for persisting compiled procedures. Procedures are recompiled by every session that uses them.
I often recommend to developers who are accustomed to using procedures in Oracle or Microsoft SQL Server, to stay away from MySQL stored procedures.
I want to alter my tables dynamically based on whether the table has specific column.
My database name is summer_cms, and there are over 50 tables in it.
What I want are below:
If a table has a column named add_time, then I would like to add a column add_user_id in it.
Similarly, I would like to add update_user_id in the table if update_time is found.
I know I should get it down in the process of creating the database schemas, but my database has been built and I have to alter it by need.
So I create a procedure to do it:
CREATE PROCEDURE ALTER_SUMMER_TABLE()
BEGIN
DECLARE tableName VARCHAR(64);
DECLARE exitence VARCHAR(64);
DECLARE ntable INT; # number of tables
DECLARE i INT; # index
SET i = 0;
# get the count of table
SELECT COUNT(DISTINCT(TABLE_NAME)) INTO ntable FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = 'summer_cms';
WHILE i < ntable DO
# select the specific table name into the variable of `tableName`.
SELECT TABLE_NAME INTO tableName
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'summer_cms'
AND COLUMN_NAME = 'add_time'
LIMIT 1 OFFSET i;
# alter table, but I get error in this clause.
ALTER TABLE tableName ADD COLUMN `add_user_id` INT NOT NULL DEFAULT 0 COMMENT 'add user id';
# check if the table has `update_time`
SELECT TABLE_NAME INTO exitence
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'summer_cms'
AND TABLE_NAME = tableName
AND COLUMN_NAME = 'update_time';
# add `update_user_id` if `update_time` be found.
IF exitence THEN
ALTER TABLE tableName ADD COLUMN `update_user_id` INT NOT NULL DEFAULT 0 COMMENT 'update user id';
END IF;
SET i = i + 1;
END WHILE;
END
But I got an error when I call this procedure.
Procedure execution failed
1146 - Table 'summer_cms.tableName' doesn't exist
Dose anyone could tell me what I was missing or wrong? Any help will be appreciated.
There a a few alterations you can make to your procedure to make it more streamlined as well as getting round a few problems.
First using a cursor to select the table names rather than using the two selects your using. Secondly to use a prepared statement to allow you to dynamically set the table name...
DELIMITER $$
CREATE DEFINER=`root`#`localhost` PROCEDURE `ALTER_SUMMER_TABLE`()
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE tableName VARCHAR(64);
declare cur cursor for SELECT TABLE_NAME
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'summer_cms'
AND COLUMN_NAME = 'add_time';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
open cur;
start_loop: loop
fetch cur into tableName;
if (done = 1 )THEN
LEAVE start_loop;
END IF;
SET #sql = CONCAT('ALTER TABLE ', tableName,' ADD COLUMN `add_user_id` INT NOT NULL DEFAULT 0 ');
PREPARE stmt FROM #sql;
EXECUTE stmt;
end loop;
close cur;
END$$
DELIMITER ;
You could do a few tweaks - only fetch table names where the column doesn't already exist for example.
Here's an example of dynamic sql
drop procedure if exists alter_table;
delimiter //
CREATE DEFINER=`root`#`localhost` procedure alter_table()
begin
declare tablename varchar(20);
set tablename = 'u';
set #sqlstmt = concat('ALTER TABLE ', tableName, ' ADD COLUMN ', char(96), 'add_user_id', char(96), ' INT NOT NULL DEFAULT 0 COMMENT', char(39), 'add user id', char(39),';');
prepare stmt from #sqlstmt;
execute stmt;
deallocate prepare stmt;
end //
delimiter ;
Note I have used ascii backticks and single quotes.
Assuming a database with a ridiculous amount of tables (200+), how can I perform SELECT * FROM <> LIMIT 3; where <> represents all the tables in the database? My goal is to get an idea of what each table contains, and the column names shown in DESCRIBE are not particularly useful. Therefore I would like to see 3 records from each table.
I know that I could easily script this in PHP by iterating over the output of show tables; however I am looking for a command to run on the MySQL interpreter (mysql> prompt).
It's described in detail under this link (haven't tried it myself though, it's just in my bookmarks):
http://www.youdidwhatwithtsql.com/mysql-clone-of-sp_msforeachtable/624
DELIMITER $$
DROP PROCEDURE IF EXISTS `usp_mysql_foreachtable`$$
CREATE PROCEDURE `usp_mysql_foreachtable`(IN sql_string VARCHAR(1000))
LANGUAGE SQL
NOT DETERMINISTIC
COMMENT 'Functional clone of sp_MsForEachTable'
BEGIN
DECLARE var_tablename VARCHAR(100);
DECLARE last_row BIT;
DECLARE table_cursor CURSOR FOR SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
AND TABLE_SCHEMA = DATABASE();
DECLARE CONTINUE HANDLER FOR NOT FOUND SET last_row = 1;
OPEN table_cursor;
FETCH table_cursor INTO var_tablename;
SET last_row = 0;
SET #var = '';
lbl_table_cursor: LOOP
SET #qry = REPLACE(sql_string, '?', var_tablename);
PREPARE q FROM #qry;
EXECUTE q;
DEALLOCATE PREPARE q;
FETCH table_cursor INTO var_tablename;
IF last_row = 1 THEN
LEAVE lbl_table_cursor;
END IF;
END LOOP lbl_table_cursor;
CLOSE table_cursor;
END$$
DELIMITER ;
Then you call it with
CALL usp_mysql_foreachtable('SELECT * FROM ? LIMIT 3;');
I have this:
SELECT TABLE_NAME, COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE column_name = 'whatever'
but what I need is something like this:
SELECT TABLE_NAME, COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE column_data = 'whatever'
So, in words, I have a value and I have no idea where it's stored. Is there a way to literally check the entire database and return the table, column?
aaaand yes, I know, db admins wouldn't be happy!
This might get you going in the right direction.
1. Create find_column stored procedure
DROP PROCEDURE IF EXISTS `find_column`;
DELIMITER $$
CREATE PROCEDURE `find_column`(IN i_value varchar(200),
OUT o_columns varchar(2000),
OUT o_message varchar(500))
MAIN_BLOCK : BEGIN
DECLARE is_numeric boolean;
CHECK_NUMERIC : BEGIN
set is_numeric = i_value REGEXP '^(-|\\+){0,1}([0-9]+\\.[0-9]*|[0-9]*\\.[0-9]+|[0-9]+)$';
END CHECK_NUMERIC;
FIND_IT : BEGIN
DECLARE bNoMoreRows BOOLEAN DEFAULT FALSE;
DECLARE v_schema varchar(64);
DECLARE v_table varchar(64);
DECLARE v_column varchar(64);
DECLARE v_data_type varchar(64);
DECLARE v_count int;
-- all schemas, tables and columns in DB
DECLARE columns CURSOR FOR
select table_schema,table_name,column_name,data_type from information_schema.columns;
DECLARE EXIT HANDLER for SQLEXCEPTION set o_message := concat('Unexpected error while trying to find schema, table and column for value : ',i_value);
declare continue handler for not found set bNoMoreRows := true;
open columns;
set o_columns = "";
COLUMN_LOOP: loop
fetch columns
into v_schema,v_table,v_column,v_data_type;
if (
(v_data_type in ('int','bigint','tinyint','decimal','smallint','mediumint') and is_numeric=1)
or (v_data_type not in ('int','bigint','tinyint','decimal','smallint','mediumint') and is_numeric=0)
)
then
SET #dyn_sql=CONCAT('select count(*) into #c from `',v_schema,'`.`',v_table,'` where `',v_column,'`=?');
SET #c = 0;
SET #v_value = i_value;
PREPARE stmt FROM #dyn_sql;
EXECUTE stmt using #v_value;
DEALLOCATE PREPARE stmt;
SET v_count = #c;
if v_count > 0 then
if length(o_columns <= 1800) then
set o_columns = concat(o_columns,",",v_schema,".",v_table,".",v_column);
end if;
end if;
end if;
if bNoMoreRows then
set o_columns = substring(o_columns,2);
close columns;
leave COLUMN_LOOP;
end if;
END loop COLUMN_LOOP;
END FIND_IT;
END MAIN_BLOCK$$
DELIMITER ;
2. Call find_column stored procedure with your value
call `find_column`('whatever',#columns,#message);
3. Check out the results
select #columns;
The is_numeric bit is lovingly ripped-off JBB's answer from this post.
It ain't perfect (what happens if the number of columns that your value exists exceeds 10 or so? If that is the case then this will only return the first 10 or so columns (depends on how long the schema.table.column name string is).
Hopefully it'll get you going in the correct direction.
An you're right. You're DB admins will be unhappy with you. But if you don't annoy them once in a while then you're not trying hard enough IMHO ;-)
Good luck.
I'm performing some database clean up and have noticed that there are a lot of columns that have both empty strings and NULL values in various columns.
Is it possible to write an SQL statement to update the empty strings to NULL for each column of each table in my database, except for the ones that do not allow NULL's?
I've looked at the information_schema.COLUMNS table and think that this might be the place to start.
It's not possible to do this with one simple SQL statement.
But you can do it using one statement for each column.
UPDATE TABLE SET COLUMN = NULL
WHERE LENGTH(COLUMN) = 0
or, if you want to null out the items that also have whitespace:
UPDATE TABLE SET COLUMN = NULL
WHERE LENGTH(TRIM(COLUMN)) = 0
I don't think it's possible within MySQL but certainly with a script language of your choice.
Start by getting all tables SHOW TABLES
Then for each table get the different columns and find out witch ones allow null, either with DESC TABLE, SHOW CREATE TABLE or SELECT * FROM information_schema.COLUMNS, take the one you rather parse
Then for each column that allows null run a normal update that changes "" to null.
Prepare to spend some time waiting :)
I figured out how to do this using a stored procedure. I'd definitely look at using a scripting language next time.
DROP PROCEDURE IF EXISTS settonull;
DELIMITER //
CREATE PROCEDURE settonull()
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE _tablename VARCHAR(255);
DECLARE _columnname VARCHAR(255);
DECLARE cur1 CURSOR FOR SELECT
CONCAT(TABLE_SCHEMA, '.', TABLE_NAME) AS table_name,
COLUMN_NAME AS column_name
FROM information_schema.COLUMNS
WHERE IS_NULLABLE = 'YES'
AND TABLE_SCHEMA IN ('table1', 'table2', 'table3');
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO _tablename, _columnname;
IF done THEN
LEAVE read_loop;
END IF;
SET #s = CONCAT('UPDATE ', _tablename, ' SET ', _columnname, ' = NULL WHERE LENGTH(TRIM(', _columnname, ')) = 0' );
PREPARE stmt FROM #s;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP;
CLOSE cur1;
END//
DELIMITER ;
CALL settonull();