Lets make it simple: I need counts of all columns from a table. Is it possible to loop through a table names instead of typing each COUNT(column_name) for every single column (if there are >20 columns)?
SELECT COUNT(column_1) AS column_1,
COUNT(column_2) AS column_2,
COUNT(column_3) AS column_3,
COUNT(column_4) AS column_4
FROM table
It would be nice to know a general principle, which could be applied not only for COUNT() but in other situations as well.
You can create the query using dynamic SQL in a stored procedure. You get the column names from the INFORMATION_SCHEMA.COLUMNS table.
SET #cols = (
SELECT GROUP_CONCAT('COUNT(`', column_name, '`) AS `', column_name, '`'))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'table');
PREPARE #stmt FROM CONCAT('SELECT ', #cols, ' FROM table');
EXECUTE #stmt;
You can use the INFORMATION_SCHEMA.COLUMNS to get all the column names of a table. Then You can use procedure or anonymous block to loop through all columns and store the results in some variable or insert into some table
CREATE PROCEDURE curdemo()
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE col_names CURSOR FOR
SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'tbl_name'
ORDER BY ordinal_position;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN col_names;
read_loop: LOOP
FETCH col_names INTO my_col;
IF done THEN
LEAVE read_loop;
END IF;
--Your Operation goes here
--select count(mycol) insert into yourtab
END LOOP;
CLOSE cur1;
END;
I'm not really sure what you mean. COUNT() returns the number of records and that would be the same for each column in a table. A row is a row.
Is it the number of columns in the table you are asking for? If so, one way is to use the INFORMATION_SCHEMA.COLUMNS like this:
SELECT COUNT(*) FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = N'your_table_name'
Related
Database is mysql/mariaDB.
I have database designed to store monthly reports about something. and their names are (example): table1, table2, table3...
I want to create function/procedure that will create/recreate view that contains all tables union (union ALL).
Something like:
1. first select all table names from information schema.
SELECT TABLE_NAME from information_schema.`TABLES` where TABLE_NAME like 'table%'
then i would try to set it in some loop to use result set from first query.
But i have problem with first step where i try to merge only one fixed table + one from first query and it returns error to me.
i try:
select * from `table4`
UNION
SELECT * from (SELECT TABLE_NAME from information_schema.`TABLES`
where TABLE_NAME like 'table%' limit 1) as dd
it returns me error: The used SELECT statements have a different number of columns ,
but when i execute sub query i get 1 result with correct name of table, and when i set that name in from clause without sub query, it works.
Any idea why it is happening, and maybe some advice how to accomplish that dynamic union.
I think a little push will help you to the correct way of handling this problem.
First, as Tim Biegeleisen suggests, the way to proceed is to use dynamic SQL, this is your only avenue if the table names cannot be absolutely determined before you try to run the query.
Second, you are correct to think that you need to start by querying the information_schema.TABLE, which you should do using a CURSOR. The results from that query should then be used to build up a query string which you then PREPARE and EXECUTE.
Third, I take it that the error message you included in your post refers specifically to the running of that query and doesn't indicate that the monthly tables differ in any way. You can't do a UNION unless the results from each part return the same number of columns.
Fourth, because we are going to build the query dynamically, this has to done within a stored procedure, it's not possible to do it in a stored function.
There are good tutorials in the mysql docs for using CURSOR and PREPARE/EXECUTE, which you should read. The version I give below will be based on those examples. I'm assuming the only input parameter will be the schema name (in case you happen to have some similarly named tables in another database on the server).
DELIMITER //
DROP PROCEDURE IF EXISTS dyn_union //
CREATE PROCEDURE dyn_union(IN v_sname VARCHAR(64))
READS SQL DATA
BEGIN
-- NB the order of declaration for variables cursor
-- and handler must be strictly observed
DECLARE sname VARCHAR(64); -- variable the schema names
DECLARE tname VARCHAR(64); -- variable the table names
DECLARE done INT DEFAULT FALSE; -- cursor control variable
DECLARE cur1 CURSOR FOR
SELECT table_schema, table_name
FROM information_schema.TABLES
WHERE table_schema = v_sname
AND table_name LIKE 'table%';
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
SET #sql = ''; -- build the query string in this var
OPEN cur1;
read_loop: LOOP -- loop over the rows returned by cursor
FETCH cur1 INTO sname, tname; -- fetching the schema and table names
IF done THEN
LEAVE read_loop;
END IF;
IF #sql = '' THEN -- build the select statement
SET #sql := CONCAT('SELECT * FROM `', sname, '`.`', tname, '`');
ELSE
SET #sql := CONCAT(#sql, ' UNION ALL SELECT * FROM `', sname, '`.`', tname, '`');
END IF;
END LOOP;
CLOSE cur1;
select #sql;
PREPARE stmt FROM #sql; -- prepare and execute the dynamically
EXECUTE stmt; -- created query.
DEALLOCATE PREPARE stmt;
END //
DELIMITER ;
-- call the procedure
CALL dyn_union('your_db_name');
I want a procedure that will search all columns for non keyboard ascii characters (Dec 16 to Dec 31 or DLE to US) and update the column by replacing them with a space ' ' or nothing ''.
I have a SELECT statement that is finding the rows I need to update, but I have to manually change all columns myself.
SELECT column_name
FROM table_name
WHERE column_name REGEXP '[[.DLE.]-[.US.]]'
Here's the UPDATE script for modifying the column values
UPDATE table
SET
column = replace(column,char(16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31), '')
I want those two to be fused together to a single function or stored procedure but I don't know how, since I'm just starting to learn MySQL.
Disclaimer
Between using REGEXP and CURSORs to loop through each table and column, these examples are not going to be lightning fast. The speed will obviously vary depending on your environment and I suggest testing them out on development before production
One column in one table
To search a single column on a single table, you basically had the UPDATE as you needed it.
UPDATE t1
SET
column_name = replace(column_name,
char(16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31), '')
WHERE column_name REGEXP '[[.DLE.]-[.US.]]'
All columns in one table
To do all columns in a table, you need to identify the table, then loop through the columns using a cursor
DELIMITER $$
CREATE PROCEDURE table_regexp_replace(in_table VARCHAR(128))
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE search_column VARCHAR(64);
DECLARE cur1 CURSOR FOR
SELECT DISTINCT `COLUMN_NAME` FROM `information_schema`.`COLUMNS`
WHERE `TABLE_NAME` = in_table ORDER BY `ORDINAL_POSITION` ;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur1;
read_loop: LOOP
-- Process the next column
FETCH cur1 INTO search_column;
-- If we're done, stop the loop
IF done THEN
LEAVE read_loop;
END IF;
-- Replace everything in this column matching the regexp
SET #new_query := CONCAT ('UPDATE ', in_table,
' SET `', search_column, '` = replace(', search_column,
', char(16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31), \'\')
WHERE ', search_column, ' REGEXP \'[[.DLE.]-[.US.]]\'') ;
PREPARE stmt FROM #new_query;
EXECUTE stmt ;
END LOOP;
CLOSE cur1;
END$$
DELIMITER ;
Then usage
CALL table_regexp_replace('my_table');
How it works
Looks convoluted, it's actually pretty straight forward.
We create a procedure with one parameter in_table which is used to specify the table to work with.
Setup a cursor that pulls the column names from the information_schema table, in their correct order
Loop through each of those columns, executing the manually created UPDATE statement against each one.
You'll notice anywhere in the UPDATE query that required quotes, they've had to be escaped using \.
\'[[.DLE.]-[.US.]]\'
All columns in all tables
You could then use this procedure in a loop through all tables, using a similar method to above. Below is how you'd pull all the table names from information_schema:
SELECT DISTINCT TABLE_NAME FROM information_schema.TABLES
WHERE TABLE_SCHEMA = 'your_database_name';
I'm reverse-engineering a MySQL database and I'd like to get a list of example values from every column in every table. I'd like to run a query like this:
select
table_name,
column_name,
group_concat(
(select distinct table_name.column_name limit 100)
separator ','
) as examples
from
information_schema.columns
where
table_schema = 'myschema'
;
I'd like the output to look something like this:
table1 column1 (123,124,234)
table1 column2 ('Joe','Sara','Bob')
MySQL won't accept table_name.column_name as valid syntax. What's the right way to write this query?
I think Sam, you are looking for something like that, or at least it would be a better approach:
select
table_name,
column_name,
group_concat((column_name) separator ',') as examples
from
information_schema.columns
where
table_schema = 'test'
GROUP BY table_name
;
Based on rene's suggestion, I wrote a stored procedure which outputs examples of values from each column in every table. It's ugly and slow, but it works. I'd welcome suggestions on how to improve this code.
DELIMITER //
CREATE PROCEDURE column_example_values(
IN db_name VARCHAR(64),
IN tbl VARCHAR(64),
IN col VARCHAR(64),
OUT result MEDIUMTEXT)
BEGIN
SET #s = CONCAT('SELECT GROUP_CONCAT(tbl1.',col,
' separator \',\') FROM (SELECT DISTINCT ',
col,' FROM ',db_name,'.',tbl,
' LIMIT 100) tbl1 INTO #result1');
PREPARE stmt FROM #s;
EXECUTE stmt;
SET result = IFNULL(#result1,'');
END;
//
DELIMITER ;
DELIMITER //
CREATE PROCEDURE all_columns_example_values(IN db_name VARCHAR(64))
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE tbl, col VARCHAR(64);
DECLARE cur1 CURSOR FOR
SELECT
table_name,
column_name
FROM
information_schema.columns
WHERE
table_schema = db_name
ORDER BY
table_name,
column_name;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
CREATE TEMPORARY TABLE results (
tbl VARCHAR(64), col VARCHAR(64), examples MEDIUMTEXT);
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO tbl, col;
IF done THEN
LEAVE read_loop;
END IF;
CALL column_example_values(db_name,tbl,col,#result);
INSERT INTO results (tbl, col, examples) VALUES (tbl, col, #result);
END LOOP;
CLOSE cur1;
SELECT * FROM results;
DROP TABLE results;
END;
//
DELIMITER ;
It can be called with
CALL all_columns_example_values('mydb');
I'm performing some database clean up and have noticed that there are a lot of columns that have both empty strings and NULL values in various columns.
Is it possible to write an SQL statement to update the empty strings to NULL for each column of each table in my database, except for the ones that do not allow NULL's?
I've looked at the information_schema.COLUMNS table and think that this might be the place to start.
It's not possible to do this with one simple SQL statement.
But you can do it using one statement for each column.
UPDATE TABLE SET COLUMN = NULL
WHERE LENGTH(COLUMN) = 0
or, if you want to null out the items that also have whitespace:
UPDATE TABLE SET COLUMN = NULL
WHERE LENGTH(TRIM(COLUMN)) = 0
I don't think it's possible within MySQL but certainly with a script language of your choice.
Start by getting all tables SHOW TABLES
Then for each table get the different columns and find out witch ones allow null, either with DESC TABLE, SHOW CREATE TABLE or SELECT * FROM information_schema.COLUMNS, take the one you rather parse
Then for each column that allows null run a normal update that changes "" to null.
Prepare to spend some time waiting :)
I figured out how to do this using a stored procedure. I'd definitely look at using a scripting language next time.
DROP PROCEDURE IF EXISTS settonull;
DELIMITER //
CREATE PROCEDURE settonull()
BEGIN
DECLARE done INT DEFAULT FALSE;
DECLARE _tablename VARCHAR(255);
DECLARE _columnname VARCHAR(255);
DECLARE cur1 CURSOR FOR SELECT
CONCAT(TABLE_SCHEMA, '.', TABLE_NAME) AS table_name,
COLUMN_NAME AS column_name
FROM information_schema.COLUMNS
WHERE IS_NULLABLE = 'YES'
AND TABLE_SCHEMA IN ('table1', 'table2', 'table3');
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN cur1;
read_loop: LOOP
FETCH cur1 INTO _tablename, _columnname;
IF done THEN
LEAVE read_loop;
END IF;
SET #s = CONCAT('UPDATE ', _tablename, ' SET ', _columnname, ' = NULL WHERE LENGTH(TRIM(', _columnname, ')) = 0' );
PREPARE stmt FROM #s;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP;
CLOSE cur1;
END//
DELIMITER ;
CALL settonull();
Is it possible to show the name of a table in a db where a specific value is present. I have different tables and i want to show only the table names that contains a specific value in any of the fields.
This will return lots of empty result sets, but the non-empty ones correspond to table/column combinations that fit your search. It only works for text, and detects columns that contain the value (as opposed to a full column match.)
DELIMITER |
DROP PROCEDURE IF EXISTS `SearchAllTables`|
CREATE PROCEDURE `SearchAllTables` (
IN _search varchar(256)
)
LANGUAGE SQL
DETERMINISTIC
SQL SECURITY DEFINER
BEGIN
-- declare stuff
declare _tableName varchar(64);
declare _columnName varchar(64);
declare _done tinyint(1) default 0;
-- we will examine every string column in the database
declare _columnCursor cursor for
select TABLE_NAME, COLUMN_NAME
from INFORMATION_SCHEMA.COLUMNS
where TABLE_SCHEMA = database()
and (DATA_TYPE like '%char%'
or DATA_TYPE like 'text');
declare CONTINUE handler for NOT FOUND
SET _done = 1;
OPEN _columnCursor;
LOOP1: LOOP
-- get the next table/column combination
FETCH _columnCursor INTO _tableName,_columnName;
IF _done = 1 THEN
CLOSE _columnCursor;
LEAVE LOOP1;
END IF;
-- query the current column to see if it holds the value
SET #query = concat(
"select '",_tableName,"' as TableName, '",
_columnName,"' as ColumnName
from ",_tableName,"
where ",_columnName," like concat('%',?,'%')
group by 1;"
);
SET #search = _search;
PREPARE _stmt FROM #query;
EXECUTE _stmt USING #search;
DEALLOCATE PREPARE _stmt;
END LOOP LOOP1;
END|
DELIMITER ;
Oh, yeah, and it's ugly... Maybe it'll help you, though!
SELECT TABLE_NAME
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'database_name'
AND COLUMN_NAME = 'column_name'