Select column names whose entries are not null - mysql

I would like to have a list of those columns of a table that have at least one non-NULL data entries in them.
In other words, I would like to get the column names for which the following returns at least one entry:
SELECT DISTINCT column_name FROM table WHERE column_name IS NOT NULL
I tried the following:
SELECT column_name
FROM information_schema.columns
WHERE table_name = "table_name"
AND EXISTS (
SELECT DISTINCT column_name FROM table_name WHERE column_name IS NOT NULL
)
But this also returns the column names where all the entries are NULL.
So how do I get only those columns with non-NULL entries?

Create from the INFORMATION_SCHEMA.COLUMNS table a string that contains the SQL you wish to execute, then prepare a statement from that string and execute it.
The SQL we wish to build will look like:
SELECT 'column_a'
FROM table_name
WHERE `column_a` IS NOT NULL
HAVING COUNT(*)
UNION ALL
SELECT 'column_b'
FROM table_name
WHERE `column_b` IS NOT NULL
HAVING COUNT(*)
-- etc.
(One could omit the WHERE clause and substitute COUNT(*) for COUNT(column), but I think that might be less efficient on indexed columns).
This can be done using the following:
SET group_concat_max_len = 4294967295;
SELECT GROUP_CONCAT(
' SELECT ',QUOTE(COLUMN_NAME),
' FROM table_name',
' WHERE `',REPLACE(COLUMN_NAME, '`', '``'),'` IS NOT NULL',
' HAVING COUNT(*)'
SEPARATOR ' UNION ALL ')
INTO #sql
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'table_name';
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
See it on sqlfiddle.

Use this procedure this will print columns names of a table which have atleast one not null rows.
create or replace procedure list_col_notNull(tblName in varchar2)
as
lv_col_name varchar2(200);
lv_ctr number;
lv_sql varchar2(400);
CURSOR cur_col_name is
SELECT column_name
FROM USER_TAB_COLUMNS U
WHERE table_name = tblName order by column_name asc;
begin
open cur_col_name;
LOOP
FETCH cur_col_name INTO lv_col_name;
EXIT WHEN cur_col_name%NOTFOUND;
lv_sql := 'select count(1) From ' || tblName || ' where ' || lv_col_name || ' is not null' ;
EXECUTE IMMEDIATE lv_sql into lv_ctr;
if lv_ctr > 0
then
dbms_output.put_line(lv_col_name);
end if;

Related

MySQL find column name where it's value match the pattern

How to search the entire database for column name equal to my rule and specific value as well.
Let's say that i want to search for column name like voucher where it's value contain that word value10
So far i can find the column name but i don't know how to match with value as well.
SELECT column_name FROM information_schema.columns
WHERE TABLE_SCHEMA = 'dbname' AND column_name LIKE '%voucher%'
So the end goal is to find any column name like voucher containing value10 within it's content.
Procedure code:
CREATE PROCEDURE search_tables ( IN column_pattern TEXT,
IN value_pattern TEXT )
BEGIN
SELECT GROUP_CONCAT (CONCAT( ' SELECT ''',
TABLE_NAME,
'.',
COLUMN_NAME,
''' AS `table.column`, ',
COLUMN_NAME,
' AS `value`\nFROM ',
TABLE_NAME,
'\nWHERE ',
COLUMN_NAME,
' LIKE ''',
value_pattern,
'''' )
SEPARATOR ' UNION ALL ')
INTO #query
FROM INFORMATION_SCHEMA.COLUMNS
WHERE column_name LIKE column_pattern
AND TABLE_SCHEMA = DATABASE();
PREPARE stmt FROM #query;
EXECUTE stmt;
DROP PREPARE stmt;
END
Test tables:
CREATE TABLE table1 (val1 VARCHAR(8), val2 TEXT);
INSERT INTO table1 VALUES
('a_01_a','b_11_b'),
('c_211_c','d_311_d'),
('e_55_e','f_00_f');
CREATE TABLE table2 (val3 CHAR(6), field4 VARCHAR(64));
INSERT INTO table2 VALUES
('x_1123','ghjghj_11_tyuyu'),
('8901_t','sdf_SDF_sdf');
Call:
CALL search_tables('%val%', '%11%');
Output:
table.column value
table1.val1 c_211_c
table1.val2 b_11_b
table1.val2 d_311_d
table2.val3 x_1123
fiddle
Create a stored procedure to loop through meta data table INFORMATION_SCHEMA to fetch all tables with column_name of choice. Further dynamic SQL is used to scan each of the tables retrieved for columns having the value of choice.
DDL and DML for setting the data for testing :
create table TESTA(COLMNA char(255),COLMNC char(255));
create table TESTB(COLMNA char(255),COLMNB char(255));
create table TESTC(COLMND char(255),COLMNA char(255));
insert into TESTA values('value0','someothercolmn');
insert into TESTB values('value0','anothersomeothercolmn');
insert into TESTB values('value1','Yetanothercolumn');
Test is to search all tables having column_name as COLMNA with value as value0. The procedure will accept column_name and column_Value, hence can be used across the database, just need to pass values as appropriate.
CREATE PROCEDURE Findtables( colmn_name VARCHAR(64),colmn_value VARCHAR(64) )
BEGIN
DECLARE tablename CHAR(64);
DECLARE c1 CURSOR FOR
SELECT table_name
FROM information_Schema
WHERE column_name = colmn_name;
OPEN c1;
lable_loop:LOOP
FETCH c1 INTO tablename;
select tablename;
SET #sql = CONCAT('SELECT * FROM ', tablename, ' WHERE ',colmn_name,' = "',colmn_value ,'" ;');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP lable_loop;
CLOSE c1;
END;
Call the stored procedure :
CALL Findtables('COLMNA','value0');
Output :
tablename
TESTA
COLMNA COLMNC
value0 someothercolmn
tablename
TESTB
COLMNA COLMNB
value0 anothersomeothercolmn
tablename
TESTC
COLMND COLMNA
Demonstration of the solution can be found in DBFIDDLE link [https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=4888a6160faf97fb75665832d6610293][1]
PS : I have to create INFORMATION_SCHEMA table in dbfiddle as metadata tables are not accessible.

MySQL show empty or null table columns

I have a quite big table and I want to find out which columns are unused (either empty or NULL for all rows). Is there an easy way to do that either with a query or with phpMyAdmin?
Thanks in advance!
You can try using sum() on each field in your table. The value will be 'null' for field which has only null values.
select sum(id), sum(name), sum(field1), sum(field2) from play_table
For a string column
SELECT SUM(CASE WHEN column is null or COALESCE(LTRIM(RTRIM(column)),'') = ''
THEN 0 ELSE 1 END) AS ZERO_IF_COLUMN_IS_ALL_NULL
FROM TABLE
For a numeric column
SELECT SUM(CASE WHEN column is null THEN 0 ELSE 1 END) AS ZERO_IF_COLUMN_IS_ALL_NULL
FROM TABLE
Yes you have to do it for each column, but you can do multiple of these in a single select statement.
SELECT column, COUNT(*)
FROM table
GROUP BY column
Then you can see which values inhabit column in which number.
This should work, not tested though.
SELECT * FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'my_table'
AND column_default is null;
SHOW columns from "TableName";
The above commands display columns of the empty table and also its respective properties
{ex: Name having varchar(50) or int AUTO_INCREMENT (Primary Key)}
DELIMITER $$
CREATE PROCEDURE sp_empty()
BEGIN
SET #sql = NULL;
SELECT GROUP_CONCAT(
CONCAT('(SELECT GROUP_CONCAT(id) FROM Table1 WHERE `',
column_name, '` IS NULL ',
CASE
WHEN data_type IN('varchar', 'char')
THEN CONCAT('OR `', column_name, '` = ''''')
WHEN data_type IN('date', 'datetime', 'time')
THEN CONCAT('OR `', column_name, '` = 0')
ELSE ''
END, ')`', column_name, '`'))
INTO #sql
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = SCHEMA()
AND table_name = 'table1'
AND column_name NOT IN ('id')
GROUP BY table_name;
SET #sql = CONCAT('SELECT ', #sql);
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END$$
DELIMITER ;
to use this, which separate your null columns

How to select column name in select statement

I want to copy/update data from Table A to Table B. Table B has some more additional columns. I have tried the following options.
1) `REPLACE INTO `B` (SHOW FIELDS FROM 'A') SELECT * FROM `A
2) `REPLACE INTO `B`
(SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS`
WHERE `TABLE_SCHEMA`='test1' AND `TABLE_NAME`='A') SELECT * FROM `A
But it throws errors. Can you guys help me how to select names with select query?
UPDATE:
3) As suggested by Jerko,
I have two tables A(warehouse_id,long,lat) B(warehouse_id,long)
Applied the following statement.
SET #query = CONCAT('REPLACE INTO `A` (SELECT ',
(SELECT GROUP_CONCAT(CONCAT('`',column_name, '`'))
FROM information_schema.columns
WHERE `TABLE_SCHEMA`='test2' AND `table_name` = 'A'),
' FROM `B`)');
PREPARE stmt FROM #query;
EXECUTE stmt;
This gives me the error
"#1054 - Unknown column 'lat' in 'field list' "
You can't do this dynamically in mysql like you are trying to do. MySQL expects your list of column names to be provided directly, not from a subquery.
If you want to do this dynamically you'll have to step back upstream to whatever language you are using to interact with MySQL such as PHP or Java.
Have you tried this?
insert into B(col1, . . ., coln)
select col1, . . ., coln
from A;
That is, list the fields from A in the select clause. List the corresponding columns for B in the insert column list.
If you need the list of columns, get them from INFORMATION_SCHEMA.COLUMNS and cut-and-paste into the query.
Actually there is a way
SET #query = CONCAT('REPLACE INTO `A` (',
(SELECT GROUP_CONCAT(CONCAT('`',column_name, '`'))
FROM information_schema.columns
WHERE `TABLE_SCHEMA`='test1' AND `table_name` = 'A'
AND column_name IN (SELECT column_name FROM information_schema.columns WHERE table_schema = 'test1' AND table_name='B')) ,
') (SELECT ',
(SELECT GROUP_CONCAT(CONCAT('`',column_name, '`'))
FROM information_schema.columns
WHERE `TABLE_SCHEMA`='test1' AND `table_name` = 'A'
AND column_name IN (SELECT column_name FROM information_schema.columns WHERE table_schema = 'test1' AND table_name='B')),
' FROM `B`)');
PREPARE stmt FROM #query;
EXECUTE stmt;
Try this
INSERT INTO B (field1,field2,...,fieldN)
SELECT (field1,field2,...,fieldN) FROM A

Using a SELECT query as column name in MySQL

I would like to get the values of the auto_increment column in my table (example). The catch is however, that I don't have the name of the auto_increment field. I'm currently using the following query to determine the name of the field:
SELECT column_name FROM information_schema.columns WHERE table_name = 'example' AND extra = 'auto_increment' LIMIT 1;
I would now like to pass the result of this query, as a 'string' to my actual query, and get the value. If I would like to do this in one go, how would I do that, because the below query, which should give me all auto_increment values used, only yields the above result -namely the auto_increment column name.
SELECT (
SELECT column_name
FROM information_schema.columns
WHERE table_name = 'example'
AND extra = 'auto_increment'
LIMIT 1
) AS pri
FROM example
Any thoughts would be appreciated :)
Many Regards,
Andreas
Here is an example of how you would do this using prepare and execute:
SELECT #s := concat('select `', column_name, '` from example e')
FROM information_schema.columns
WHERE table_name = 'example' AND extra = 'auto_increment'
LIMIT 1
prepare stmt from #s;
execute stmt;
deallocate prepare stmt;

MySQL sorting table by column names

I have already built a table with field names in arbitrary order. I want those field names to be in alphabetical order so that I can use them in my dropdown list. Is it possible with a query?
Select columns from a specific table using INFORMATION_SCHEMA.COLUMNS and sort alphabetically with ORDER BY:
SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = '[schemaname]'
AND table_name = '[tablename]'
ORDER BY column_name
Note: The following code will alter the specified table and reorder the columns in alphabetical order
This should do the trick. It's a bit messy and lengthy, and you'll have to change the database name and table name, but for this one, the only requirement is that there is a database named "test" and that you are running these commands in it:
Let's create the tables we need:
-- CREATE TESTING TABLE IN A DATABASE NAMED "test"
DROP TABLE IF EXISTS alphabet;
CREATE TABLE alphabet (
d varchar(10) default 'dee' not null
, f varchar(21)
, e tinyint
, b int NOT NULL
, a varchar(1)
, c int default '3'
);
-- USE A COMMAND STORAGE TABLE
DROP TABLE IF EXISTS loadcommands;
CREATE TABLE loadcommands (
id INT NOT NULL AUTO_INCREMENT
, sqlcmd VARCHAR(1000)
, PRIMARY KEY (id)
);
Now let's create the two stored procedures required for this to work:
Separating them since one will be responsible for loading the commands, and including a cursor to immediately work with it isn't plausible (at least for me and my mysql version):
-- PROCEDURE TO LOAD COMMANDS FOR REORDERING
DELIMITER //
CREATE PROCEDURE reorder_loadcommands ()
BEGIN
DECLARE limitoffset INT;
SET #rank = 0;
SET #rankmain = 0;
SET #rankalter = 0;
SELECT COUNT(column_name) INTO limitoffset
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet';
INSERT INTO loadcommands (sqlcmd)
SELECT CONCAT(t1.cmd, t2.position) AS commander FROM (
SELECT #rankalter:=#rankalter+1 AS rankalter, CONCAT('ALTER TABLE '
, table_name, ' '
, 'MODIFY COLUMN ', column_name, ' '
, column_type, ' '
, CASE
WHEN character_set_name IS NOT NULL
THEN CONCAT('CHARACTER SET ', character_set_name, ' COLLATE ', collation_name, ' ')
ELSE ' '
END
, CASE
WHEN is_nullable = 'NO' AND column_default IS NULL
THEN 'NOT NULL '
WHEN is_nullable = 'NO' AND column_default IS NOT NULL
THEN CONCAT('DEFAULT \'', column_default, '\' NOT NULL ')
WHEN is_nullable = 'YES' THEN 'DEFAULT NULL '
END
) AS cmd
, column_name AS columnname
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet'
ORDER BY columnname
) t1
INNER JOIN (
SELECT #rankmain:=#rankmain+1 AS rownum, position FROM (
SELECT 0 AS rownum, 'FIRST' AS position
, '' AS columnname
UNION
SELECT #rank:=#rank+1 AS rownum, CONCAT('AFTER ', column_name) AS position
, column_name AS columnname
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet'
ORDER BY columnname
LIMIT limitoffset
) inner_table
) t2 ON t1.rankalter = t2.rownum
;
END//
DELIMITER ;
If anyone thinks/sees that I'm missing to include any important column attributes in the ALTER command, please hesitate not and mention it! Now to the next procedure. This one just executes the commands following the order of column id from the loadcommands table. :
-- PROCEDURE TO RUN EACH REORDERING COMMAND
DELIMITER //
CREATE PROCEDURE reorder_executecommands ()
BEGIN
DECLARE sqlcommand VARCHAR(1000);
DECLARE isdone INT DEFAULT FALSE;
DECLARE reorderCursor CURSOR FOR
SELECT sqlcmd FROM loadcommands ORDER BY id;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET isdone = TRUE;
OPEN reorderCursor;
read_loop:LOOP
FETCH reorderCursor INTO sqlcommand;
IF isdone THEN
LEAVE read_loop;
END IF;
SET #sqlcmd = sqlcommand;
PREPARE stmt FROM #sqlcmd;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP read_loop;
CLOSE reorderCursor;
END//
DELIMITER ;
The SQL is long, so if someone can point out ways (and has tested them) to make this shorter I'd gladly do it, but for now, this at least works on my end. I also didn't need to put dummy data in the alphabet table. Checking the results can be done using the SHOW... command.
The last part:
-- TO TEST; AFTER RUNNING DDL COMMANDS:
SHOW CREATE TABLE alphabet; -- SEE ORIGINAL ORDER
CALL reorder_loadcommands(); -- PREPARE COMMANDS
CALL reorder_executecommands(); -- RUN COMMANDS
SHOW CREATE TABLE alphabet; -- SEE NEW ORDER
Perhaps later on I could make reorder_loadcommands dynamic and accept table and schema parameters, but I guess this is all for now..