MySQL find column name where it's value match the pattern - mysql

How to search the entire database for column name equal to my rule and specific value as well.
Let's say that i want to search for column name like voucher where it's value contain that word value10
So far i can find the column name but i don't know how to match with value as well.
SELECT column_name FROM information_schema.columns
WHERE TABLE_SCHEMA = 'dbname' AND column_name LIKE '%voucher%'
So the end goal is to find any column name like voucher containing value10 within it's content.

Procedure code:
CREATE PROCEDURE search_tables ( IN column_pattern TEXT,
IN value_pattern TEXT )
BEGIN
SELECT GROUP_CONCAT (CONCAT( ' SELECT ''',
TABLE_NAME,
'.',
COLUMN_NAME,
''' AS `table.column`, ',
COLUMN_NAME,
' AS `value`\nFROM ',
TABLE_NAME,
'\nWHERE ',
COLUMN_NAME,
' LIKE ''',
value_pattern,
'''' )
SEPARATOR ' UNION ALL ')
INTO #query
FROM INFORMATION_SCHEMA.COLUMNS
WHERE column_name LIKE column_pattern
AND TABLE_SCHEMA = DATABASE();
PREPARE stmt FROM #query;
EXECUTE stmt;
DROP PREPARE stmt;
END
Test tables:
CREATE TABLE table1 (val1 VARCHAR(8), val2 TEXT);
INSERT INTO table1 VALUES
('a_01_a','b_11_b'),
('c_211_c','d_311_d'),
('e_55_e','f_00_f');
CREATE TABLE table2 (val3 CHAR(6), field4 VARCHAR(64));
INSERT INTO table2 VALUES
('x_1123','ghjghj_11_tyuyu'),
('8901_t','sdf_SDF_sdf');
Call:
CALL search_tables('%val%', '%11%');
Output:
table.column value
table1.val1 c_211_c
table1.val2 b_11_b
table1.val2 d_311_d
table2.val3 x_1123
fiddle

Create a stored procedure to loop through meta data table INFORMATION_SCHEMA to fetch all tables with column_name of choice. Further dynamic SQL is used to scan each of the tables retrieved for columns having the value of choice.
DDL and DML for setting the data for testing :
create table TESTA(COLMNA char(255),COLMNC char(255));
create table TESTB(COLMNA char(255),COLMNB char(255));
create table TESTC(COLMND char(255),COLMNA char(255));
insert into TESTA values('value0','someothercolmn');
insert into TESTB values('value0','anothersomeothercolmn');
insert into TESTB values('value1','Yetanothercolumn');
Test is to search all tables having column_name as COLMNA with value as value0. The procedure will accept column_name and column_Value, hence can be used across the database, just need to pass values as appropriate.
CREATE PROCEDURE Findtables( colmn_name VARCHAR(64),colmn_value VARCHAR(64) )
BEGIN
DECLARE tablename CHAR(64);
DECLARE c1 CURSOR FOR
SELECT table_name
FROM information_Schema
WHERE column_name = colmn_name;
OPEN c1;
lable_loop:LOOP
FETCH c1 INTO tablename;
select tablename;
SET #sql = CONCAT('SELECT * FROM ', tablename, ' WHERE ',colmn_name,' = "',colmn_value ,'" ;');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP lable_loop;
CLOSE c1;
END;
Call the stored procedure :
CALL Findtables('COLMNA','value0');
Output :
tablename
TESTA
COLMNA COLMNC
value0 someothercolmn
tablename
TESTB
COLMNA COLMNB
value0 anothersomeothercolmn
tablename
TESTC
COLMND COLMNA
Demonstration of the solution can be found in DBFIDDLE link [https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=4888a6160faf97fb75665832d6610293][1]
PS : I have to create INFORMATION_SCHEMA table in dbfiddle as metadata tables are not accessible.

Related

how to get data from dynamic table in sql

How can I get data from chosen table of my database?
I'm going to work with database in c# application and I have the database includes that tables:
MyTable1;
MyTable2;
...
And I have tbl variable that is equal to tbl = "MyTable2";. I want to execute the code as following:select * from tbl
I try to execute this code:
SELECT *
FROM (
SELECT TABLE_NAME
FROM information_schema.tables
WHERE TABLE_NAME = 'MyTable1'
);
But the code returned error that Every derived table must have its own alias
I want to get all data from table whose name is equal to my variable (tbl) and its value can also be changed. How can I do it?
You might be able to do this using a prepared statement in MySQL:
SELECT TABLE_NAME
INTO #table
FROM information_schema.tables
WHERE TABLE_NAME = 'MyTable1';
SET #query = CONCAT('SELECT * FROM ', #table);
PREPARE stmt FROM #query;
EXECUTE stmt;
SELECT *
FROM (
SELECT TABLE_NAME
FROM information_schema.tables
WHERE TABLE_NAME = 'MyTable1'
) AS Blah
Try this:
DECLARE #SELECT nvarchar(500)
SET #SELECT = 'SELECT * FROM ' + #tbl
EXECUTE sp_executesql #SELECT

MySql table into JSON format

I would like to convert like 100 tables from MySql to MongoDb
So I think that the easiest way will be to import the data as JSON
Any generic query to convert MySql data to JSON format without using PHP /ruby/Python?
SELECT CONCAT(CONCAT('{"CompanyId":', company_id),"}") AS JSON FROM company;
Try this:
SET #schema = 'test_db';
SET #table = 'test';
SELECT CONCAT(
'SELECT CONCAT(TRIM(TRAILING ', QUOTE(','), ' FROM CONCAT(', QUOTE('{'), ',',
GROUP_CONCAT(QUOTE('"'), ',', QUOTE(COLUMN_NAME), ',',
QUOTE('"'), ',', QUOTE(':'), ',', QUOTE('"'), ',', COLUMN_NAME, ',',
QUOTE('"'),',', QUOTE(',')),
')), ''}'') FROM ', #table
)
INTO #qry FROM
(SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS c
WHERE TABLE_SCHEMA = #schema AND TABLE_NAME = #table) t;
SELECT #qry;
PREPARE stmt FROM #qry;
EXECUTE stmt;
Above query generates table data in JSON format. Using the query you can prepare a stored procedure taking database and table names as input parameters and populate your data. If you want to exclude some of the columns in a table, just modify the query which selects data from INFORMATION_SCHEMA database like SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS c WHERE TABLE_SCHEMA = #schema AND TABLE_NAME = #table AND COLUMN_NAME NOT IN (**columns to exclude**).
I extended oardic's answer to create a mysql procedure to generate json from table name along with null check. Here key is set to value null instead of empty string.
create procedure createJSON (IN tabelSchema varchar(255), IN tableName varchar(255), IN whereCondition varchar(255), OUT outJson longtext)
BEGIN
DROP TEMPORARY TABLE IF EXISTS RowJson;
SELECT CONCAT(
'CREATE TEMPORARY TABLE RowJson AS SELECT CONCAT(TRIM(TRAILING ', QUOTE(','), ' FROM CONCAT(', QUOTE('{'), ',',
GROUP_CONCAT(QUOTE('"'), ',', QUOTE(COLUMN_NAME), ',',
QUOTE('"'), ',', QUOTE(':'), ',', 'IF(ISNULL(', COLUMN_NAME, '), "null", CONCAT(',QUOTE('"'),",", COLUMN_NAME, ",", QUOTE('"'),')),',
QUOTE(',')),
')), ''}'') as json FROM ', tableName, ' WHERE ', whereCondition
)
INTO #qry FROM
(SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS c
WHERE TABLE_SCHEMA = tabelSchema AND TABLE_NAME = tableName) t;
PREPARE stmt FROM #qry;
EXECUTE stmt;
SELECT json into outJson FROM RowJson;
END;

How to select column name in select statement

I want to copy/update data from Table A to Table B. Table B has some more additional columns. I have tried the following options.
1) `REPLACE INTO `B` (SHOW FIELDS FROM 'A') SELECT * FROM `A
2) `REPLACE INTO `B`
(SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS`
WHERE `TABLE_SCHEMA`='test1' AND `TABLE_NAME`='A') SELECT * FROM `A
But it throws errors. Can you guys help me how to select names with select query?
UPDATE:
3) As suggested by Jerko,
I have two tables A(warehouse_id,long,lat) B(warehouse_id,long)
Applied the following statement.
SET #query = CONCAT('REPLACE INTO `A` (SELECT ',
(SELECT GROUP_CONCAT(CONCAT('`',column_name, '`'))
FROM information_schema.columns
WHERE `TABLE_SCHEMA`='test2' AND `table_name` = 'A'),
' FROM `B`)');
PREPARE stmt FROM #query;
EXECUTE stmt;
This gives me the error
"#1054 - Unknown column 'lat' in 'field list' "
You can't do this dynamically in mysql like you are trying to do. MySQL expects your list of column names to be provided directly, not from a subquery.
If you want to do this dynamically you'll have to step back upstream to whatever language you are using to interact with MySQL such as PHP or Java.
Have you tried this?
insert into B(col1, . . ., coln)
select col1, . . ., coln
from A;
That is, list the fields from A in the select clause. List the corresponding columns for B in the insert column list.
If you need the list of columns, get them from INFORMATION_SCHEMA.COLUMNS and cut-and-paste into the query.
Actually there is a way
SET #query = CONCAT('REPLACE INTO `A` (',
(SELECT GROUP_CONCAT(CONCAT('`',column_name, '`'))
FROM information_schema.columns
WHERE `TABLE_SCHEMA`='test1' AND `table_name` = 'A'
AND column_name IN (SELECT column_name FROM information_schema.columns WHERE table_schema = 'test1' AND table_name='B')) ,
') (SELECT ',
(SELECT GROUP_CONCAT(CONCAT('`',column_name, '`'))
FROM information_schema.columns
WHERE `TABLE_SCHEMA`='test1' AND `table_name` = 'A'
AND column_name IN (SELECT column_name FROM information_schema.columns WHERE table_schema = 'test1' AND table_name='B')),
' FROM `B`)');
PREPARE stmt FROM #query;
EXECUTE stmt;
Try this
INSERT INTO B (field1,field2,...,fieldN)
SELECT (field1,field2,...,fieldN) FROM A

Select column names whose entries are not null

I would like to have a list of those columns of a table that have at least one non-NULL data entries in them.
In other words, I would like to get the column names for which the following returns at least one entry:
SELECT DISTINCT column_name FROM table WHERE column_name IS NOT NULL
I tried the following:
SELECT column_name
FROM information_schema.columns
WHERE table_name = "table_name"
AND EXISTS (
SELECT DISTINCT column_name FROM table_name WHERE column_name IS NOT NULL
)
But this also returns the column names where all the entries are NULL.
So how do I get only those columns with non-NULL entries?
Create from the INFORMATION_SCHEMA.COLUMNS table a string that contains the SQL you wish to execute, then prepare a statement from that string and execute it.
The SQL we wish to build will look like:
SELECT 'column_a'
FROM table_name
WHERE `column_a` IS NOT NULL
HAVING COUNT(*)
UNION ALL
SELECT 'column_b'
FROM table_name
WHERE `column_b` IS NOT NULL
HAVING COUNT(*)
-- etc.
(One could omit the WHERE clause and substitute COUNT(*) for COUNT(column), but I think that might be less efficient on indexed columns).
This can be done using the following:
SET group_concat_max_len = 4294967295;
SELECT GROUP_CONCAT(
' SELECT ',QUOTE(COLUMN_NAME),
' FROM table_name',
' WHERE `',REPLACE(COLUMN_NAME, '`', '``'),'` IS NOT NULL',
' HAVING COUNT(*)'
SEPARATOR ' UNION ALL ')
INTO #sql
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'table_name';
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
See it on sqlfiddle.
Use this procedure this will print columns names of a table which have atleast one not null rows.
create or replace procedure list_col_notNull(tblName in varchar2)
as
lv_col_name varchar2(200);
lv_ctr number;
lv_sql varchar2(400);
CURSOR cur_col_name is
SELECT column_name
FROM USER_TAB_COLUMNS U
WHERE table_name = tblName order by column_name asc;
begin
open cur_col_name;
LOOP
FETCH cur_col_name INTO lv_col_name;
EXIT WHEN cur_col_name%NOTFOUND;
lv_sql := 'select count(1) From ' || tblName || ' where ' || lv_col_name || ' is not null' ;
EXECUTE IMMEDIATE lv_sql into lv_ctr;
if lv_ctr > 0
then
dbms_output.put_line(lv_col_name);
end if;

MySQL sorting table by column names

I have already built a table with field names in arbitrary order. I want those field names to be in alphabetical order so that I can use them in my dropdown list. Is it possible with a query?
Select columns from a specific table using INFORMATION_SCHEMA.COLUMNS and sort alphabetically with ORDER BY:
SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = '[schemaname]'
AND table_name = '[tablename]'
ORDER BY column_name
Note: The following code will alter the specified table and reorder the columns in alphabetical order
This should do the trick. It's a bit messy and lengthy, and you'll have to change the database name and table name, but for this one, the only requirement is that there is a database named "test" and that you are running these commands in it:
Let's create the tables we need:
-- CREATE TESTING TABLE IN A DATABASE NAMED "test"
DROP TABLE IF EXISTS alphabet;
CREATE TABLE alphabet (
d varchar(10) default 'dee' not null
, f varchar(21)
, e tinyint
, b int NOT NULL
, a varchar(1)
, c int default '3'
);
-- USE A COMMAND STORAGE TABLE
DROP TABLE IF EXISTS loadcommands;
CREATE TABLE loadcommands (
id INT NOT NULL AUTO_INCREMENT
, sqlcmd VARCHAR(1000)
, PRIMARY KEY (id)
);
Now let's create the two stored procedures required for this to work:
Separating them since one will be responsible for loading the commands, and including a cursor to immediately work with it isn't plausible (at least for me and my mysql version):
-- PROCEDURE TO LOAD COMMANDS FOR REORDERING
DELIMITER //
CREATE PROCEDURE reorder_loadcommands ()
BEGIN
DECLARE limitoffset INT;
SET #rank = 0;
SET #rankmain = 0;
SET #rankalter = 0;
SELECT COUNT(column_name) INTO limitoffset
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet';
INSERT INTO loadcommands (sqlcmd)
SELECT CONCAT(t1.cmd, t2.position) AS commander FROM (
SELECT #rankalter:=#rankalter+1 AS rankalter, CONCAT('ALTER TABLE '
, table_name, ' '
, 'MODIFY COLUMN ', column_name, ' '
, column_type, ' '
, CASE
WHEN character_set_name IS NOT NULL
THEN CONCAT('CHARACTER SET ', character_set_name, ' COLLATE ', collation_name, ' ')
ELSE ' '
END
, CASE
WHEN is_nullable = 'NO' AND column_default IS NULL
THEN 'NOT NULL '
WHEN is_nullable = 'NO' AND column_default IS NOT NULL
THEN CONCAT('DEFAULT \'', column_default, '\' NOT NULL ')
WHEN is_nullable = 'YES' THEN 'DEFAULT NULL '
END
) AS cmd
, column_name AS columnname
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet'
ORDER BY columnname
) t1
INNER JOIN (
SELECT #rankmain:=#rankmain+1 AS rownum, position FROM (
SELECT 0 AS rownum, 'FIRST' AS position
, '' AS columnname
UNION
SELECT #rank:=#rank+1 AS rownum, CONCAT('AFTER ', column_name) AS position
, column_name AS columnname
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet'
ORDER BY columnname
LIMIT limitoffset
) inner_table
) t2 ON t1.rankalter = t2.rownum
;
END//
DELIMITER ;
If anyone thinks/sees that I'm missing to include any important column attributes in the ALTER command, please hesitate not and mention it! Now to the next procedure. This one just executes the commands following the order of column id from the loadcommands table. :
-- PROCEDURE TO RUN EACH REORDERING COMMAND
DELIMITER //
CREATE PROCEDURE reorder_executecommands ()
BEGIN
DECLARE sqlcommand VARCHAR(1000);
DECLARE isdone INT DEFAULT FALSE;
DECLARE reorderCursor CURSOR FOR
SELECT sqlcmd FROM loadcommands ORDER BY id;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET isdone = TRUE;
OPEN reorderCursor;
read_loop:LOOP
FETCH reorderCursor INTO sqlcommand;
IF isdone THEN
LEAVE read_loop;
END IF;
SET #sqlcmd = sqlcommand;
PREPARE stmt FROM #sqlcmd;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP read_loop;
CLOSE reorderCursor;
END//
DELIMITER ;
The SQL is long, so if someone can point out ways (and has tested them) to make this shorter I'd gladly do it, but for now, this at least works on my end. I also didn't need to put dummy data in the alphabet table. Checking the results can be done using the SHOW... command.
The last part:
-- TO TEST; AFTER RUNNING DDL COMMANDS:
SHOW CREATE TABLE alphabet; -- SEE ORIGINAL ORDER
CALL reorder_loadcommands(); -- PREPARE COMMANDS
CALL reorder_executecommands(); -- RUN COMMANDS
SHOW CREATE TABLE alphabet; -- SEE NEW ORDER
Perhaps later on I could make reorder_loadcommands dynamic and accept table and schema parameters, but I guess this is all for now..