Count number of occurrences in SQL - mysql

I have a table with the following structure:
(table_name, column_name)
and for each row in this table I need to query the column_name in the table_name and do a COUNT(column_name) GROUP BY column_name of the values in there.
Currently I do
SELECT * FROM this table
/*and then*/ foreach row: do another query with:
SELECT column_name, COUNT(column_name) GROUP BY column_name
Is there any way I can do this in a single query?
Something like
SELECT column_name, COUNT(column_name)
GROUP BY column_name
FOREACH(SELECT table_name, column_name FROM my_initial_table)
I know the last query is not valid, it's just an example for what I am looking to to achieve if it's possible.
LE:
The table that tells me where to look has 2 varchar columns
Ex:
|++++++++++++++++++++++++++++++++
| table_name | column_name |
|+++++++++++++++++++++++++++++++|
| logsa | foo |
|===============================|
| logsa | bar |
|===============================|
| testx | baz |
|===============================|
This tells me that now i have too look in columns foo and bar of table logsa and column baz of table testx
Every column in every table has VARCHAR as datattype and i just need to count those that are the same. that's way i was doing
SELECT column_name, COUNT(column_name)
GROUP BY column_name

If you are working in MySql, you can't directly use parametrized column names.
There is an indirect way of doing this using stored procedures and prepared statements.
some sloppy first-draft code...
notice the difference between backticks ` and quotes '
CREATE PROCEDURE CountTables()
BEGIN
DECLARE done TINYINT DEFAULT 0;
DECLARE table_name varchar(30), colunn_name varchar(30);
DECLARE cur1 CURSOR FOR SELECT
table_name, column_name
FROM ColumnTable;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
CREATE TEMPORARY TABLE t1( table_name varchar(30), column_name varchar(30), count int);
OPEN cur1;
START TRANSACTION;
read_loop: LOOP
FETCH FROM cur1 INTO table_name, column_name;
IF done THEN LEAVE read_loop; END IF;
SET insert_sql = CONCAT( "INSERT INTO `t1`(`table_name`, `column_name`, `count`) SELECT ",
"'", table_name, "', '", column_name, "', count(`", column_name, "`)",
" FROM `", table_name, "`"
);
PREPARE insert_stmt FROM insert_sql;
EXECUTE insert_stmt;
DEALLOCATE PREPARE insert_stmt;
END LOOP;
COMMIT;
SELECT * FROM t1 GROUP BY column_name;
DROP TEMPORARY TABLE t1;
END;
Oh ya, don't forget to call the procedure:
CALL CountTables();

A subquery should be able to help you here. This is an example (untested) which should work with some refinement. I am going to call your table Schema for the purposes of my example, and in the subquery I am going to alias it as ColumnCount to make the code (hopefully) more readable.
SELECT Schema.table_name, Schema.column_name,
(SELECT COUNT(*) FROM Schema ColumnCount
WHERE Schema.column_name = ColumnCount.column_name) AS ColumnUsageCount
FROM Schema

No need to do a subselect, just do this, if there's only one table involved.
SELECT max(table_name) as table_name, column_name, COUNT(*) as occurrence
FROM initial_table
GROUP BY column_name
ORDER BY column_name
The Max(table_name) substitutes for the non-existent function Whatever(table_name).
In MySQL you can also use group_concat(table_name) as table_names. Try it out!
The ORDER BY is optional (and in MySQL not needed) but can be handy if you want it sorted by column name as well.
If you want to list the unique occurrences of the combi table_name + column_name do:
SELECT table_name, column_name, COUNT(*) as occurrence
FROM initial_table
GROUP BY table_name, column_name
ORDER BY table_name, column_name

in PHP
$result=mysql_query("SELECT column_name FROM $table_name");
$row_count = mysql_num_rows($result);
I'm not too familiar with mysql, but i think its along those lines

Related

MySQL find column name where it's value match the pattern

How to search the entire database for column name equal to my rule and specific value as well.
Let's say that i want to search for column name like voucher where it's value contain that word value10
So far i can find the column name but i don't know how to match with value as well.
SELECT column_name FROM information_schema.columns
WHERE TABLE_SCHEMA = 'dbname' AND column_name LIKE '%voucher%'
So the end goal is to find any column name like voucher containing value10 within it's content.
Procedure code:
CREATE PROCEDURE search_tables ( IN column_pattern TEXT,
IN value_pattern TEXT )
BEGIN
SELECT GROUP_CONCAT (CONCAT( ' SELECT ''',
TABLE_NAME,
'.',
COLUMN_NAME,
''' AS `table.column`, ',
COLUMN_NAME,
' AS `value`\nFROM ',
TABLE_NAME,
'\nWHERE ',
COLUMN_NAME,
' LIKE ''',
value_pattern,
'''' )
SEPARATOR ' UNION ALL ')
INTO #query
FROM INFORMATION_SCHEMA.COLUMNS
WHERE column_name LIKE column_pattern
AND TABLE_SCHEMA = DATABASE();
PREPARE stmt FROM #query;
EXECUTE stmt;
DROP PREPARE stmt;
END
Test tables:
CREATE TABLE table1 (val1 VARCHAR(8), val2 TEXT);
INSERT INTO table1 VALUES
('a_01_a','b_11_b'),
('c_211_c','d_311_d'),
('e_55_e','f_00_f');
CREATE TABLE table2 (val3 CHAR(6), field4 VARCHAR(64));
INSERT INTO table2 VALUES
('x_1123','ghjghj_11_tyuyu'),
('8901_t','sdf_SDF_sdf');
Call:
CALL search_tables('%val%', '%11%');
Output:
table.column value
table1.val1 c_211_c
table1.val2 b_11_b
table1.val2 d_311_d
table2.val3 x_1123
fiddle
Create a stored procedure to loop through meta data table INFORMATION_SCHEMA to fetch all tables with column_name of choice. Further dynamic SQL is used to scan each of the tables retrieved for columns having the value of choice.
DDL and DML for setting the data for testing :
create table TESTA(COLMNA char(255),COLMNC char(255));
create table TESTB(COLMNA char(255),COLMNB char(255));
create table TESTC(COLMND char(255),COLMNA char(255));
insert into TESTA values('value0','someothercolmn');
insert into TESTB values('value0','anothersomeothercolmn');
insert into TESTB values('value1','Yetanothercolumn');
Test is to search all tables having column_name as COLMNA with value as value0. The procedure will accept column_name and column_Value, hence can be used across the database, just need to pass values as appropriate.
CREATE PROCEDURE Findtables( colmn_name VARCHAR(64),colmn_value VARCHAR(64) )
BEGIN
DECLARE tablename CHAR(64);
DECLARE c1 CURSOR FOR
SELECT table_name
FROM information_Schema
WHERE column_name = colmn_name;
OPEN c1;
lable_loop:LOOP
FETCH c1 INTO tablename;
select tablename;
SET #sql = CONCAT('SELECT * FROM ', tablename, ' WHERE ',colmn_name,' = "',colmn_value ,'" ;');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP lable_loop;
CLOSE c1;
END;
Call the stored procedure :
CALL Findtables('COLMNA','value0');
Output :
tablename
TESTA
COLMNA COLMNC
value0 someothercolmn
tablename
TESTB
COLMNA COLMNB
value0 anothersomeothercolmn
tablename
TESTC
COLMND COLMNA
Demonstration of the solution can be found in DBFIDDLE link [https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=4888a6160faf97fb75665832d6610293][1]
PS : I have to create INFORMATION_SCHEMA table in dbfiddle as metadata tables are not accessible.

MySQL Select columns where the column name contains a substring

I have a table with a lot of fields about a person and then several recommendations of other people.
They are named:
"recommendation_1_name" "recommendation_1_company" 'recommendation_1_contact"
"recommendation_2_name" "recommendation_2_company" "recommendation_2_contact"
and so on.
I am trying to come up with a statement that allows me to only get the recommendations.
I imported an excel file into the table so it's just one large table.
This is what I have and it is returning an Empty set.
select * from questionnaire where 'COLUMN_NAME' like '%recommendation%';
I've been playing around with it making a table with only the recommendation fields and it still doesn't return anything.
Mysql: select recommendation_1_name, recommendation_2_name etc... from (table) where (USER) = (USERID) or however you can uniquely identify that user.
This Query generates you dynamic a SELECT query with all fields like 'recommendation%'. You only must setup the Databasename, and the Tablename. You can directly query the result of my query or add the WHERE clause.
SELECT
CONCAT( 'SELECT ',
GROUP_CONCAT(COLUMN_NAME SEPARATOR ',\n')
)
FROM information_schema.columns
WHERE TABLE_SCHEMA = 'DBNAME'
AND TABLE_NAME = 'TABLENAME'
AND COLUMN_NAME LIKE 'recommendation%';
You really need to normalize your schema.
But just as an experiment and example for some other cases (maybe somebody really need it). Here is solution to get this case resolved using stored procedure:
CREATE PROCEDURE `get_recommendations`()
BEGIN
DECLARE Q VARCHAR(100);
DECLARE C_NAME VARCHAR(100);
DECLARE cur CURSOR FOR SELECT GROUP_CONCAT(column_name) as `columns`
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'test'
AND TABLE_NAME ='questionnaire'
AND COLUMN_NAME LIKE '%recommendation%'
;
SET Q = 'SELECT ';
OPEN cur;
FETCH cur INTO C_NAME;
SET Q = CONCAT(Q,C_NAME,' ');
CLOSE cur;
SET #Q = CONCAT(Q,'FROM questionnaire;');
PREPARE stmt FROM #Q;
EXECUTE stmt ;
END
Don't forget to replace TABLE_SCHEMA = 'test' with your real database name.

How to write MySQL to get count of distinct values in a table?

Suppose database.tbl contains two fields: fruit and price.
Row 1: fruit='apple', price=0.60.
Row 2: fruit='pear', price=0.60.
Row 3: fruit='peach', price=0.50.
I want to run a query which returns the count of distinct values for each field, i.e. it should return
fruit 3
price 2
because there are 3 fruits, but only 2 distinct prices.
I've tried
SELECT C.COLUMN_NAME, COUNT(DISTINCT C.COLUMN_NAME) FROM tbl JOIN (SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA='database' AND TABLE_NAME='tbl') AS C GROUP BY C.COLUMN_NAME;
but this returns:
fruit 1
price 1
The only way I know to do it is to take the column names from the INFORMATION_SCHEMA query and put them in a temp table. Then iterate over the column names in the temp table and execute dynamic sql to query the distinct values in those columns. Here is an example in T-SQL, so you'll have to make some changes.
CREATE TABLE #values
(
COLUMN_NAME VARCHAR(100),
DISTINCT_COUNT INT
)
DECLARE #column AS VARCHAR(100) = ''
SELECT COLUMN_NAME INTO #columns FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA='dbo' AND TABLE_NAME='tbl'
SELECT TOP 1#column = COLUMN_NAME
FROM #columns
WHERE COLUMN_NAME > #column
ORDER BY COLUMN_NAME ASC
WHILE (##ROWCOUNT > 0)
BEGIN
DECLARE #sql AS NVARCHAR(1000) = N'INSERT INTO #values (COLUMN_NAME, DISTINCT_COUNT) SELECT ''' + #column + ''', COUNT(DISTINCT ' + #column + N') FROM [db].[dbo].[tbl]'
EXECUTE sp_executesql #sql
SELECT TOP 1 #column = COLUMN_NAME
FROM #columns
WHERE COLUMN_NAME > #column
ORDER BY COLUMN_NAME ASC
END
SELECT * FROM #values
DROP TABLE #columns
DROP TABLE #values
This is pretty rough and just an example of how it can be done. If this were production code, I'd make sure the variable types are all consistent and make sure the column names are safe. This will help: How To have Dynamic SQL in MySQL Stored Procedure

Select column names whose entries are not null

I would like to have a list of those columns of a table that have at least one non-NULL data entries in them.
In other words, I would like to get the column names for which the following returns at least one entry:
SELECT DISTINCT column_name FROM table WHERE column_name IS NOT NULL
I tried the following:
SELECT column_name
FROM information_schema.columns
WHERE table_name = "table_name"
AND EXISTS (
SELECT DISTINCT column_name FROM table_name WHERE column_name IS NOT NULL
)
But this also returns the column names where all the entries are NULL.
So how do I get only those columns with non-NULL entries?
Create from the INFORMATION_SCHEMA.COLUMNS table a string that contains the SQL you wish to execute, then prepare a statement from that string and execute it.
The SQL we wish to build will look like:
SELECT 'column_a'
FROM table_name
WHERE `column_a` IS NOT NULL
HAVING COUNT(*)
UNION ALL
SELECT 'column_b'
FROM table_name
WHERE `column_b` IS NOT NULL
HAVING COUNT(*)
-- etc.
(One could omit the WHERE clause and substitute COUNT(*) for COUNT(column), but I think that might be less efficient on indexed columns).
This can be done using the following:
SET group_concat_max_len = 4294967295;
SELECT GROUP_CONCAT(
' SELECT ',QUOTE(COLUMN_NAME),
' FROM table_name',
' WHERE `',REPLACE(COLUMN_NAME, '`', '``'),'` IS NOT NULL',
' HAVING COUNT(*)'
SEPARATOR ' UNION ALL ')
INTO #sql
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'table_name';
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
See it on sqlfiddle.
Use this procedure this will print columns names of a table which have atleast one not null rows.
create or replace procedure list_col_notNull(tblName in varchar2)
as
lv_col_name varchar2(200);
lv_ctr number;
lv_sql varchar2(400);
CURSOR cur_col_name is
SELECT column_name
FROM USER_TAB_COLUMNS U
WHERE table_name = tblName order by column_name asc;
begin
open cur_col_name;
LOOP
FETCH cur_col_name INTO lv_col_name;
EXIT WHEN cur_col_name%NOTFOUND;
lv_sql := 'select count(1) From ' || tblName || ' where ' || lv_col_name || ' is not null' ;
EXECUTE IMMEDIATE lv_sql into lv_ctr;
if lv_ctr > 0
then
dbms_output.put_line(lv_col_name);
end if;

MySQL sorting table by column names

I have already built a table with field names in arbitrary order. I want those field names to be in alphabetical order so that I can use them in my dropdown list. Is it possible with a query?
Select columns from a specific table using INFORMATION_SCHEMA.COLUMNS and sort alphabetically with ORDER BY:
SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = '[schemaname]'
AND table_name = '[tablename]'
ORDER BY column_name
Note: The following code will alter the specified table and reorder the columns in alphabetical order
This should do the trick. It's a bit messy and lengthy, and you'll have to change the database name and table name, but for this one, the only requirement is that there is a database named "test" and that you are running these commands in it:
Let's create the tables we need:
-- CREATE TESTING TABLE IN A DATABASE NAMED "test"
DROP TABLE IF EXISTS alphabet;
CREATE TABLE alphabet (
d varchar(10) default 'dee' not null
, f varchar(21)
, e tinyint
, b int NOT NULL
, a varchar(1)
, c int default '3'
);
-- USE A COMMAND STORAGE TABLE
DROP TABLE IF EXISTS loadcommands;
CREATE TABLE loadcommands (
id INT NOT NULL AUTO_INCREMENT
, sqlcmd VARCHAR(1000)
, PRIMARY KEY (id)
);
Now let's create the two stored procedures required for this to work:
Separating them since one will be responsible for loading the commands, and including a cursor to immediately work with it isn't plausible (at least for me and my mysql version):
-- PROCEDURE TO LOAD COMMANDS FOR REORDERING
DELIMITER //
CREATE PROCEDURE reorder_loadcommands ()
BEGIN
DECLARE limitoffset INT;
SET #rank = 0;
SET #rankmain = 0;
SET #rankalter = 0;
SELECT COUNT(column_name) INTO limitoffset
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet';
INSERT INTO loadcommands (sqlcmd)
SELECT CONCAT(t1.cmd, t2.position) AS commander FROM (
SELECT #rankalter:=#rankalter+1 AS rankalter, CONCAT('ALTER TABLE '
, table_name, ' '
, 'MODIFY COLUMN ', column_name, ' '
, column_type, ' '
, CASE
WHEN character_set_name IS NOT NULL
THEN CONCAT('CHARACTER SET ', character_set_name, ' COLLATE ', collation_name, ' ')
ELSE ' '
END
, CASE
WHEN is_nullable = 'NO' AND column_default IS NULL
THEN 'NOT NULL '
WHEN is_nullable = 'NO' AND column_default IS NOT NULL
THEN CONCAT('DEFAULT \'', column_default, '\' NOT NULL ')
WHEN is_nullable = 'YES' THEN 'DEFAULT NULL '
END
) AS cmd
, column_name AS columnname
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet'
ORDER BY columnname
) t1
INNER JOIN (
SELECT #rankmain:=#rankmain+1 AS rownum, position FROM (
SELECT 0 AS rownum, 'FIRST' AS position
, '' AS columnname
UNION
SELECT #rank:=#rank+1 AS rownum, CONCAT('AFTER ', column_name) AS position
, column_name AS columnname
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'test'
AND table_name = 'alphabet'
ORDER BY columnname
LIMIT limitoffset
) inner_table
) t2 ON t1.rankalter = t2.rownum
;
END//
DELIMITER ;
If anyone thinks/sees that I'm missing to include any important column attributes in the ALTER command, please hesitate not and mention it! Now to the next procedure. This one just executes the commands following the order of column id from the loadcommands table. :
-- PROCEDURE TO RUN EACH REORDERING COMMAND
DELIMITER //
CREATE PROCEDURE reorder_executecommands ()
BEGIN
DECLARE sqlcommand VARCHAR(1000);
DECLARE isdone INT DEFAULT FALSE;
DECLARE reorderCursor CURSOR FOR
SELECT sqlcmd FROM loadcommands ORDER BY id;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET isdone = TRUE;
OPEN reorderCursor;
read_loop:LOOP
FETCH reorderCursor INTO sqlcommand;
IF isdone THEN
LEAVE read_loop;
END IF;
SET #sqlcmd = sqlcommand;
PREPARE stmt FROM #sqlcmd;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END LOOP read_loop;
CLOSE reorderCursor;
END//
DELIMITER ;
The SQL is long, so if someone can point out ways (and has tested them) to make this shorter I'd gladly do it, but for now, this at least works on my end. I also didn't need to put dummy data in the alphabet table. Checking the results can be done using the SHOW... command.
The last part:
-- TO TEST; AFTER RUNNING DDL COMMANDS:
SHOW CREATE TABLE alphabet; -- SEE ORIGINAL ORDER
CALL reorder_loadcommands(); -- PREPARE COMMANDS
CALL reorder_executecommands(); -- RUN COMMANDS
SHOW CREATE TABLE alphabet; -- SEE NEW ORDER
Perhaps later on I could make reorder_loadcommands dynamic and accept table and schema parameters, but I guess this is all for now..