I was thinking about proper database design while going through some databases where out of 150,000 records some columns had only 5-20 values set. It got me thinking that columns with low utilization should be transferred to a pivot table and was hoping to run a report that would tell me which to evaluate.
I tried a foreach, but it isn't working for me. Any tips?
foreach('SELECT table_name,
column_name
FROM information_schema.columns
WHERE table_schema = "mydb"',
'SELECT ${2}, utilization
FROM mydb.${1}.${2}
LEFT JOIN
(
SELECT sum(secondary_email)/count(*) AS utilization
FROM (
SELECT
CASE
WHEN secondary_email IS NULL THEN 0
ELSE 1
END AS secondary_email
FROM offices ) AS c )
GROUP BY ${2} ')
I was hoping for a better answer, feel free to out do me here. Copy and paste the output here and remove the final UNION ALL
SELECT
CONCAT('SELECT ', QUOTE(tb), ', ',QUOTE(col),', sum(`has_value`)/count(*) AS utilization FROM (SELECT CASE WHEN `',col,'` IS NULL THEN 0 ELSE 1 END AS has_value FROM ',tb,' ) AS c UNION ALL') SearchSQL
FROM
(
SELECT table_schema db,table_name tb,column_name col FROM information_schema.columns
WHERE table_schema = 'myDB' AND
(column_type LIKE 'char(%' OR column_type LIKE 'varchar(%' OR column_type LIKE '%text')
) A
Related
I'm writing MySQL to get all unique IDs from all tables in a database.
The database has tables like record_20181201, record_20181202, ...
The tables are automatically generated based on date (all tables have the same schema, and one column name is visitorId).
The SQL query I made is like,
SELECT UNIQUE(visitorId) FROM databaseName.record_20181201;
I can only query one table at a time using this..
Is there a way to query all tables in the database and select all unique visitorIds there?
SELECT DISTINCT VisitorID FROM DBName.Table1
UNION
SELECT DISTINCT VisitorID FROM DBName.Table2
UNION
SELECT DISTINCT VisitorID FROM DBName.Table3
Try this!!!!!!Hope this helps..
You could generate a query string with UNION's from the INFORMATION_SCHEMA.
Then run that query, or put it in a view.
A UNION will return a unique combined result of the unioned queries.
While a UNION ALL would just stick the results together.
SELECT GROUP_CONCAT(Qry ORDER BY TblSchema, TblName SEPARATOR ' UNION ')
FROM
(
SELECT
TABLE_SCHEMA as TblSchema,
TABLE_NAME as TblName,
CONCAT('select visitorId from ',TABLE_SCHEMA,'.',TABLE_NAME,'\r\n') as Qry
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME LIKE 'record_201812%'
AND COLUMN_NAME = 'visitorId'
) Q;
Run below query you will get a single query that will give unique visitorId from all table.
SELECT
CONCAT('SELECT DISTINCT visitorId FROM (',
REPLACE(query_string, ',', ' UNION '),
') union_table') AS final_query
FROM
(SELECT
CONCAT(GROUP_CONCAT('SELECT visitorId FROM ', table_name)) AS query_string
FROM
information_schema.tables
WHERE
table_name LIKE 'record_%') table_a;
you will get below query that will fetch unique visitorId from all tables.
SELECT DISTINCT
visitorId
FROM
(
SELECT visitorId FROM record_20181201
UNION
SELECT visitorId FROM record_20181202
UNION
SELECT visitorId FROM record_20181203
) union_table
I have tried to export CSV from the MySQL Database with the Header too. But I am getting the following error-
ERROR 1222 (21000): The used SELECT statements have a different number of columns
The MySQL Query is as below-
SELECT * FROM (
(SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'ORDERS'
AND TABLE_SCHEMA = 'REMOTE'
ORDER BY ORDINAL_POSITION)
UNION ALL
(SELECT * FROM ORDERS WHERE orderDate>='2018-05-07')) AS TBL
INTO OUTFILE 'C:/xampp/htdocs/WES/DBScript/SS_Orders.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n';
Usually you have to do it like so:
SELECT *
FROM (
select 'A', 'B', 'C'
union all
select a, b, c from table
) t
INTO OUTFILE [...]
Note that you'd have to ensure that all the column data types correctly match, meaning they will all need to be varchar or similar.
For the headers you could kind of automate them with something like:
SELECT * FROM (
SELECT MAX(CASE WHEN ORDINAL_POSITION = 1 THEN COLUMN_NAME END),
MAX(CASE WHEN ORDINAL_POSITION = 2 THEN COLUMN_NAME END),
MAX(CASE WHEN ORDINAL_POSITION = 3 THEN COLUMN_NAME END),
MAX(CASE WHEN ORDINAL_POSITION = 4 THEN COLUMN_NAME END),
MAX(CASE WHEN ORDINAL_POSITION = 5 THEN COLUMN_NAME END),
MAX(CASE WHEN ORDINAL_POSITION = 6 THEN COLUMN_NAME END),
[...]
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'ORDERS'
AND TABLE_SCHEMA = 'REMOTE'
UNION ALL
SELECT [...]
) t
INTO OUTFILE [...]
But you've still got to know the number of columns. If you want to do it dynamically, that essentially will require dynamic SQL.
It may be easier to just write a Python script to do it.
How can I get a table name by his position(row)? I got many tables.
For example in columns to find from a table it works this way:
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'database name here'
AND TABLE_NAME = 'table name here'
AND ORDINAL_POSITION = 2;
I need something like this only to find table name by their position(row) in the database.
Using MySQL.
Thanks.
If I understand you correctly, you need something like that
SELECT position, TABLE_NAME
FROM (
SELECT #row := #row +1 AS position, TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
JOIN (
SELECT #row :=0
)r
WHERE TABLE_SCHEMA = 'TABLE_SCHEMA here'
)tmp
WHERE position =5
and a different approach
SET #row =0;
SELECT TABLE_NAME
FROM (
SELECT #row := #row +1 AS position, TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'TABLE_SCHEMA here'
)tmp
WHERE position =5
Looks like you want something like that:
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'database name here'
AND TABLE_NAME = (
SELECT TABLE_NAME
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'database name here'
ORDER BY CREATE_TIME ASC
LIMIT 1 -- take one
OFFSET 1 -- after 1st row
)
This will return all colums from the table that was created as second table for that DB.
I have a table where most rows are sprinkled with NULLs....but I only want to match those particular rows that hold nothing but NULLs, except for 2 or 3 columns.
Something like
SELECT *
FROM sometable
WHERE
ALL(col1, col2, col3) IS NULL;
doesn't work.
Do I really have to write
WHERE
co1 IS NULL
AND
col2 IS NULL
...
AND col150 IS NULL
all the way out??
I am afraid there's an easier way in doing this.
But you can take help from
INFORMATION_SCHEMA.COLUMNS
table.
You can get all the column names of a table by this query.
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'YOUR_DATABASE_NAME'
AND TABLE_NAME = 'YOUR_TABLE_NAME';
You can modify this query to make it use in your case.
SELECT
SUBSTRING(
GROUP_CONCAT(
(
CASE
WHEN COLUMN_NAME NOT IN ('COL1','COLN') THEN
CONCAT(
COLUMN_NAME,
' IS NULL AND '
)
ELSE
''
END
) SEPARATOR ''
),
1,
LENGTH(
GROUP_CONCAT(
(
CASE
WHEN COLUMN_NAME NOT IN ('ID') THEN
CONCAT(
COLUMN_NAME,
' IS NULL AND '
)
ELSE
''
END
) SEPARATOR ''
)
) - 4
)
FROM
INFORMATION_SCHEMA. COLUMNS
WHERE
TABLE_SCHEMA = 'YOUR_DATABASE'
AND TABLE_NAME = 'YOUR_TABLE';
The above query will generate a string like 'COL2 IS NULL AND COL3 IS NULL'
Thus you can grab this string and put it in the where condition of your original query.
N:B: Mention only those columns IN THE WHERE CLAUSE [[CASE WHEN COLUMN_NAME NOT IN ('COL1','COLN')]] which are not supposed to check NULLABILITY.
Look at the query below. DB: ORACLE
select table_name, column_name, data_type from all_tab_cols
where
data_type = 'DATE' and
OWNER = 'OWNER_NAME'
O/P:
TABLE_NAME COLUMN_NAME DATA_TYPE
T1 C1 DATE
T1 C2 DATE
T2 C3 DATE
T2 C4 DATE
Now, I got the result perfectly. I want to build up a query which processes further. From the result about, I want to pickup the table_name, column_name and apply a filter on column_name.
Example:
TABLE: T1
C1 c2
01-01-2001 01-01-2011
02-02-1990 05-05-1700
03-03-1753 10-10-1764
Like wise another Table...
Simply speaking, I want all the columns from DB where year of date field less than a particular year. I've tried but couldn't able to do it.
I've got it..
This query does it..
select table_name
, column_name
,to_number(extractvalue(
xmltype(dbms_xmlgen.getxml(
'select count(*) c from '||owner||'.'||table_name ||' WHERE extract(year from ' || column_name || ') < 1753'
))
,'/ROWSET/ROW/C')) as count1
from all_tab_cols
where
data_type = 'DATE' and
OWNER = 'OWNER_NAME'