MYSQL How to UNION Multiple "Unknown" Tables - mysql

I have some History Tables with Structure of "TableNameYYMM"
How can I UNION all History Tables which Starts with "TableName"?
I need a search over all History Tables.
I tryed it with PREPARE Execute and DEALLOCATE.
But i get everytime a SQL Error.
SET group_concat_max_len = 2048;
SET #startQuery = (SELECT CONCAT('SELECT col1, col2, col3 FROM ',TABLE_SCHEMA,'.', TABLE_NAME)
FROM information_schema.tables
WHERE
ENGINE = 'MyISAM'
AND TABLE_NAME like 'hist%'
ORDER BY TABLE_NAME
LIMIT 0,1);
SET #subquery = (SELECT GROUP_CONCAT(#startquery, 'UNION ALL SELECT col1, col2, col3 FROM ',TABLE_SCHEMA,'.', TABLE_NAME)
FROM information_schema.tables
WHERE
ENGINE = 'MyISAM'
AND TABLE_NAME like 'hist%'
ORDER BY TABLE_NAME
LIMIT 1,1000);
PREPARE stmt1 FROM '? AS combinedTable';
EXECUTE stmt1 USING #subquery;
DEALLOCATE PREPARE stmt1;
On Part 1 (#startquery) i try to get the first part of the Query
"select xxx from table1"
On Part 2 (#subquery) I tried to get all unions (from table2-max 1000)
select xxx from table1
UNION ALL select xxx from table2
UNION ALL select xxx from table3
...
I hope someone have an idea about this problem.

/*
create table history1701 (id int,col1 int,col2 int);
create table history1702 (id int,col1 int,col2 int);
create table history1703 (id int,col1 int,col2 int);
create table history1704 (id int,col1 int,col2 int);
insert into history1701 values (1,1,1);
insert into history1702 values (2,2,2);
insert into history1703 values (3,3,3);
insert into history1704 values (4,4,4);
*/
SET #startQuery = (SELECT CONCAT('SELECT col1, col2 FROM ',TABLE_SCHEMA,'.', TABLE_NAME)
FROM information_schema.tables
WHERE
# ENGINE = 'MyISAM' AND
TABLE_NAME like 'hist%17%'
ORDER BY TABLE_NAME
LIMIT 0,1);
SET #subquery = (
SELECT group_CONCAT(' UNION ALL SELECT col1, col2 FROM ',TABLE_SCHEMA,'.', TABLE_NAME order by table_name separator ' ' )
FROM information_schema.tables
WHERE
TABLE_NAME like 'hist%17%'
and table_name <> (
select table_name from information_schema.tables WHERE
# ENGINE = 'MyISAM' AND
TABLE_NAME like 'hist%17%'
ORDER BY TABLE_NAME
LIMIT 0,1
)
);
select #startquery;
set #subquery = concat(#startquery,#subquery);
PREPARE stmt1 FROM #subquery;
EXECUTE stmt1 ;
DEALLOCATE PREPARE stmt1;
In your code The limit 1,1000 in the first set #subquery returns a null value (not good) and generates a comma after each union all (also not good). I have amended this to exlude the first history table and changed the group concat separator to a space and moved the order by to within the group_concat.

Related

How to get first and last partition name for each partition in MYSQL

I'm using MySQL 5.6.16
SELECT TABLE_NAME, partition_name, partition_ordinal_position FROM information_schema.PARTITIONS
WHERE table_name IN(
SELECT DISTINCT table_name FROM information_schema.PARTITIONS WHERE table_schema = 'cbb' AND partition_name IS NOT NULL
)
ORDER BY partition_ordinal_position asc
;
I want to get the partition_name from the first and the last partition_ordinal_position grouped by table_name.
Like this :
table_name
first_partition
last_partition
table1
P2020
P2025
table2
P2021
P2030
Test this:
SELECT DISTINCT
table_name,
FIRST_VALUE(partition_name) OVER (PARTITION BY table_name ORDER BY partition_ordinal_position ASC) first_partition,
FIRST_VALUE(partition_name) OVER (PARTITION BY table_name ORDER BY partition_ordinal_position DESC) last_partition
FROM information_schema.partitions
WHERE table_schema = 'cbb'
AND partition_name IS NOT NULL;
I'm using MySQL 5.6.16 – Lunartist
SELECT DISTINCT
t0.table_name,
( SELECT t1.partition_name
FROM information_schema.partitions t1
WHERE t0.table_schema = t1.table_schema
AND t0.table_name = t1.table_name
ORDER BY t1.partition_ordinal_position ASC
LIMIT 1) first_partition,
( SELECT t2.partition_name
FROM information_schema.partitions t2
WHERE t0.table_schema = t2.table_schema
AND t0.table_name = t2.table_name
ORDER BY t2.partition_ordinal_position DESC
LIMIT 1) last_partition
FROM information_schema.partitions t0
WHERE t0.table_schema = 'cbb'
AND t0.partition_name IS NOT NULL;

Subquery in FROM clause using information_schema.tables

I'am trying to get all tables from a database where table name contains 'logs' and get the sum last value in each table of a column named flag.
Query I tried:
Select SUM(flag) FROM (SELECT table_name
FROM information_schema.tables
WHERE table_schema = 'db_test' AND table_name like '%logs') as c ORDER BY id DESC Limit 1;
But I'am having an issue with the subquery, I think the whole query is wrong.
I have broken this down into baby steps - nothing to stop you adjusting to taste.
drop table if exists onelog,twolog;
create table onelog (id int,flag int);
create table twolog (id int,flag int);
insert into onelog values (1,10),(2,1);
insert into twolog values (1,20),(2,1);
set #sql =
(
select group_concat(
concat('select id,flag from '
,tname, ' where id = (select max(id) from ', tname, ') union all'
)
)
from
(
select table_name tname from information_schema.tables where table_name like '%log' and table_schema = 'sandbox'
) s
)
;
set #sql = substring(#sql,1, length(#sql) - 10);
set #sql = replace(#sql,'union all,','union all ');
set #sql = concat('select sum(flag) from (', #sql , ' ) s');
#select #sql;
prepare sqlstmt from #sql;
execute sqlstmt;
deallocate prepare sqlstmt;
+-----------+
| sum(flag) |
+-----------+
| 2 |
+-----------+
1 row in set (0.001 sec)

mySQL query across multiple schemas

I have a multi-schema structure (same table across all schemas).
Here is how I can fetch all the schemas:
select table_schema from information_schema.tables where table_name = 'messages' and TABLE_TYPE = 'BASE TABLE'
This query is for a join between 'messages' table across two tables-
select *, '1' as customer from customer_1.messages union select *, '2' as customer from customer_2.messages
How do I use the perform the above query for all the schemas from the first query? Need not be a join. Could be anything that helps me return the results across all schemas.
Note: There could be 5000-10000 schemas
DELIMITER ##;
CREATE PROCEDURE join_all_data()
BEGIN
SET #counter := 0;
SELECT GROUP_CONCAT('select *, ',
#counter := #counter + 1,
' as customer from ',
table_schema,
'.messages'
SEPARATOR ' UNION ALL ')
FROM information_schema.tables
WHERE table_name = 'messages'
AND table_type = 'BASE TABLE'
INTO #sql;
PREPARE stmt FROM #sql;
EXECUTE stmt;
DROP PREPARE stmt;
END
##;
DELIMITER ;
CALL join_all_data;

Count all MySQL tables with similar column name

I have a stored procedure that should execute the sample code:
IF EXISTS (
SELECT TABLE_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME IN ('TB1', 'TB2', 'TB3')
AND COLUMN_NAME = 'COL'
AND table_schema = 'TestDB'
GROUP BY TABLE_NAME
HAVING count (TABLE_NAME) = 3
)
THEN
/*Used for debugging*/
SELECT 'Tables EXIST' INTO msg;
ELSE
/*Used for debugging*/
SELECT 'NON-EXISTENT TABLES !' INTO msg;
END IF
​
Basically, I want the above snippet to select all specified tables that has a column named 'COL'. I only want to do this using information_schema, nothing else.
Upon check, the test statement (IF EXISTS) always returned false. That is, the ELSE statement always execute even though my three test tables (TB1, TB2 and TB3) have the COL field. What sin am I commiting ?
You could do it as a function like this:
Function:
CREATE DEFINER=`root`#`localhost` FUNCTION `new_function`() RETURNS varchar(64) CHARSET utf8
BEGIN
declare cnt int;
declare msg varchar(64);
select count(*) from (
select distinct
table_name
from information_schema.columns
where table_name in ('TB1', 'TB2', 'TB3')
and column_name = 'COL'
and table_schema = 'TestDB'
) tbls into cnt;
if cnt = 3 then
select 'tables exist' into msg;
else
select 'tables do not exists' into msg;
end if;
RETURN msg;
END
And then run this:
create schema TestDB;
use TestDB;
create table TB1(
COL int
);
create table TB2(
COL int
);
create table TB3(
COL int
);
select new_function();
This will return:
Tables Exist
Something parallel to this should work
select distinct
table_name
from information_schema.columns
where column_name = 'CHARACTER_SET_NAME'
and table_name in ('CHARACTER_SETS', 'COLLATIONS', 'COLUMNS', 'TRIGGERS', 'FOO', 'BAR')
;
where you have this
SELECT TABLE_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME IN ('TB1', 'TB2', 'TB3')
AND COLUMN_NAME = 'COL'
AND table_schema = 'TestDB'
GROUP BY TABLE_NAME
HAVING count (TABLE_NAME) = 3
)
i.e. this:
select distinct
table_name
from information.schema.columns
where table_name in ('TB1', 'TB2', 'TB3')
and column_name = 'COL'
and table_schema = 'TestDB'
and if you want to make sure all three tables exist and have a 'COL' column write a conditional to check that this evaluates to 3
select count(*) from (
select distinct
table_name
from information.schema.columns
where table_name in ('TB1', 'TB2', 'TB3')
and column_name = 'COL'
and table_schema = 'TestDB'
) tbls
;

resultset of particular query

I have this query. When I execute I got Query statement but I want result set of this query.
My query is:
SELECT CONCAT('select * from ', table_name, ' union all ') AS CUSTOMQUERY
FROM information_schema.tables
WHERE table_name LIKE '%custom%'
I get this result
select * from custom_70006 union all
select * from custom_704306 union all
select * from custom_700436 union all
select * from custom_7000643 union all
select * from custom_7000634 union all
select * from custom_700046 union all
select * from custom_700063 union all
select * from custom_700062 union all
But I want result set of this particular column with corresponding data and last union all should remove.
Please help me with relevant query.
I think this is what you are looking for:
SET GLOBAL group_concat_max_len = 4294967295;
SELECT #query1 := GROUP_CONCAT(CONCAT('SELECT * FROM ', table_name) SEPARATOR
' UNION ALL ') AS CUSTOMQUERY
FROM INFORMATION_SCHEMA.tables
WHERE TABLE_SCHEMA = SCHEMA()
AND table_name LIKE '%custom%';
PREPARE stmt FROM #query1; EXECUTE stmt; DEALLOCATE PREPARE stmt;
Example: SQLFiddle
or modify your query as:
SELECT #query1 := CONCAT('select * from ', table_name, ' union all ') AS
CUSTOMQUERY
FROM information_schema.tables
WHERE table_name LIKE '%custom%'
to remove last 'union all' string from your query:
SET #query1 = TRIM(TRAILING 'union all' FROM TRIM(#query1));
PS: Default value of group_concat_max_len is only 1024. So you need to set it to higher value otherwise the output of your query will get stripped and you may get syntax error.