select column from multiple unknown tables - mysql

I have a lot of dynamically created nearly similar looking tables according to the scheme "prefix + number", eg "t1", "t2", "t343" etc. All those tables have a cross-table unique row named identifier that I like to select within one query:
SELECT
`identifier`
FROM
(
SELECT
`TABLE_NAME`
FROM
information_schema.TABLES
WHERE
`TABLE_NAME` LIKE 't%'
);
But this returns: ERROR 1248 (42000): Every derived table must have its own alias
EDIT: according to the comments I modified my query like this:
SELECT
A.identifier
FROM
(
SELECT
`TABLE_NAME` AS identifier
FROM
information_schema.TABLES
WHERE
`TABLE_NAME` LIKE 't%'
) A;
But this selects only the table names from the subquery but not the column identifier from these tables.

When you create the table dynamically, and you want to query all of them, you can create an SQL statement dynamically like:
select
group_concat(
concat(
'SELECT ',
'''',TABLE_SCHEMA,'.',TABLE_NAME,''',',
TABLE_SCHEMA,'.',TABLE_NAME,'.', COLUMN_NAME,
' FROM ',
TABLE_SCHEMA,'.',TABLE_NAME)
separator ' union all ')
from information_schema.`COLUMNS` c
where table_schema='test' -- the schema name where your tables are
and table_name regexp '^t[0-9]+$' -- table name starts with t and ends with number
and COLUMN_NAME = 'i' -- i choose `i` as the column to be selected
;
This will produce a SQL statement like:
select
'test.t1',
test.t1.i
from
test.t1
union all
select
'test.t2',
test.t2.i
from
test.t2
union all
select
'test.t3',
test.t3.i
from
test.t3
When putting all of this in a stored procedure, you can use PREPARE and EXECUTE to execute this created statement.
Above is just an example of an SQL statement, you should change it to your needs.

Related

Select query using the result (columns from the select from Info_schema)

I need to copy/ insert all values with a certain where clause from table A to table B (basically from Main tables to respective history tables).
I don't want to specify the column names as I want to create a generic approach which will be able to use for all the tables that will need the ingestion.
Unfortunately, the attributes in table A are not always in the same order as it is in tableB, so I can't use select * into #temp from TableA and then insert into tableB from #temp. Plus TableB has generic 3 sys columns which we are generated for audit purposes.
My idea was to use the Info schema to get the column names. Then somehow use the result to get all the values from the asking table and add on top the generic sys columns. Is it possible to do?
I got the column names by using Info schema.
Select
COLUMN_NAME
FROM INFORMATION_SCHEMA.columns
where TABLE_NAME = 'TableA'
The SYS columns are:
sys_date=Getdate ()
,sys_flag='1'
,sys_name=SYSTEM_USER
Use Dynamic Query SQL Server
Please change table Name accordingly.
Declare #Table_Name varchar(50)
SET #Table_Name ='LoginMst'
Declare #Query varchar(8000)
Declare #ColumnNames varchar (8000)
set #ColumnNames = ''
select #ColumnNames =
case when #ColumnNames = ''
then column_name
else #ColumnNames + coalesce(',' + column_name, '')
end
from information_schema.columns where Table_Name=#Table_Name
SET #Query='insert into '+#Table_Name+'_Log ('+#ColumnNames+',sys_date,sys_flag,sys_name'+')
select '+#ColumnNames+',Getdate(),''1'',SYSTEM_USER from '+ 'LoginMst'
--print #Query
Exec(#Query)
You will require iterating all the tables you wish to take backup of. You will require adding the where clause too.

ALTER TABLE ADD COLUMN and automatic COLUMNS list

I am looking to add a new column to table using the suggestions provided here and here
In essence, I would like the (fields_in_orig_table) to be populated automatically, and not having to enter them manually (have many columns and this changes from table to table):
CREATE TABLE games_new LIKE games_orig;
ALTER TABLE games_new ADD COLUMN location varchar(256);
INSERT INTO games_new (fields_in_orig_table) SELECT * FROM games_orig;
RENAME TABLE games_orig TO games_old, games_new TO games_orig;
DROP TABLE games_old;
My thought goes around this:
CREATE TABLE games_new LIKE games_orig;
ALTER TABLE games_new ADD COLUMN version varchar(256);
INSERT INTO games_new
(SELECT CONCAT(GROUP_CONCAT(column_name ORDER BY ordinal_position
SEPARATOR " , "), " ") AS columns
FROM information_schema.columns
WHERE table_schema = 'games' AND table_name = 'games_orig' )
SELECT * FROM games_orig;
RENAME TABLE games_orig TO games_old, games_new TO games_orig;
DROP TABLE games_old;
This gives me syntax error (near the Select concat....).
The original syntax to get comma delimited column listings is:
SELECT CONCAT("'", GROUP_CONCAT(column_name ORDER BY ordinal_position SEPARATOR "', '"), "'") AS columns
FROM information_schema.columns
WHERE table_schema = 'db_name' AND table_name = 'tbl_name'
In my query, I have removed the extra quotes, as I figure my query does not require quotes as part of the column listing.
What am I doing wrong here? Who could help, please?
INSERT INTO games_new
VALUES (SELECT CONCAT....
When inserting values in the table use INSERT INTO tablename VALUES (fields) instead of INSERT INTO tablename SET(fields).
CREATE TABLE games_new LIKE games_orig;
ALTER TABLE games_new ADD COLUMN version varchar(256);
INSERT INTO games_new
VALUES
(SELECT CONCAT(GROUP_CONCAT(column_name ORDER BY ordinal_position
SEPARATOR ' , '), ' ') AS columns
FROM information_schema.columns
WHERE table_schema = 'games' AND table_name = 'games_orig' )
SELECT * FROM games_orig;
RENAME TABLE games_orig TO games_old, games_new TO games_orig;
DROP TABLE games_old;
You haven't specified which column you want to insert into, because your nested query is returning only 1 value
INSERT INTO games_new (column_name_u_want_to_insert_value_into)
SELECT cast(concat(group_concat(column_name ORDER BY ordinal_position SEPARATOR " , "), " ") AS CHAR) AS columns
FROM information_schema.columns
WHERE table_name = 'games_orig';
also, if you are running all the statements together, add semicolon(;) for the insert query as well

Mysql How to only select from a column if the column exists

I need to be able to check if a column exists and if it does then I want to SELECT from it.
I am trying lots of different variations but I'm not even sure if this is possible.
Here's my latest attempt:
SELECT
IF (EXISTS (SELECT `Period` AS `Period` FROM myview), `PERIOD`,
IF (EXISTS (SELECT `Country` AS `COUNTRY` FROM myview),`COUNTRY` FROM myview ;
Any ideas?
EDIT
I had seen the other question on here: MySQL, Check if a column exists in a table with SQL
But I still struggle with the if statement. I can check to see if the column exists using the answer in the question above. But my question is - how to execute a select statement from that column if the result is found to be true.
EDIT 2
The answer below indicates that I should use the BEGIN and END statement and this makes sense. However, my query complains at the first line. It says 'unexpected IF' - can anybody confirm if this is the right syntax fro MYSQL?
if( exists (SELECT *
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'db_name'
AND TABLE_NAME = 'view_name'
AND COLUMN_NAME = 'column_name') )
begin
SELECT `column_name` FROM `view_name`
end
Thanks in advance.
This query will give you whether a column exists.
SELECT *
FROM information_schema.COLUMNS
WHERE
TABLE_SCHEMA = 'db_name'
AND TABLE_NAME = 'table_name'
AND COLUMN_NAME = 'column_name'
If you want to check if some columns exist then perform a select statement you need to first check your columns exist. Then perform the select:
if (exists (SELECT * FROM information_schema.COLUMNS WHERE TABLE_NAME = 'myview' AND COLUMN_NAME = 'Period') and exists (SELECT * FROM information_schema.COLUMNS WHERE TABLE_NAME = 'myview' AND COLUMN_NAME = 'Country'))
begin
select `Period`, `Country` from myview
end
If the IF condition is true, then you will execute anything inside the BEGIN and END.
I came across the same situation where I had some product tables created by sheets uploaded by users. Sometimes, the sheets did not have column named "obsolete", so I had to import all products from the sheet but not the obsolete ones.
I am not modifying my query based on the original question that was asked, but here is my solution:
SELECT
t2.model,
(
SELECT
COUNT(*) AS _count
FROM db.table t1
WHERE
`obsolete`=1
AND t1.model=t2.model
) AS `obsolete`
FROM (
SELECT
0 AS `obsolete`,
t3.model
FROM db.table t3
) t2
There are 2 most important parts in this query:
We are selecting 0 AS obsolete as dummy to fool MySql which will be used even if column does not exist when selecting COUNT(*).
We have named tables as t1 & t2 to match the column model as t1.model=t2.model.

Normalizing table data in mySQL procedurally

I have a large data set in a denormalized format. Here is an example of the column names:
foreign_key_ID, P1, P2, P3, P4, P5.... D1, D2, D3.... etc..
These fields all contain similar type of data.
I need to normalize this into my existing table structure:
insert into new_table (id, name, index)
select foreign_key_id, P1, 1
from denormalized_table;
But that means that I need to run separate queries for each field in my denormalized table, just changing a few things:
insert into new_table (id, name, index)
select foreign_key_id, P2, 2
from denormalized_table;
This is getting tedious considering how many of these fields I have.
Is there a way this can be automated into a single operation? I.e.: iterate through the fields (I don't mind creating a list of eligible fields once, somewhere), pull off the last digit of that field name (ie "1" in "P1" and "2" for "P2") use the field name and the extracted index # in the sub-select.
Here's a start:
SELECT column_name, substr(column_name,2) AS `index`
FROM information_schema.columns
WHERE table_schema = 'mydatabasename'
AND table_name = 'denormalized_table'
AND column_name REGEXP '^[PD][0-9]+$'
ORDER BY column_name
You can modify the select list in that statement, to have MySQL generate statements for you:
SELECT CONCAT('INSERT INTO new_table (id, name, `index`) SELECT foreign_key_id, '
,column_name,', ',substr(column_name,2)
,' FROM denormalized_table ;') AS stmt
FROM information_schema.columns
WHERE table_schema = 'mydatabasename'
AND table_name = 'denormalized_table'
AND column_name REGEXP '^[PD][0-9]+$'
ORDER BY column_name
The output from that would be a set of MySQL INSERT statements that you could then execute.
If the number of rows and total size of the data to be inserted is not too large, you could and you want to get the whole conversion done in "one operation", then you could generate a single INSERT INTO ... SELECT statement, using the UNION ALL operator. I would get the majority of the statement like this:
SELECT CONCAT('UNION ALL SELECT foreign_key_id, '
,column_name,', ',substr(column_name,2)
,' FROM denormalized_table ') AS stmt
FROM information_schema.columns
WHERE table_schema = 'mydatabasename'
AND table_name = 'denormalized_table'
AND column_name REGEXP '^[PD][0-9]+$'
ORDER BY column_name
I would take the output from that, and replace the very first UNION ALL with an INSERT INTO .... That would give me a single statement to run to get the whole conversion done.
hat you're looking for is Dynamic SQL. This is where you execute SQL statements that you can assemble programmatically. You can run any arbitrary SQL code that's in a string, as long as you're in a Stored Procedure. See this link: How To have Dynamic SQL in MySQL Stored Procedure
Basically, you can build a string using mySQL statements by iterating over a set of columns. You can use the SHOW COLUMNS syntax (see http://dev.mysql.com/doc/refman/5.0/en/show-columns.html) to return a collection then loop over that resultset and build your dynamic query string and execute that way.
SHOW COLUMNS FROM myTable WHERE Field NOT IN (pkey, otherFieldIDontWantToInclude)

Converting many table into one table mysql databae

I have a mysql database with hundreds of tables. I want to convert this into a single table, where the table name will be added to the records, meaning that for each row / data in a particular table, I will add a new field, where in this new field the value will be the table name. Is there any way to do this automatically, rather than I have to create a new field on every table and insert the table name manually, this will take year. The record that I have on this db is around 400000 records. Can mysqldump with a script do this? or any other way ?
My suggestion is to generate syntax of the form:
create table xxx as
select t.*, 'table1' as TableName from table1 t union all
select t.*, 'table2' as TableName from table2 t union all
. . .
You can create the select statements using:
select concat('select t.*, ''', Table_Name, ''' as TableName from ', Table_Name, ' union all '
from Information_Schema.Tables
You can then put the create table line in front and remove the final union all.