I have a mysql database with hundreds of tables. I want to convert this into a single table, where the table name will be added to the records, meaning that for each row / data in a particular table, I will add a new field, where in this new field the value will be the table name. Is there any way to do this automatically, rather than I have to create a new field on every table and insert the table name manually, this will take year. The record that I have on this db is around 400000 records. Can mysqldump with a script do this? or any other way ?
My suggestion is to generate syntax of the form:
create table xxx as
select t.*, 'table1' as TableName from table1 t union all
select t.*, 'table2' as TableName from table2 t union all
. . .
You can create the select statements using:
select concat('select t.*, ''', Table_Name, ''' as TableName from ', Table_Name, ' union all '
from Information_Schema.Tables
You can then put the create table line in front and remove the final union all.
Related
I have a big data raw(bronze) table with ~400 columns. In preparation for this table moving forward to other level tables in prepared (or silver level), I am picking up, let's say, 395 columns from the raw table; however, I don't like to type the name of all 399 columns in my SQL query.
Is there any solution in SQL to save some time?
Instead of
SELECT col1, col2, col3, ..., col395 FROM table
something like
SELECT * EXCEPT col400 FROM table
SELECT CONCAT_WS(' ',
'SELECT',
GROUP_CONCAT(column_name),
'FROM database_name.table_name') query_text
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = 'database_name'
AND table_name = 'table_name'
AND column_name NOT IN ('excess_column_1', 'excess_column_2', ...);
Insert your database and table names, fill the list of the columns to be excluded, execute the query - and it will produce needed query text.
You may convert this to the stored procedure which composes and executes needed query dynamically and call this SP instead of the query.
First do
EXPLAIN SELECT * FROM tbl;
SHOW WARNINGS;
Then edit the output to remove the column(s) you don't want.
(Next time, think about whether it is wise to have that many columns in a table; 400 is very high.)
I want to create a table / view from variable multiple table names that I get from a SELECT query.
It's possible to create a table from multiple known tables like so:
CREATE TABLE new_table AS
SELECT column_1, column_2
FROM clients_1, clients_2, ... clients_n;
To get list of tables I can use something like:
SELECT DISTINCT table_name FROM information_schema.columns WHERE table_name like '%clients_%';
Which returns:
table_name
1 clients_1
2 clients_2
How can I use the table names result as a list in CREATE TABLE FROM clause?
I tried something like this with a WITH:
WITH mytable AS
(SELECT DISTINCT table_name FROM information_schema.columns WHERE table_name like '%clients_%')
CREATE TABLE new_table AS
SELECT column_1, column_2
FROM mytable;
But it's mostly not working.
And even if it does, for example by not using WITH and selecting * columns -
CREATE TABLE new_table AS
SELECT *
FROM (SELECT DISTINCT table_name FROM information_schema.columns WHERE
table_name like '%clients_%');
new_table is just a copy of mytable/nested query.
Ideas?
Thanks!
I ended up going with a VIEW like so:
CREATE OR REPLACE VIEW clients_all AS
SELECT column_1, column_2 FROM client1
UNION ALL
SELECT column_1, column_2 FROM client2
The downside is I'll have to update the view whenever adding new client_n table,
And explicitly specify the name of the tables rather than inferring them from a query,
But I probably had to break it into two queries anyway even if creating a table as originally intended.
I have a large database (~50,000 rows) with 20 columns, and I want to "split" the data based upon the values in the third column (called FEATURE_CLASS). The values of FEATURE_CLASS are all of type VARCHAR(), and I want to create however many tables I'd need to replace the single, large table with many smaller tables each entitled with whatever the original table's FEATURE_CLASS value was.
Not sure of the best way to go about this, I was thinking something along the lines of creating a temporary table which would serve as an index, each row carrying a unique value of FEATURE_CLASS, then to iterate over the temp table and perform copying operations for each row of the temp table. I'm not sure really where to go from here, any help/ideas would be appreciated. Thanks!
The basic idea is to create a result set of statements that will create the tables for you and populate with the correct data. Run the below to generate the statements. You can then execute these statements manually (copy/paste) or from within a script.
SQL Server Example
SELECT DISTINCT
'SELECT * INTO [' + TableName.FEATURE_CLASS + '] FROM TableName WHERE FEATURE_CLASS = ''' + TableName.FEATURE_CLASS + ''';'
FROM
TableName
If you have any special characters in the FEATURE_CLASS column, you might want to consider removing them in the script above to prevent table names that are either invalid or tough to work with.
For example:
...
'SELECT * INTO [' + REPLACE(TableName.FEATURE_CLASS, '.', '') + '] FROM TableName WHERE FEATURE_CLASS = ''' + TableName.FEATURE_CLASS + ''';'
...
MySQL Example
SELECT DISTINCT
CONCAT('CREATE TABLE `', DB1.FEATURE_CLASS,
'` AS SELECT * FROM DB1 WHERE FEATURE_CLASS = ''',
DB1.FEATURE_CLASS, ''';') AS statements
FROM DB1;
This will give you a MySQL command something like this:
CREATE TABLE `feature_class_value` AS
SELECT * FROM DB1
WHERE FEATURE_CLASS = 'feature_class_value';
Check out the MySQL docs for more info on CREATE TABLE SELECT options https://dev.mysql.com/doc/refman/5.7/en/create-table-select.html.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How can I merge two MySql tables?
I want to merge multiple tables that have the same structure and make one large table. The tables have similar names, so I want to use the LIKE statement. Can anyone tell me how I can do this?
The tables are very simple, each having an ID column and a few other columns, but there are a large amount of tables, all of which have names like 'TX-xxx', where 'TX' means Texas, and 'xxx' are the counties in Texas; you know there are more than 200 counties in Texas. (In fact, I have to do this for all the states.) So I want to use the statement "LIKE 'TX-___'".
Thanks!
You would have to give more information so we know exactly what you want but you could create a view
CREATE VIEW myViewName AS
select *
from table1
union all
select * from
table2
This way it would show the information from all your tables (and can be limited so in the selects to not show everything) and when table1, table2, etc are changed the view will reflect this. You can change it at anytime and fetch from it as you would a table:
select * from myViewName
Now for grabbing from specific tables I am not sure how you can do this in mysql though I have done it in tsql. This previous question would help you so you might have something like:
-- Create temporary table of varchar(200) to store the name of the tables. Depending on how you want to go through the array maybe an id number (int).
insert into tempTableName (name)
SELECT table_name FROM information_schema.tables WHERE table_schema = 'database_name' and table_name like 'TX_%';
declare #sqlQuery varchar(max)
--Then you will want to loop through the array and build up an sql statement
-- For each loop through:
if len(#sqlQuery) = 0 begin -- first time through
set #sqlQuery = 'select col1, col2, col3 from ' + currentTableName
end else begin -- second+ time through
set #sqlQuery = 'union all select col1, col2, col3 from ' + currentTableName
end
-- after the loop add the create view. Could double check it worked by checking length = 0 again
set #sqlQuery = 'CREATE VIEW myViewName AS ' + #sqlQuery
Once the query string is built up you will execute it with
PREPARE stmt FROM #sqlQuery;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
If I understand your question correctly UNION is what you need. Something like
SELECT field1, field2
FROM (
SELECT field1, field2 from table1
UNION
SELECT field1, field2 from table2
) all_tables
WHERE all_tables.field1 like "%whatever%
Assuming they have the same columns or similar:
insert into #table
Select * from (Select * from tbl1
Union
select * from tbl2
Union
select * from tbl3)
If they don't have the same number/type of columns then you should provide us with that information.
Current situation:
INSERT INTO othertbl
SELECT *
FROM tbl
WHERE id = '1'
So i want to copy a record from tbl to othertbl. Both tables have an autoincremented unique index. Now the new record should have a new index, rather then the value of the index of the originating record else copying results in a index not unique error.
A solution would be to not use the * but since these tables have quite some columns i really think it's getting ugly.
So,.. is there a better way to copy a record which results in a new record in othertbl which has a new autoincremented index without having to write out all columns in the query and using a NULL value for the index.
-hope it makes sense....-
Think you're gonna have to drop the * and specify the columns fella
If you're using SQL Server you could get a list of columns using
SELECT column_name+', ' from INFORMATION_SCHEMA.COLUMNS where table_name = 'tbl'
Building an insert statement using the result from the above should be easy.
You can dump the list of non-auto-increment columns for your table with this query, and then use it in the insert and select statements:
SELECT group_concat(column_name)
from INFORMATION_SCHEMA.COLUMNS
where table_schema = 'myschema'
and table_name = 'tbl'
and extra != 'auto_increment';
Try:
INSERT ...
SELECT *
FROM tbl
ON DUPLICATE KEY UPDATE `id`=NULL
http://dev.mysql.com/doc/refman/5.1/en/insert-select.html