I need to copy data from one table to another. The tables do not have all the same columns, or order; but the data to be copied is always in the same columns; that is data from column foo should be copied to columns foo.
If it was only two tables I could just hardcode the column names like:
INSERT INTO table_target ( column1, column2, column4 )
SELECT column1, column2, column4 FROM table_source;
However there are a couple dozen tables, and some extra transformation needs to be done, so it would be nice if I could just say: Copy any matching columns and ignore the rest.
I've managed to figure out how to get a list of the common columns, but now I'm stuck.
SELECT src.col
FROM (SELECT COLUMN_NAME as col
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'table_target') as trg
INNER JOIN
(SELECT COLUMN_NAME as col
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'table_source') as src ON (src.col=trg.col)
;
I trick I have used in the past to good effect is to write a query that returns SQL, then just copy-paste it into the db command shell. In this case, this would be your query:
SELECT CONCAT(
'INSERT INTO table_target (',
GROUP_CONCAT(trg.col), -- produces output like "col1, col2, col3"
') SELECT ',
GROUP_CONCAT(trg.col), -- produces output like "col1, col2, col3"
' FROM table_source;') as sql_stmt
FROM (
(SELECT COLUMN_NAME as col
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'table_target') as trg
INNER JOIN
(SELECT COLUMN_NAME as col
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'table_source') as src ON src.col=trg.col) x;
This makes use of mysql's handy GROUP_CONCAT function that aggregates the value into a CSV - perfect for creating a list of column names for generating SQL
I also wrapped your query in an alias so we can select from it.
Related
I have a table that have 200 columns and I want to select all columns that have
'completed' or 'incomplete' values. this table have 180 columns that can have 'completed' or 'incomplete' values and other columns are dates, signatures, ids, etc.
So The first idea is write a long query like ,
SELECT column2,column4,column5,...,column199 FROM table WHERE column2 like 'completed' or column2 like 'incomplete' or column4 like....
But this query is too simple and too long.
Is there any way to write something like below?
select
[Columns that have values like 'completed' or 'incomplete'( if the value is not them do not select them)]
from Table
Please someone help or any idea, Thanks
For this particular case, you can do:
where greatest(col1, col2, col3) = 'completed' and
least(col4, col5, col6) = 'incompleted'
The use of least() and greatest() is specifically because there are two values and is based on the ordering of the values.
I should note that the data structure is suspicious. You would be better served by having 180 rows for each of the statuses.
This select query should get you all the columns in your table.
SELECT column_name
FROM information_schema.columns
WHERE table_schema = 'your_db'
AND table_name = 'your_table'
ORDER BY ordinal_position
From the results, you can make a string that will concat the column names in whatever fashion you want. Then you'll end up with a string that will be generated and will have all the columns that you need and you wouldn't have to type all the column names individually. Then you can execute the string using the EXECUTE command. Here is a sample of how it would look like.
SET #s = CONCAT('SELECT * FROM your_table WHERE greatest(', SELECT GROUP_CONCAT(column_name)
FROM information_schema.columns
WHERE table_schema = 'your_db'
AND table_name = 'your_table'
GROUP BY table_name
ORDER BY ordinal_position, ') = ''completed'');
EXECUTE #s
Please keep in mind that the above example is not executable. The 's need to be adjusted properly to accommodate for string literals and such but this should be able to give you information about how to use dynamic sql to get the result that you need.
Let us say I have two tables with many columns so I do not want to name the column names explicitly in my query but i want to avoid duplicate names.
If I do:
CREATE TABLE new_table
SELECT a.*, b.*
FROM table1 a
INNER JOIN table2 b ON a.myID = b.myId
WHERE a.age > 10 and b.ice = 'melted'
I will get an error saying: duplicate column name myId, I could also get more errors if more column names in a and b are the same.
How can I avoid this issue by automatically adding a prefix to all column names in a.* and b.* w/o explicitly mentioning all the column names - very tedious to do so!
Thanks!
Unfortunately, you will have to list the columns in case table have matching column names. However, you can use information_schema to get the column names, format those and copy paste in the query to save the pain, e.g.:
SELECT GROUP_CONCAT(CONCAT('a.', COLUMN_NAME)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'schema' AND TABLE_NAME = 'table';
The above query should give you comma separated column names with a. prefix. You can then use the same query for table b, get the names out and use it in the main SELECT query.
Update
As #Uueerdo has rightly said, you can add alias to columns as well, e.g.:
SELECT GROUP_CONCAT(CONCAT('a.', COLUMN_NAME, ' AS a_', COLUMN_NAME))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'schema' AND TABLE_NAME = 'table';
In my experience ORMs will run an initial DESCRIBE query so it can do this sort of stuff for you once it has the column names. But if you insist on doing it dynamically in a single query, you could do this with pure MySQL:
-- config
SET #database = 'your_database';
SET #tableA = 'table1';
SET #tableB = 'table2';
-- table alias "a" columns
SET #fieldsA = NULL;
SELECT GROUP_CONCAT(CONCAT('a.', COLUMN_NAME), ' AS ',CONCAT('`a.', COLUMN_NAME,'`')) INTO #fieldsA
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = #database AND TABLE_NAME = #tableA;
-- table alias "b" columns
SET #fieldsB = NULL;
SELECT GROUP_CONCAT(CONCAT('b.', COLUMN_NAME), ' AS ',CONCAT('`b.', COLUMN_NAME,'`')) INTO #fieldsB
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = #database AND TABLE_NAME = #tableB;
-- some variables for readability
SET #fields = CONCAT(' ', #fieldsA, ',', #fieldsB,' ');
SET #tableAliasA = CONCAT(' ',#database, '.', #tableA,' a ');
SET #tableAliasB = CONCAT(' ',#database, '.', #tableB,' b ');
-- generate our final query
SET #query = CONCAT('CREATE TABLE new_table SELECT', #fields,
'FROM', #tableAliasA,
'INNER JOIN', #tableAliasB,
'ON a.myID = b.myId WHERE a.age > 10 and b.ice = ''melted''');
-- finally run the query:
PREPARE stmt1 FROM #query;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;
-- if you have problems with the above query, uncomment the following to get the query so you can run it separately
-- SELECT #query;
I'd strongly advise against using this sort of solution though. I'd sooner run an initial DESCRIBE query as earlier stated, then generate your query based on that. Another solution is to create a temporary table as a copy of the second table, then rename problematic columns, then proceed to join on it to produce the data you need to create your new_table. MySQL has no issues with result columns having the same name, the issue here is trying to create a table with two columns with the same name. So essentially what you're trying to do is a star select but excluding a column.
Another approach is to just select only the primary key from both:
SELECT a.myID as `aId`, b.myId as `bId` then create your table containing only that. Then if you ever need data from a particular table, just LEFT JOIN on it to grab the information you're looking for. You can take this a step further and set up a VIEW to do this sort of thing for you. VIEWs can join tables for you and make it very easy to select whatever columns you're looking for. You can also setup multiple VIEWs as well. Also note that views behave just like tables for the purpose of joins. You can JOIN a view with a table, or you can join a view with a view, etc.
So rather than do what you're trying to do -- creating a new table with the data from two other tables -- consider whether you're actually looking for a VIEW.
I have two tables.
First table have column called column_name which contains values columns names of the second table like ( column1 , column2 , etc).
I need to select columns from the second table depending on the result of query column_name from the first table .
I need help solving this issue but I cant develop it:
If You work with Oracle, i suggest You to use listagg function (http://www.oracle-developer.net/display.php?id=515)
And Query will be like:
Select column_name from all_tab_columns where table_name = name_of_your_table2 and column_name in (select listagg...)
if I understand correctly.
A Case in point is getting data from tables which is generated every day with new name like below:
select table_name from INFORMATION_SCHEMA.TABLES where table_name like 'ifhcraw%';
ifhcraw_2016_03_31_24
ifhcraw_2016_04_01_8
ifhcraw_2016_04_02_14
ifhcraw_2016_04_03_20
ifhcraw_2016_04_05_8
ifhcraw_2016_04_06_14
As you can see, there is a name convention based on rule - "ifhcraw+year+month+day+hour". But the hour of generation is not known.
Is there any way to create some SQL script which can get all data from the tables "where table_name like 'ifhcraw%'"
You can use GROUP_CONCAT to combine all the table names in a single string.
SELECT #queries := GROUP_CONCAT(CONCAT('SELECT * FROM ', table_name) SEPARATOR ' UNION ')
FROM INFORMATION_SCHEMA.TABLES
WHERE table_name LIKE 'ifhcraw%';
PREPARE stmt FROM #queries;
EXECUTE stmt;
The first query finds all the matching table names, and creates a query like SELECT * FROM <tablename>. Then it uses GROUP_CONCAT to connect them all with UNION, so the resulting query looks like:
SELECT * FROM table1
UNION
SELECT * FROM table2
UNION
SELECT * FROM table3
...
Note that by default GROUP_CONCAT is limited to returning 1024 characters. If you have lots of these tables, you'll need to increase group_concat_max_len to get everything.
I have written this simple query which would pull out all the data from table into a CSV file.
SELECT Group_concat(Concat(column_name))
FROM information_schema.columns
WHERE table_name = 'subject_assignment'
AND table_schema = 'newschema2'
UNION ALL
SELECT (SELECT Group_concat('`', column_name, '`')
FROM information_schema.columns
WHERE table_name = 'subject_assignment'
AND table_schema = 'newschema2')
FROM subject_assignment
INTO OUTFILE 'D:\\export\\asd.csv'
Now, the first bit works great but I have issues with the second part.
Instead of pulling out data from columns specified in column list it just displays me all the column names over and over again.
Could you please suggest what I am doing wrong?
Thanks.
In your second SELECT you do not select any column from subject_assignment. Instead, you're selecting single string value made from concatenated column names. And you're selecting it as many times as the row count of subject_assignment.
UPDATE:
If you want to dynamically create column names and then select data from them, see this: https://stackoverflow.com/a/17573774/925196