I need to fetch data from database for backup in the form of insert statements
I need to do it on a button click in c#. So i think an sql query or stored procedure will be appropriate to do this, rather than mysqldump.
Secondly I need them for all tables. Instead of writing table and column names. They should be fetched from information_schema, because the query will not need to be changed for different scema
If there already exists a solution, please guide me.
Update : I have prepared a solution, it is posted, still looking for the better one.
To get Data of whole database - SqlFiddle Demo
To get data of only one table - - SqlFiddle Demo
I have made a complex but acceptable solution. But needs improvement.
This is a complex procedure with complex coding especially the query which fetches all rows of all columns into a single result by group_concat and formats with a complex concatenation.
Need it simplified, efficient and working in all scenarios.
Some details of my solution : Following is the important part, other is just conditions/Looping (I am not handy with documentation also it needs time and suggestions, someone might help me in its formatting and improvement, Sorry for any inconvenience, however I will be glad for any help from you and me)
Note: group_concat(yourColumn separator ' --anySeparator-- ') is merging all rows of your column as one such that Rows are separated by --anySeparator--
select group_concat(column_name separator '`,`') into #cns1 from
information_schema.columns where table_schema=dn and table_name=#tn;
1 : column_names are got as a single value separated by
`,` => #cs1 = id`,`ename`,`did
select group_concat(column_name separator '`,"\',\'",`') into #cns2
from information_schema.columns where table_schema=dn and table_name=#tn;
2 : column_names are got as a single value separated by
`','` => #cn2 = id`','`ename`','`did
set #cns1=concat("`",#cns1,"`"); set #cns2=concat("`",#cns2,"`");
3: Missing letter (`) is put at beginning and end of Column names
set #res=concat(#res," insert into ",#tn,"(",#cns1,") values ('");
4: Simply makes res= " insert into emp(`id` , `ename` ,`did` ) values(" Here you can see why have I put separators (MySql Formatting is achieved)
set #temp := '';
set #q := concat("select group_concat(concat(",#cns2,") separator \"'),('\")
from ",dn,".",#tn, " into #temp");
Above is the most crucial statement It gets all data rows from table as rows of a single column and further these rows are merged being separated by '),('
5.1 concat(",#cns2,") gets values of all columns in a single one.
5.2 After outer most concat now #q is
#q = "select group_concat(`id`','`ename`','`,did` separator '),(' from
mydb.emp into #temp";
5.3 : group_concat will merge all rows of that combined column into one value.
Columns values will be joined through separators existing in #cns2 and rows level joining will be with '),('
prepare s1 from #q;
execute s1;deallocate prepare s1;
set #res = concat(#res,#temp,");");
#q is executed
set #res = concat(#res,#temp,");");
6 : And We will get result as
res was = insert into emp(`id`,`ename`,`did`) values ('
#temp = 1','e1','4'),('2','e2','4'),
('3','e3','2'),('4','e4','4'),('5','e5','3
And after #res = concat(#res,#temp,");"); we get
insert into emp(`id`,`ename`,`did`) values ('1','e1','4'),('2','e2','4'),
('3','e3','2'),('4','e4','4'),('5','e5','3);
select concat("insert into users (id,name,password) values ('",id,"'"),
concat(",'",username,"'"),
concat(",'",password,"'),") INTO OUTFILE 'c:\\datas\\asd.txt' from users
Related
I'm trying to use a select statement to get all of the columns from a certain MySQL table except one. Is there a simple way to do this?
EDIT: There are 53 columns in this table (NOT MY DESIGN)
Actually there is a way, you need to have permissions of course for doing this ...
SET #sql = CONCAT('SELECT ', (SELECT REPLACE(GROUP_CONCAT(COLUMN_NAME), '<columns_to_omit>,', '') FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '<table>' AND TABLE_SCHEMA = '<database>'), ' FROM <table>');
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
Replacing <table>, <database> and <columns_to_omit>
(Do not try this on a big table, the result might be... surprising !)
TEMPORARY TABLE
DROP TABLE IF EXISTS temp_tb;
CREATE TEMPORARY TABLE ENGINE=MEMORY temp_tb SELECT * FROM orig_tb;
ALTER TABLE temp_tb DROP col_a, DROP col_f,DROP col_z; #// MySQL
SELECT * FROM temp_tb;
DROP syntax may vary for databases #Denis Rozhnev
Would a View work better in this case?
CREATE VIEW vwTable
as
SELECT
col1
, col2
, col3
, col..
, col53
FROM table
You can do:
SELECT column1, column2, column4 FROM table WHERE whatever
without getting column3, though perhaps you were looking for a more general solution?
If you are looking to exclude the value of a field, e.g. for security concerns / sensitive info, you can retrieve that column as null.
e.g.
SELECT *, NULL AS salary FROM users
To the best of my knowledge, there isn't. You can do something like:
SELECT col1, col2, col3, col4 FROM tbl
and manually choose the columns you want. However, if you want a lot of columns, then you might just want to do a:
SELECT * FROM tbl
and just ignore what you don't want.
In your particular case, I would suggest:
SELECT * FROM tbl
unless you only want a few columns. If you only want four columns, then:
SELECT col3, col6, col45, col 52 FROM tbl
would be fine, but if you want 50 columns, then any code that makes the query would become (too?) difficult to read.
While trying the solutions by #Mahomedalid and #Junaid I found a problem. So thought of sharing it. If the column name is having spaces or hyphens like check-in then the query will fail. The simple workaround is to use backtick around column names. The modified query is below
SET #SQL = CONCAT('SELECT ', (SELECT GROUP_CONCAT(CONCAT("`", COLUMN_NAME, "`")) FROM
INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'users' AND COLUMN_NAME NOT IN ('id')), ' FROM users');
PREPARE stmt1 FROM #SQL;
EXECUTE stmt1;
If the column that you didn't want to select had a massive amount of data in it, and you didn't want to include it due to speed issues and you select the other columns often, I would suggest that you create a new table with the one field that you don't usually select with a key to the original table and remove the field from the original table. Join the tables when that extra field is actually required.
You could use DESCRIBE my_table and use the results of that to generate the SELECT statement dynamically.
My main problem is the many columns I get when joining tables. While this is not the answer to your question (how to select all but certain columns from one table), I think it is worth mentioning that you can specify table. to get all columns from a particular table, instead of just specifying .
Here is an example of how this could be very useful:
select users.*, phone.meta_value as phone, zipcode.meta_value as zipcode
from users
left join user_meta as phone
on ( (users.user_id = phone.user_id) AND (phone.meta_key = 'phone') )
left join user_meta as zipcode
on ( (users.user_id = zipcode.user_id) AND (zipcode.meta_key = 'zipcode') )
The result is all the columns from the users table, and two additional columns which were joined from the meta table.
I liked the answer from #Mahomedalid besides this fact informed in comment from #Bill Karwin. The possible problem raised by #Jan Koritak is true I faced that but I have found a trick for that and just want to share it here for anyone facing the issue.
we can replace the REPLACE function with where clause in the sub-query of Prepared statement like this:
Using my table and column name
SET #SQL = CONCAT('SELECT ', (SELECT GROUP_CONCAT(COLUMN_NAME) FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'users' AND COLUMN_NAME NOT IN ('id')), ' FROM users');
PREPARE stmt1 FROM #SQL;
EXECUTE stmt1;
So, this is going to exclude only the field id but not company_id
Yes, though it can be high I/O depending on the table here is a workaround I found for it.
SELECT *
INTO #temp
FROM table
ALTER TABLE #temp DROP COlUMN column_name
SELECT *
FROM #temp
It is good practice to specify the columns that you are querying even if you query all the columns.
So I would suggest you write the name of each column in the statement (excluding the one you don't want).
SELECT
col1
, col2
, col3
, col..
, col53
FROM table
I agree with the "simple" solution of listing all the columns, but this can be burdensome, and typos can cause lots of wasted time. I use a function "getTableColumns" to retrieve the names of my columns suitable for pasting into a query. Then all I need to do is to delete those I don't want.
CREATE FUNCTION `getTableColumns`(tablename varchar(100))
RETURNS varchar(5000) CHARSET latin1
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE res VARCHAR(5000) DEFAULT "";
DECLARE col VARCHAR(200);
DECLARE cur1 CURSOR FOR
select COLUMN_NAME from information_schema.columns
where TABLE_NAME=#table AND TABLE_SCHEMA="yourdatabase" ORDER BY ORDINAL_POSITION;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur1;
REPEAT
FETCH cur1 INTO col;
IF NOT done THEN
set res = CONCAT(res,IF(LENGTH(res)>0,",",""),col);
END IF;
UNTIL done END REPEAT;
CLOSE cur1;
RETURN res;
Your result returns a comma delimited string, for example...
col1,col2,col3,col4,...col53
I agree that it isn't sufficient to Select *, if that one you don't need, as mentioned elsewhere, is a BLOB, you don't want to have that overhead creep in.
I would create a view with the required data, then you can Select * in comfort --if the database software supports them. Else, put the huge data in another table.
At first I thought you could use regular expressions, but as I've been reading the MYSQL docs it seems you can't. If I were you I would use another language (such as PHP) to generate a list of columns you want to get, store it as a string and then use that to generate the SQL.
Based on #Mahomedalid answer, I have done some improvements to support "select all columns except some in mysql"
SET #database = 'database_name';
SET #tablename = 'table_name';
SET #cols2delete = 'col1,col2,col3';
SET #sql = CONCAT(
'SELECT ',
(
SELECT GROUP_CONCAT( IF(FIND_IN_SET(COLUMN_NAME, #cols2delete), NULL, COLUMN_NAME ) )
FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = #tablename AND TABLE_SCHEMA = #database
),
' FROM ',
#tablename);
SELECT #sql;
If you do have a lots of cols, use this sql to change group_concat_max_len
SET ##group_concat_max_len = 2048;
I agree with #Mahomedalid's answer, but I didn't want to do something like a prepared statement and I didn't want to type all the fields, so what I had was a silly solution.
Go to the table in phpmyadmin->sql->select, it dumps the query: copy, replace and done! :)
While I agree with Thomas' answer (+1 ;)), I'd like to add the caveat that I'll assume the column that you don't want contains hardly any data. If it contains enormous amounts of text, xml or binary blobs, then take the time to select each column individually. Your performance will suffer otherwise. Cheers!
Just do
SELECT * FROM table WHERE whatever
Then drop the column in you favourite programming language: php
while (($data = mysql_fetch_array($result, MYSQL_ASSOC)) !== FALSE) {
unset($data["id"]);
foreach ($data as $k => $v) {
echo"$v,";
}
}
The answer posted by Mahomedalid has a small problem:
Inside replace function code was replacing "<columns_to_delete>," by "", this replacement has a problem if the field to replace is the last one in the concat string due to the last one doesn't have the char comma "," and is not removed from the string.
My proposal:
SET #sql = CONCAT('SELECT ', (SELECT REPLACE(GROUP_CONCAT(COLUMN_NAME),
'<columns_to_delete>', '\'FIELD_REMOVED\'')
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = '<table>'
AND TABLE_SCHEMA = '<database>'), ' FROM <table>');
Replacing <table>, <database> and `
The column removed is replaced by the string "FIELD_REMOVED" in my case this works because I was trying to safe memory. (The field I was removing is a BLOB of around 1MB)
You can use SQL to generate SQL if you like and evaluate the SQL it produces. This is a general solution as it extracts the column names from the information schema. Here is an example from the Unix command line.
Substituting
MYSQL with your mysql command
TABLE with the table name
EXCLUDEDFIELD with excluded field name
echo $(echo 'select concat("select ", group_concat(column_name) , " from TABLE") from information_schema.columns where table_name="TABLE" and column_name != "EXCLUDEDFIELD" group by "t"' | MYSQL | tail -n 1) | MYSQL
You will really only need to extract the column names in this way only once to construct the column list excluded that column, and then just use the query you have constructed.
So something like:
column_list=$(echo 'select group_concat(column_name) from information_schema.columns where table_name="TABLE" and column_name != "EXCLUDEDFIELD" group by "t"' | MYSQL | tail -n 1)
Now you can reuse the $column_list string in queries you construct.
I wanted this too so I created a function instead.
public function getColsExcept($table,$remove){
$res =mysql_query("SHOW COLUMNS FROM $table");
while($arr = mysql_fetch_assoc($res)){
$cols[] = $arr['Field'];
}
if(is_array($remove)){
$newCols = array_diff($cols,$remove);
return "`".implode("`,`",$newCols)."`";
}else{
$length = count($cols);
for($i=0;$i<$length;$i++){
if($cols[$i] == $remove)
unset($cols[$i]);
}
return "`".implode("`,`",$cols)."`";
}
}
So how it works is that you enter the table, then a column you don't want or as in an array: array("id","name","whatevercolumn")
So in select you could use it like this:
mysql_query("SELECT ".$db->getColsExcept('table',array('id','bigtextcolumn'))." FROM table");
or
mysql_query("SELECT ".$db->getColsExcept('table','bigtextcolumn')." FROM table");
May be I have a solution to Jan Koritak's pointed out discrepancy
SELECT CONCAT('SELECT ',
( SELECT GROUP_CONCAT(t.col)
FROM
(
SELECT CASE
WHEN COLUMN_NAME = 'eid' THEN NULL
ELSE COLUMN_NAME
END AS col
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'employee' AND TABLE_SCHEMA = 'test'
) t
WHERE t.col IS NOT NULL) ,
' FROM employee' );
Table :
SELECT table_name,column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'employee' AND TABLE_SCHEMA = 'test'
================================
table_name column_name
employee eid
employee name_eid
employee sal
================================
Query Result:
'SELECT name_eid,sal FROM employee'
I use this work around although it may be "Off topic" - using mysql workbench and the query builder -
Open the columns view
Shift select all the columns you want in your query (in your case all but one which is what i do)
Right click and select send to SQL Editor-> name short.
Now you have the list and you can then copy paste the query to where ever.
If it's always the same one column, then you can create a view that doesn't have it in it.
Otherwise, no I don't think so.
I would like to add another point of view in order to solve this problem, specially if you have a small number of columns to remove.
You could use a DB tool like MySQL Workbench in order to generate the select statement for you, so you just have to manually remove those columns for the generated statement and copy it to your SQL script.
In MySQL Workbench the way to generate it is:
Right click on the table -> send to Sql Editor -> Select All Statement.
The accepted answer has several shortcomings.
It fails where the table or column names requires backticks
It fails if the column you want to omit is last in the list
It requires listing the table name twice (once for the select and another for the query text) which is redundant and unnecessary
It can potentially return column names in the wrong order
All of these issues can be overcome by simply including backticks in the SEPARATOR for your GROUP_CONCAT and using a WHERE condition instead of REPLACE(). For my purposes (and I imagine many others') I wanted the column names returned in the same order that they appear in the table itself. To achieve this, here we use an explicit ORDER BY clause inside of the GROUP_CONCAT() function:
SELECT CONCAT(
'SELECT `',
GROUP_CONCAT(COLUMN_NAME ORDER BY `ORDINAL_POSITION` SEPARATOR '`,`'),
'` FROM `',
`TABLE_SCHEMA`,
'`.`',
TABLE_NAME,
'`;'
)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE `TABLE_SCHEMA` = 'my_database'
AND `TABLE_NAME` = 'my_table'
AND `COLUMN_NAME` != 'column_to_omit';
I have a suggestion but not a solution.
If some of your columns have a larger data sets then you should try with following
SELECT *, LEFT(col1, 0) AS col1, LEFT(col2, 0) as col2 FROM table
If you use MySQL Workbench you can right-click your table and click Send to sql editor and then Select All Statement This will create an statement where all fields are listed, like this:
SELECT `purchase_history`.`id`,
`purchase_history`.`user_id`,
`purchase_history`.`deleted_at`
FROM `fs_normal_run_2`.`purchase_history`;
SELECT * FROM fs_normal_run_2.purchase_history;
Now you can just remove those that you dont want.
Let us say I have two tables with many columns so I do not want to name the column names explicitly in my query but i want to avoid duplicate names.
If I do:
CREATE TABLE new_table
SELECT a.*, b.*
FROM table1 a
INNER JOIN table2 b ON a.myID = b.myId
WHERE a.age > 10 and b.ice = 'melted'
I will get an error saying: duplicate column name myId, I could also get more errors if more column names in a and b are the same.
How can I avoid this issue by automatically adding a prefix to all column names in a.* and b.* w/o explicitly mentioning all the column names - very tedious to do so!
Thanks!
Unfortunately, you will have to list the columns in case table have matching column names. However, you can use information_schema to get the column names, format those and copy paste in the query to save the pain, e.g.:
SELECT GROUP_CONCAT(CONCAT('a.', COLUMN_NAME)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'schema' AND TABLE_NAME = 'table';
The above query should give you comma separated column names with a. prefix. You can then use the same query for table b, get the names out and use it in the main SELECT query.
Update
As #Uueerdo has rightly said, you can add alias to columns as well, e.g.:
SELECT GROUP_CONCAT(CONCAT('a.', COLUMN_NAME, ' AS a_', COLUMN_NAME))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'schema' AND TABLE_NAME = 'table';
In my experience ORMs will run an initial DESCRIBE query so it can do this sort of stuff for you once it has the column names. But if you insist on doing it dynamically in a single query, you could do this with pure MySQL:
-- config
SET #database = 'your_database';
SET #tableA = 'table1';
SET #tableB = 'table2';
-- table alias "a" columns
SET #fieldsA = NULL;
SELECT GROUP_CONCAT(CONCAT('a.', COLUMN_NAME), ' AS ',CONCAT('`a.', COLUMN_NAME,'`')) INTO #fieldsA
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = #database AND TABLE_NAME = #tableA;
-- table alias "b" columns
SET #fieldsB = NULL;
SELECT GROUP_CONCAT(CONCAT('b.', COLUMN_NAME), ' AS ',CONCAT('`b.', COLUMN_NAME,'`')) INTO #fieldsB
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = #database AND TABLE_NAME = #tableB;
-- some variables for readability
SET #fields = CONCAT(' ', #fieldsA, ',', #fieldsB,' ');
SET #tableAliasA = CONCAT(' ',#database, '.', #tableA,' a ');
SET #tableAliasB = CONCAT(' ',#database, '.', #tableB,' b ');
-- generate our final query
SET #query = CONCAT('CREATE TABLE new_table SELECT', #fields,
'FROM', #tableAliasA,
'INNER JOIN', #tableAliasB,
'ON a.myID = b.myId WHERE a.age > 10 and b.ice = ''melted''');
-- finally run the query:
PREPARE stmt1 FROM #query;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;
-- if you have problems with the above query, uncomment the following to get the query so you can run it separately
-- SELECT #query;
I'd strongly advise against using this sort of solution though. I'd sooner run an initial DESCRIBE query as earlier stated, then generate your query based on that. Another solution is to create a temporary table as a copy of the second table, then rename problematic columns, then proceed to join on it to produce the data you need to create your new_table. MySQL has no issues with result columns having the same name, the issue here is trying to create a table with two columns with the same name. So essentially what you're trying to do is a star select but excluding a column.
Another approach is to just select only the primary key from both:
SELECT a.myID as `aId`, b.myId as `bId` then create your table containing only that. Then if you ever need data from a particular table, just LEFT JOIN on it to grab the information you're looking for. You can take this a step further and set up a VIEW to do this sort of thing for you. VIEWs can join tables for you and make it very easy to select whatever columns you're looking for. You can also setup multiple VIEWs as well. Also note that views behave just like tables for the purpose of joins. You can JOIN a view with a table, or you can join a view with a view, etc.
So rather than do what you're trying to do -- creating a new table with the data from two other tables -- consider whether you're actually looking for a VIEW.
I'm pretty new to MySql and this might seem like a stupid question, but i can't seem to find the answer anywhere.
I have two tables that I'm working with.
The first table contains a lot of columns filled with Integer values.
such as: No_of_Landlines No_of_mobiles
The second table is a Rule Table which contains a rule name followed by the actual rule.
Eg:
ID RuleName Rule
1 Example No_of_Landlines > No_of_Mobiles
The query that i want to run is this:
SELECT *
FROM DemoTable
WHERE (SELECT Rule from RuleTable WHERE ID=1)
What i want it to do is this:
SELECT *
FROM DemoTable
WHERE No_of_Landlines > No_of_Mobiles
When i run the first query against my database, it returns empty because the value of the rule being fetched from the Rule Table is a string and not a column name that I want, unlike query two under it.
How do I make the values of the string being returned into column names and execute the entire query?
One of my friends who's good with MSSQL suggested this:
DECLARE #a varchar(4000)
SET #a=(SELECT Rule FROM RuleDB WHERE ID=1)
print #a
EXEC('SELECT * FROM DemoTable WHERE ' + #a)
I tried this in MSSQL and it does work the way I want it to.
Is there a way to achieve this in MySql?
You can use EXECUTE command, this is a link to the syntax! .
Here is an example script:
SELECT #Q:= CONCAT('SELECT * FROM DemoTable WHERE ', Rules.Rule) FROM Rules WHERE ID=3;
PREPARE stq FROM #Q;
EXECUTE stq;
DEALLOCATE PREPARE stq;
In this script you create que query you want to execute in the variable #Q concatenating part of the query with the result of the rule you are fetching from the Rules table.
Then you prepare the statment, execute and deallocate.
You can define variables in MySQL and use them the same way you used them in MSSQL.
The complete documentation can be found here:
http://dev.mysql.com/doc/refman/5.5/en/user-variables.html
There are some similar questions around but they aren't quite what I'm looking for, so forgive me if you think this is answered elsewhere.
I am basically looking for an easy way to do things as I have over 4000 tables to get data from. This kind of follows on from my previous post: mysql search for segment of table name
The general situation is that I have a database filled with tables and I only want about a quarter of this which comes to around 4000 tables. I have a list of the individual table names thanks to my previous post, but I want the data that goes with them.
I know that for an individual one I can do SELECT table1.*, table2.*; or something similar but I don't want to go through all 4000 or so.
They all end with the same thing, e.g. staff_name, manager_name, customer_name so I can use
SHOW TABLES LIKE '%_name'
to see the table names that I want in the database. Someone suggested using dynamic mysql, but I don't even know where to start with that. Any suggestions?
Generic example (in PHP):
Constructing dynamic SQL or building your SQL queries with the aid of a programming language would look like this (in PHP for ex.):
$pdos = $pdo->query("SHOW TABLES LIKE '%_name'");
$tables = $pdos->fetchAll();
$query = 'SELECT * FROM '.implode(' UNION SELECT * FROM ');
$pdo->query($query);
The fetchAll method will return an array containing the names of each table selected.
The implode($glue, $array) function takes an array and concatenates every value in the array using the $glue parameter - usually you take an array of values and implode them using $glue = ',' to create a coma separated list of values.
In our case the implode has a partial query as $glue in order to create one big UNION JOIN query.
Once the final $query is build it should look something like:
SELECT * FROM table_1_name
UNION
SELECT * FROM table_2_name
UNION
SELECT * FROM table_3_name
....
....
UNION
SELECT * FROM table_4000_name
The result should contain all of the DISTINCT rows from all 4000 tables.
Specific example (in SQL-only format):
SELECT GROUP_CONCAT(
CONCAT('select * from ', table_name)
SEPARATOR ' union '
)
INTO #my_variable
FROM information_schema.tables
WHERE table_schema = 'dbname'
AND table_name LIKE '%_name';
PREPARE my_statement FROM #my_variable;
EXECUTE my_statement;
The first statement will get all of the table names from the information_schema database;
The CONCAT function prefixes every table name with a a 'SELECT * FROM ' string;
The GROUP_CONCAT does the job that implode would have done in PHP;
The INTO clause makes sure the values are saved inside a variable named my_variable;
The PREPARE statement takes a string value (such as the one you saved in my_variable) and checks if the value is an SQL query;
The EXECUTE statement takes a "prepared statement" and well... executes it.
#my_variable is a temporary variable but it can only be of a scalar type (varchar, int, date, datetime, binary, float, double etc.) it is not an array.
The GROUP_CONCAT function is an "aggregate function" which means it takes an aggregate value (similar concept to an array - in our case the result set of our query) and outputs a simple string result.
I would suggest generating the SQL statement.
Try doing:
select concat('select * from ', table_name) as query
from Information_Schema.tables
where table_schema = <dbname> and
table_name like <whatever>
You can then run this as a bunch of queries by copying into a query editor window.
If you want everything as one query, then do:
select concat('select * from ', table_name, ' union all ') as query
from Information_Schema.tables
where table_schema = <dbname> and
table_name like <whatever>
And remove the final "union all".
This has the table name matching a like. Leave out the table_name part of the WHERE to get all tables. Or, include specific tables using table_name in ().
I'm trying to use a select statement to get all of the columns from a certain MySQL table except one. Is there a simple way to do this?
EDIT: There are 53 columns in this table (NOT MY DESIGN)
Actually there is a way, you need to have permissions of course for doing this ...
SET #sql = CONCAT('SELECT ', (SELECT REPLACE(GROUP_CONCAT(COLUMN_NAME), '<columns_to_omit>,', '') FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '<table>' AND TABLE_SCHEMA = '<database>'), ' FROM <table>');
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
Replacing <table>, <database> and <columns_to_omit>
(Do not try this on a big table, the result might be... surprising !)
TEMPORARY TABLE
DROP TABLE IF EXISTS temp_tb;
CREATE TEMPORARY TABLE ENGINE=MEMORY temp_tb SELECT * FROM orig_tb;
ALTER TABLE temp_tb DROP col_a, DROP col_f,DROP col_z; #// MySQL
SELECT * FROM temp_tb;
DROP syntax may vary for databases #Denis Rozhnev
Would a View work better in this case?
CREATE VIEW vwTable
as
SELECT
col1
, col2
, col3
, col..
, col53
FROM table
You can do:
SELECT column1, column2, column4 FROM table WHERE whatever
without getting column3, though perhaps you were looking for a more general solution?
If you are looking to exclude the value of a field, e.g. for security concerns / sensitive info, you can retrieve that column as null.
e.g.
SELECT *, NULL AS salary FROM users
To the best of my knowledge, there isn't. You can do something like:
SELECT col1, col2, col3, col4 FROM tbl
and manually choose the columns you want. However, if you want a lot of columns, then you might just want to do a:
SELECT * FROM tbl
and just ignore what you don't want.
In your particular case, I would suggest:
SELECT * FROM tbl
unless you only want a few columns. If you only want four columns, then:
SELECT col3, col6, col45, col 52 FROM tbl
would be fine, but if you want 50 columns, then any code that makes the query would become (too?) difficult to read.
While trying the solutions by #Mahomedalid and #Junaid I found a problem. So thought of sharing it. If the column name is having spaces or hyphens like check-in then the query will fail. The simple workaround is to use backtick around column names. The modified query is below
SET #SQL = CONCAT('SELECT ', (SELECT GROUP_CONCAT(CONCAT("`", COLUMN_NAME, "`")) FROM
INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'users' AND COLUMN_NAME NOT IN ('id')), ' FROM users');
PREPARE stmt1 FROM #SQL;
EXECUTE stmt1;
If the column that you didn't want to select had a massive amount of data in it, and you didn't want to include it due to speed issues and you select the other columns often, I would suggest that you create a new table with the one field that you don't usually select with a key to the original table and remove the field from the original table. Join the tables when that extra field is actually required.
You could use DESCRIBE my_table and use the results of that to generate the SELECT statement dynamically.
My main problem is the many columns I get when joining tables. While this is not the answer to your question (how to select all but certain columns from one table), I think it is worth mentioning that you can specify table. to get all columns from a particular table, instead of just specifying .
Here is an example of how this could be very useful:
select users.*, phone.meta_value as phone, zipcode.meta_value as zipcode
from users
left join user_meta as phone
on ( (users.user_id = phone.user_id) AND (phone.meta_key = 'phone') )
left join user_meta as zipcode
on ( (users.user_id = zipcode.user_id) AND (zipcode.meta_key = 'zipcode') )
The result is all the columns from the users table, and two additional columns which were joined from the meta table.
I liked the answer from #Mahomedalid besides this fact informed in comment from #Bill Karwin. The possible problem raised by #Jan Koritak is true I faced that but I have found a trick for that and just want to share it here for anyone facing the issue.
we can replace the REPLACE function with where clause in the sub-query of Prepared statement like this:
Using my table and column name
SET #SQL = CONCAT('SELECT ', (SELECT GROUP_CONCAT(COLUMN_NAME) FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'users' AND COLUMN_NAME NOT IN ('id')), ' FROM users');
PREPARE stmt1 FROM #SQL;
EXECUTE stmt1;
So, this is going to exclude only the field id but not company_id
Yes, though it can be high I/O depending on the table here is a workaround I found for it.
SELECT *
INTO #temp
FROM table
ALTER TABLE #temp DROP COlUMN column_name
SELECT *
FROM #temp
It is good practice to specify the columns that you are querying even if you query all the columns.
So I would suggest you write the name of each column in the statement (excluding the one you don't want).
SELECT
col1
, col2
, col3
, col..
, col53
FROM table
I agree with the "simple" solution of listing all the columns, but this can be burdensome, and typos can cause lots of wasted time. I use a function "getTableColumns" to retrieve the names of my columns suitable for pasting into a query. Then all I need to do is to delete those I don't want.
CREATE FUNCTION `getTableColumns`(tablename varchar(100))
RETURNS varchar(5000) CHARSET latin1
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE res VARCHAR(5000) DEFAULT "";
DECLARE col VARCHAR(200);
DECLARE cur1 CURSOR FOR
select COLUMN_NAME from information_schema.columns
where TABLE_NAME=#table AND TABLE_SCHEMA="yourdatabase" ORDER BY ORDINAL_POSITION;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
OPEN cur1;
REPEAT
FETCH cur1 INTO col;
IF NOT done THEN
set res = CONCAT(res,IF(LENGTH(res)>0,",",""),col);
END IF;
UNTIL done END REPEAT;
CLOSE cur1;
RETURN res;
Your result returns a comma delimited string, for example...
col1,col2,col3,col4,...col53
I agree that it isn't sufficient to Select *, if that one you don't need, as mentioned elsewhere, is a BLOB, you don't want to have that overhead creep in.
I would create a view with the required data, then you can Select * in comfort --if the database software supports them. Else, put the huge data in another table.
At first I thought you could use regular expressions, but as I've been reading the MYSQL docs it seems you can't. If I were you I would use another language (such as PHP) to generate a list of columns you want to get, store it as a string and then use that to generate the SQL.
Based on #Mahomedalid answer, I have done some improvements to support "select all columns except some in mysql"
SET #database = 'database_name';
SET #tablename = 'table_name';
SET #cols2delete = 'col1,col2,col3';
SET #sql = CONCAT(
'SELECT ',
(
SELECT GROUP_CONCAT( IF(FIND_IN_SET(COLUMN_NAME, #cols2delete), NULL, COLUMN_NAME ) )
FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = #tablename AND TABLE_SCHEMA = #database
),
' FROM ',
#tablename);
SELECT #sql;
If you do have a lots of cols, use this sql to change group_concat_max_len
SET ##group_concat_max_len = 2048;
I agree with #Mahomedalid's answer, but I didn't want to do something like a prepared statement and I didn't want to type all the fields, so what I had was a silly solution.
Go to the table in phpmyadmin->sql->select, it dumps the query: copy, replace and done! :)
While I agree with Thomas' answer (+1 ;)), I'd like to add the caveat that I'll assume the column that you don't want contains hardly any data. If it contains enormous amounts of text, xml or binary blobs, then take the time to select each column individually. Your performance will suffer otherwise. Cheers!
Just do
SELECT * FROM table WHERE whatever
Then drop the column in you favourite programming language: php
while (($data = mysql_fetch_array($result, MYSQL_ASSOC)) !== FALSE) {
unset($data["id"]);
foreach ($data as $k => $v) {
echo"$v,";
}
}
The answer posted by Mahomedalid has a small problem:
Inside replace function code was replacing "<columns_to_delete>," by "", this replacement has a problem if the field to replace is the last one in the concat string due to the last one doesn't have the char comma "," and is not removed from the string.
My proposal:
SET #sql = CONCAT('SELECT ', (SELECT REPLACE(GROUP_CONCAT(COLUMN_NAME),
'<columns_to_delete>', '\'FIELD_REMOVED\'')
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = '<table>'
AND TABLE_SCHEMA = '<database>'), ' FROM <table>');
Replacing <table>, <database> and `
The column removed is replaced by the string "FIELD_REMOVED" in my case this works because I was trying to safe memory. (The field I was removing is a BLOB of around 1MB)
You can use SQL to generate SQL if you like and evaluate the SQL it produces. This is a general solution as it extracts the column names from the information schema. Here is an example from the Unix command line.
Substituting
MYSQL with your mysql command
TABLE with the table name
EXCLUDEDFIELD with excluded field name
echo $(echo 'select concat("select ", group_concat(column_name) , " from TABLE") from information_schema.columns where table_name="TABLE" and column_name != "EXCLUDEDFIELD" group by "t"' | MYSQL | tail -n 1) | MYSQL
You will really only need to extract the column names in this way only once to construct the column list excluded that column, and then just use the query you have constructed.
So something like:
column_list=$(echo 'select group_concat(column_name) from information_schema.columns where table_name="TABLE" and column_name != "EXCLUDEDFIELD" group by "t"' | MYSQL | tail -n 1)
Now you can reuse the $column_list string in queries you construct.
I wanted this too so I created a function instead.
public function getColsExcept($table,$remove){
$res =mysql_query("SHOW COLUMNS FROM $table");
while($arr = mysql_fetch_assoc($res)){
$cols[] = $arr['Field'];
}
if(is_array($remove)){
$newCols = array_diff($cols,$remove);
return "`".implode("`,`",$newCols)."`";
}else{
$length = count($cols);
for($i=0;$i<$length;$i++){
if($cols[$i] == $remove)
unset($cols[$i]);
}
return "`".implode("`,`",$cols)."`";
}
}
So how it works is that you enter the table, then a column you don't want or as in an array: array("id","name","whatevercolumn")
So in select you could use it like this:
mysql_query("SELECT ".$db->getColsExcept('table',array('id','bigtextcolumn'))." FROM table");
or
mysql_query("SELECT ".$db->getColsExcept('table','bigtextcolumn')." FROM table");
May be I have a solution to Jan Koritak's pointed out discrepancy
SELECT CONCAT('SELECT ',
( SELECT GROUP_CONCAT(t.col)
FROM
(
SELECT CASE
WHEN COLUMN_NAME = 'eid' THEN NULL
ELSE COLUMN_NAME
END AS col
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'employee' AND TABLE_SCHEMA = 'test'
) t
WHERE t.col IS NOT NULL) ,
' FROM employee' );
Table :
SELECT table_name,column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'employee' AND TABLE_SCHEMA = 'test'
================================
table_name column_name
employee eid
employee name_eid
employee sal
================================
Query Result:
'SELECT name_eid,sal FROM employee'
I use this work around although it may be "Off topic" - using mysql workbench and the query builder -
Open the columns view
Shift select all the columns you want in your query (in your case all but one which is what i do)
Right click and select send to SQL Editor-> name short.
Now you have the list and you can then copy paste the query to where ever.
If it's always the same one column, then you can create a view that doesn't have it in it.
Otherwise, no I don't think so.
I would like to add another point of view in order to solve this problem, specially if you have a small number of columns to remove.
You could use a DB tool like MySQL Workbench in order to generate the select statement for you, so you just have to manually remove those columns for the generated statement and copy it to your SQL script.
In MySQL Workbench the way to generate it is:
Right click on the table -> send to Sql Editor -> Select All Statement.
The accepted answer has several shortcomings.
It fails where the table or column names requires backticks
It fails if the column you want to omit is last in the list
It requires listing the table name twice (once for the select and another for the query text) which is redundant and unnecessary
It can potentially return column names in the wrong order
All of these issues can be overcome by simply including backticks in the SEPARATOR for your GROUP_CONCAT and using a WHERE condition instead of REPLACE(). For my purposes (and I imagine many others') I wanted the column names returned in the same order that they appear in the table itself. To achieve this, here we use an explicit ORDER BY clause inside of the GROUP_CONCAT() function:
SELECT CONCAT(
'SELECT `',
GROUP_CONCAT(COLUMN_NAME ORDER BY `ORDINAL_POSITION` SEPARATOR '`,`'),
'` FROM `',
`TABLE_SCHEMA`,
'`.`',
TABLE_NAME,
'`;'
)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE `TABLE_SCHEMA` = 'my_database'
AND `TABLE_NAME` = 'my_table'
AND `COLUMN_NAME` != 'column_to_omit';
I have a suggestion but not a solution.
If some of your columns have a larger data sets then you should try with following
SELECT *, LEFT(col1, 0) AS col1, LEFT(col2, 0) as col2 FROM table
If you use MySQL Workbench you can right-click your table and click Send to sql editor and then Select All Statement This will create an statement where all fields are listed, like this:
SELECT `purchase_history`.`id`,
`purchase_history`.`user_id`,
`purchase_history`.`deleted_at`
FROM `fs_normal_run_2`.`purchase_history`;
SELECT * FROM fs_normal_run_2.purchase_history;
Now you can just remove those that you dont want.