Say I have a view in my database, and I want to send a file to someone to create that view's output as a table in their database.
mysqldump of course only exports the 'create view...' statement (well, okay, it includes the create table, but no data).
What I have done is simply duplicate the view as a real table and dump that. But for a big table it's slow and wasteful:
create table tmptable select * from myview
Short of creating a script that mimics the behaviour of mysqldump and does this, is there a better way?
One option would be to do a query into a CSV file and import that. To select into a CSV file:
From http://www.tech-recipes.com/rx/1475/save-mysql-query-results-into-a-text-or-csv-file/
SELECT order_id,product_name,qty
FROM orders
INTO OUTFILE '/tmp/orders.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
OK, so based on your CSV failure comment, start with Paul's answer. Make the following change to it:
- FIELDS TERMINATED BY ','
+ FIELDS TERMINATED BY ',' ESCAPED BY '\'
When you're done with that, on the import side you'll do a "load data infile" and use the same terminated / enclosed / escaped statements.
Same problem here my problem is that I want to export view definition (84 fields and millions of records) as a "create table" statement, because view can variate along time and I want an automatic process. So that's what I did:
Create table from view but with no records
mysql -uxxxx -pxxxxxx my_db -e "create table if not exists my_view_def as select * from my_view limit 0;"
Export new table definition. I'm adding a sed command to change table name my_view_def to match original view name ("my_view")
mysqldump -uxxxx -pxxxxxx my_db my_view_def | sed s/my_view_def/my_view/g > /tmp/my_view.sql
drop temporary table
mysql -uxxxx -pxxxxxx my_db -e "drop table my_view_def;"
Export data as a CSV file
SELECT * from my_view into outfile "/tmp/my_view.csv" fields terminated BY ";" ENCLOSED BY '"' LINES TERMINATED BY '\n';
Then you'll have two files, one with the definition and another with the data in CSV format.
Related
I need a shell script to load data into mysql db. The script is the next:
# !bin/bash
qry="DROP TABLE IF EXISTS tmp_x;
CREATE TEMPORARY TABLE tmp_x AS SELECT * FROM x.y LIMIT 0;
LOAD DATA INFILE 'path/xxx.csv'
INTO TABLE tmp_x
FIELDS TERMINATED BY "\,"
ENCLOSED BY "\""
LINES TERMINATED BY "\\n"
IGNORE 1 ROWS;"
mysql --host=xxx --user=xxx --password=xxx db << EOF
$qry
EOF
I get the following error message:
ERROR 1064 (42000) at line 3: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the
right syntax to use near '
ENCLOSED BY "
LINES TERMINATED BY \n
IGNORE 1 ROWS' at line 3
I think it is something to do escaping some character, I tried changing to single quotes but it does not work neither.
I am workin on Ubuntu 18.
Any help will be very grateful.
Try this:
#!/bin/bash
mysql --host=xxx --user=xxx --password=xxx db << EOF
DROP TABLE IF EXISTS tmp_x;
CREATE TEMPORARY TABLE tmp_x AS SELECT * FROM x.y LIMIT 0;
LOAD DATA INFILE 'path/xxx.csv'
INTO TABLE tmp_x
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\\n'
IGNORE 1 ROWS;
EOF
If you really must use a variable, you'll need to play with quoting:
#!/bin/bash
qry="DROP TABLE IF EXISTS tmp_x;
CREATE TEMPORARY TABLE tmp_x AS SELECT * FROM x.y LIMIT 0;
LOAD DATA INFILE 'path/xxx.csv'
INTO TABLE tmp_x
FIELDS TERMINATED BY \",\"
ENCLOSED BY \"\\\"\"
LINES TERMINATED BY \"\\n\"
IGNORE 1 ROWS;"
mysql --host=xxx --user=xxx --password=xxx db << EOF
$qry
EOF
It can be troublesome to use double-quoted strings in your SQL, since you're using double-quotes as the string delimiter in bash. In other words, which is the double-quote that ends the bash string, and which should be treated as a literal double-quote character in the SQL?
To resolve this, use single-quotes for string delimiters in the SQL.
Another issue: There's no need to put a backslash before , for the field terminator.
Another issue: The \n needs another backslash.
Here's what I tried and it seems to work:
qry="DROP TABLE IF EXISTS tmp_x;
CREATE TEMPORARY TABLE tmp_x AS SELECT * FROM x.y LIMIT 0;
LOAD DATA INFILE 'path/xxx.csv'
INTO TABLE tmp_x
FIELDS TERMINATED BY ','
ENCLOSED BY '\"'
LINES TERMINATED BY '\\\n'
IGNORE 1 ROWS;"
I only printed the query, I haven't tested running it.
How to use mysqldump to dump specified column only from specified table in database?
I need something like this
mysqldump --skip-lock-tables -q -Q -c -e -h localhost -u username -pPassword DatabaseName TableName Field1 Field5 | gzip > /tmp/dump.sql.gz
But I get errors only
Using mysqldump it's not possible right now, but you may use into outfile utility to get the desired output. In your case the query will look like:
SELECT col1, col2 FROM DatabaseName.TableName INTO OUTFILE "c:/output.txt" FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "\n";
Later on you can use this file to upload in another table called TableName2 with just two columns (ie. col1 and col2) by using following sql:
LOAD DATA LOCAL INFILE 'c:/output.txt' INTO TABLE TableName2
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n';
i am using bash to load a file in mysql, and i have:
mysql --local-infile=1 -u user data_base_1 < file.sql
and file.sql is :
..$ cat file.sql
load data local infile '/folder/load.csv' into table table_1 fields terminated by '|'
The code works fine.
The problem is that if the PK of one row in the file exist, the row is not inserted, and i need if the row exist insert and replace the row in the table. How can i do it?
Who can help me?
Thanks
You can specify REPLACE with LOAD DATA:
LOAD DATA LOCAL INFILE '/folder/load.csv' REPLACE INTO TABLE table_1 FIELDS TERMINATED BY '|'
Or else use the mysqlimport --replace option.
http://dev.mysql.com/doc/refman/5.6/en/mysqlimport.html#option_mysqlimport_replace
You could load into a temporary table and then execute two SQL statements:
UPDATE table
WHERE ... (match found)
;
INSERT into table(...)
SELECT ...
FROM temp_table
WHERE NOT EXISTS(...)
Consider this code:
mysql> select * into outfile 'atmout12.csv' fields terminated by ',' optionally enclosed by '"' lines terminated by '\n' from atm_atm;
ERROR 1086 (HY000): File 'atmout12.csv' already exists
mysql> select * into outfile 'atmout1.csv' fields terminated by ',' optionally enclosed by '"' lines terminated by '\n' from atm_atm;
Query OK, 2822 rows affected (0.02 sec)
I used the above snippet to convert a table data to a CSV file. As you can see the query ran fine, but I am unable to locate where the file is.
I do an ls in the folder and can't locate it. I am using Ubuntu 11.04
The file will be locate in your data directory.
example: datadir=/opt/data/db_name.
Inside the particular database (db_name) folder/dir will contain your .csv file.
OR else we can give the output file in particular location , to generate like that the user should have super privileges.
example :
mysql > use db_name
mysql> select * into outfile 'atmout1.csv' from atm_atm;
or
mysql> select * into outfile '/opt/example.csv' from atm_atm;
NOTE :above output file will be in db_name folder.
I have a column in a MySQL table that consists of comma-delimited strings. I would like to convert this column into a set of distinct strings that occur in the column (for any row in the table -- the set includes strings that occur in any row of the table in this column). What is the easiest way to accomplish this?
The solution doesn't need to be pure MySQL. It could involve unix, perl, etc.
You could probably get a quick-and-dirty list of distinct strings from a comma-delimited column using SELECT INTO OUTFILE, sed, and LOAD DATA INFILE.
Basically you want to dump the text data to a flat file, using a comma as the line delimiter so each string will be treated as a separate row when you load it back into the database. Then load the extracted into a new table and select the distinct values.
Off the top of my head, the code would look something like this:
select str
into outfile
'/tmp/your_table_data.txt'
lines terminated by ','
from your_table;
sed -e 's/\\,/,/g' -e 's/,$//' /tmp/your_table_data.txt > /tmp/commadelimited.txt
create table your_table_parsed(str text);
load data infile '/tmp/commadelimited.txt'
ignore into table your_table_parsed
fields terminated by ','
lines terminated by ',';
select distinct str from your_table_parsed;
The way I chose was to run the select mysql command outside of the mysql shell and pipe the results into tr and sort --uniq
mysql my_db [-p -u -h] -se "select my_column from my_table;" | tr ',' '\n' | sort -u
This is pretty simple and seems to give the correct results as far as I can tell.