When I run the SQL script to export data out of MySQL database, using command line, I get the above error.
The SQL query works fine when run in phpMyAdmin, but just when run from command line throws an error.
Here is the command line I am using:
cat my_export | mysql -uxyzuser -pabcpassword mydb > export072911.txt
The code in my_export is as follows:
SELECT CONCAT( custfirstname, ' ', custlastname ) AS fullname, custcompany, \
SPACE( 10 ) AS custtitle, custaddressone, custaddresstwo, custcity, custstate, \
custzip, SPACE( 10 ) AS dummy, custphone, SPACE( 10 ) AS custfax, custemail, \
event_id, SPACE( 10 ) AS ticket1, SPACE( 10 ) AS ticket2, \
SPACE( 10 ) AS ticket3, SPACE( 10 ) AS ticket4, orderdate, b.quantity, \
FROM order_master a \
LEFT JOIN order_detail b ON b.order_master_id = a.id \
LEFT JOIN customer c ON c.email = a.custemail \
WHERE a.orderdate > '2010-12-01'\
AND a.event_id = '30' \
AND a.orderstatus = 'O' \
AND b.litype = 'ITEM' \
AND b.reftag = 'PKG' \
ORDER BY a.orderdate DESC;
You can safely delete all the backslashes and use input redirection rather than piping. The backslashes are needed if you are working with the SQL as a shell variable, but not for piping or redirection.
mysql -uxyzuser -pabcpassword mydb < my_export > export072911.txt
UPDATE After a quick test of my own, it looks like the pipe works just as well as input redirection as long as the backslashes are removed.
Related
Im trying to have a sql output file from a query between three tables.
Below is my code:
mysqldump --user=user --password=password --host=localhost dbname --where="SELECT v.ITEM_ID , v.CODE ,\
s.SENSOR , d.DESTINATIONCODE \
FROM V_TABLE v, S_TABLE s ,D_TABLE d \
WHERE s.ITEM_ID = v.ITEM_ID \
AND s.CREATIONDATETIME < '2014-2-16 00:00:00'\
AND v.DESTINATION_ID=d.ID" > /var/www/dumps/output_15_12_2017.sql
this give me an error:
mysqldump: Couldn't execute 'SELECT /*!40001 SQL_NO_CACHE */ * FROM `COMMANDSPOOL` WHERE SELECT v.ITEM_ID , v.CODE ,s.SENSOR , d.DESTINATIONCODE FROM V_TABLE v, S_TABLE s ,D_TABLE d WHERE s.ITEM_ID = v.ITEM_ID AND s.CREATIONDATETIME < '2014-2-16 00:00:00' AND v.DESTINATION_ID=d.ID': You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SELECT v.ITEM_ID , v.CODE ,s.SENSOR , d.DESTINATIONCODE FROM V_TABLE' at line 1 (1064 )
maybe it is an error in using \
To use a query like that you could send it into the mysql client, (not the mysqldump utility) and then redirect that output to the file instead:
echo "SELECT v.ITEM_ID , v.CODE ,\
s.SENSOR , d.DESTINATIONCODE \
FROM V_TABLE v, S_TABLE s ,D_TABLE d \
WHERE s.ITEM_ID = v.ITEM_ID \
AND s.CREATIONDATETIME < '2014-2-16 00:00:00'\
AND v.DESTINATION_ID=d.ID\
" | mysql --user=user --password=password --host=localhost dbname > /var/www/dumps/output_15_12_2017.sql
If you're going this route then the --batch and --table options can be useful.
I am using cloudera quick start edition CDH 5.7
I used below query on terminal window:
sqoop import \
--connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" \
--username=retail_dba \
--password=cloudera \
--query="select * from orders join order_items on orders.order_id = order_items.order_item_order_id where \$CONDITIONS" \
--target-dir /user/cloudera/order_join \
--split-by order_id \
--num-mappers 4
Q: What is the purpose of the $CONDITIONS ? Why used in this query ? Can anybody can explain to me.
$CONDITIONS is used internally by sqoop to modify query to achieve task splitting and fetching metadata.
To fetch metadata, sqoop replaces \$CONDITIONS with 1= 0
select * from table where 1 = 0
To fetch all data (1 mapper), sqoop replaces \$CONDITIONS with 1= 1
select * from table where 1 = 1
In the case of multiple mappers, sqoop replaces \$CONDITIONS with range query to fetch a subset of data from RDBMS.
For example, id lies between 1 to 100 and we are using 4 mappers.
Select * From table WHERE id >= 1' AND 'id < 25
Select * From table WHERE id >= 25' AND 'id < 50
Select * From table WHERE id >= 50' AND 'id < 75
Select * From table WHERE id >= 75' AND 'id <= 100
I have a linux script that gets a variable and I store it to var JOB_EXEC_ID
I am trying to pass the value of this to a MySQL query
Here is MySQL query set-up
print "JOB EXEC ID value for DataMeer Job ${LOADJOB} is : ${JobExecId} " |
tee -a ${LOGDIR}/${LOGFILE}
#Log on to MySQL to get the DataId
#Remove first the output file that would house the dataid
rm -f ${SCRDIR}/list_dataid.csv
mysql -u root -pmonday1 ${DAPDBNAME} < ${SCRDIR}/dataid_query.nosql
SQLRTN=$?
if [[ ${SQLRTN} != 0 ]]
then
print "Return code from sqlcall - DAP : ${SQLRTN}" |
tee -a ${LOGDIR}/${LOGFILE}
print "Exiting $Script - 55 " |
tee -a ${LOGDIR}/${LOGFILE}
exit 55
fi
The file dataid_query.nosql looks like this:
set #job_exec_id=10151
select d.id DataId
from data d inner join dap_job_configuration djc on d.dap_job_configuration__id = djc.id
left outer join dap_job_execution dje on djc.id = dje.dap_job_configuration__id and dje.created_data__id = d.id
where dje.id=#job_exec_id
into OUTFILE "/home/app1ebb/cs/list_dataid.csv"
I want to pass the value of JOB_EXEC_ID to the set command that is currently hardcoded right now with a value of 10151
in place of
mysql -u root -pmonday1 ${DAPDBNAME} < ${SCRDIR}/dataid_query.nosql
SQLRTN=$?
this lines
sed "1 s/[0-9]*$/${JOB_EXEC_ID}/" > /tmp/dataid_query.nosql
mysql -u root -pmonday1 ${DAPDBNAME} < /tmp/dataid_query.nosql
SQLRTN=$?
rm /tmp/dataid_query.nosql
i have a script to synchronize/output my mysql database with another server.
i know how to call this SQL query in shell. however, is it possible to convert this script's output from php to shell script to keep the format shown below (basically values separated by "|" and some URLs)? i need the output in this specific format shown below:
$sql=mysql_query("SELECT a, b, c, d, e FROM tableA LEFT JOIN tableB USING (a)");
while ($res=mysql_fetch_array($sql))
{
echo $res['a']."|".$res['b']."|".$res['c']."|http://www.url.com/".$res['e']."/".$res['a'].".php|http://www.url.com/link/".urlencode($res['b'])."\n";
}
thanks
You can do all these concatenation with a simple Query passed to mysql from command line :
mysql -u username -ppassword -D dbname << eof
SET sql_mode='STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,PIPES_AS_CONCAT';
SELECT a || '|' || b || '|' || c || '|http://www.url.com/' || e || '/' || a || '.php|http://www.url.com/link/' || b AS joinedColum FROM tableA LEFT JOIN tableB USING (a);
eof
The first SET, set the pipe as concat operator in MySQL so you can join string with SQL standard double pipe instead of using the annoying CONCAT function ... After that the result set is a concatenation of strings directly in the SQL query.
In this way you can execute this command from a bash script without the necessity to have a php interpreter installed ...
I met some problem that wired me the whole day. I'm using flask/flask-sqlalchemy/postgresql, and I want to do this:
published_sites = msg_published.query \
.filter( and_(*filter_clause) ) \
.group_by( msg_published.site_id ) \
.order_by( order_clause ) \
.paginate( page_no, per_page, error_out = False )
but in mysql, it is OK, and in postgresql it is wrong and ask for the other fields besides site_id either in a group by clause or in a aggregation function, I know that postgresql is stricter on SQL than mysql , so I must select the site_id in the query object of msg_published, but in pure sqlalchemy I can do like this:
published_sites = session.query( msg_published.site_id ) \
.filter( and_(*filter_clause) ) \
.group_by( msg_published.site_id ) \
.order_by( order_clause ) \
.paginate( page_no, per_page, error_out = False )
and in flask-sqlalchemy, how to get it work?
You're most of the way there. to do in PostgreSQL what MySQL allows requires a subselect.
published_sites_ids = session.query( msg_published.site_id ) \
.filter( and_(*filter_clause) ) \
.group_by( msg_published.site_id ) \
.order_by( order_clause ) \
.paginate( page_no, per_page, error_out = False )
published_sites = session.query(msg_published) \
.filter(msg_published.id.in_(published_sites_ids))