i have a script to synchronize/output my mysql database with another server.
i know how to call this SQL query in shell. however, is it possible to convert this script's output from php to shell script to keep the format shown below (basically values separated by "|" and some URLs)? i need the output in this specific format shown below:
$sql=mysql_query("SELECT a, b, c, d, e FROM tableA LEFT JOIN tableB USING (a)");
while ($res=mysql_fetch_array($sql))
{
echo $res['a']."|".$res['b']."|".$res['c']."|http://www.url.com/".$res['e']."/".$res['a'].".php|http://www.url.com/link/".urlencode($res['b'])."\n";
}
thanks
You can do all these concatenation with a simple Query passed to mysql from command line :
mysql -u username -ppassword -D dbname << eof
SET sql_mode='STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,PIPES_AS_CONCAT';
SELECT a || '|' || b || '|' || c || '|http://www.url.com/' || e || '/' || a || '.php|http://www.url.com/link/' || b AS joinedColum FROM tableA LEFT JOIN tableB USING (a);
eof
The first SET, set the pipe as concat operator in MySQL so you can join string with SQL standard double pipe instead of using the annoying CONCAT function ... After that the result set is a concatenation of strings directly in the SQL query.
In this way you can execute this command from a bash script without the necessity to have a php interpreter installed ...
Related
Insert into D.d
Select * from A.a join B.b on
A.a.a1=B.b.b1
Join C.c on C.c.c1=B.b.b1
I have complex statements for which i need to extract source db name ( in above statement source DB are A,B,C and source tables are a,b,c &Target Db is D and target table is d)
Need output like
SourceDB SourceTbl TargetDB Targettbl
A,B,C a,b,c D d
Or we can get values in json format as well for each field.. Also this needs to accomodate for update and delete statements as well. Please assist
Thanks
You can use the SQLPARSE to parse the statement. I am providing a code below which is not optimally and efficiently written, but it has the logic to get the information
import sqlparse
raw = 'Insert into D.d ' \
'Select * from A.a join B.b on ' \
'A.a.a1=B.b.b1 Join C.c on C.c.c1=B.b.b1;'
parsed = sqlparse.parse(raw)[0]
tgt_switch = "N"
src_switch = "N"
src_table=[]
tgt_table= ""
for items in parsed.tokens:
#print(items,items.ttype)
if str(items) == "into":
tgt_switch ="Y"
if tgt_switch == "Y" and items.ttype is None:
tgt_switch = "N"
tgt_table = items
if str(items).lower() == "from" or str(items).lower() == "join":
src_switch = "Y"
if src_switch == "Y" and items.ttype is None:
src_switch = "N"
src_table.append(str(items))
target_db = str(tgt_table).split(".")[0]
target_tbl = str(tgt_table).split(".")[1]
print("Target DB is {} and Target table is {}".format(target_db,target_tbl))
for obj in src_table:
src_db = str(obj).split(".")[0]
src_tbl = str(obj).split(".")[1]
print("Source DB is {} and Source table is {}".format(src_db, src_tbl))
Snowflake does not offer any SQL statement parsing support. You can hack at it with regex'es, of course, or use any of the tools on the market.
If this query ran, and ran successfully, you can use ACCESS_HISTORY view https://docs.snowflake.com/en/sql-reference/account-usage/access_history.html to see which tables (A.a, B.b, C.c, D.d) and columns (A.a.a1, B.b.b1, C.c.c1, D.d.d1) it accessed and how (read or write).
This question already has answers here:
Using a command that needs backticks to be passed as part of an argument in bash
(2 answers)
Closed 3 years ago.
I have a query
Select id, contact_name from Profile where TRIM(IFNULL(contact_name,'')) <> ''
and I am trying to pass it into a Bash script.
mysql --user abc -psomePass MyData -e "SELECT `id`,`contact_name` from `Profile` where TRIM(IFNULL(`contact_name`,'') <> ''" | while read term_id; do
done
but it doesn't like the singe quotes ''. How do I format this in the script?
I have tried
TRIM(IFNULL(`contact_name`,\'\') <> \'\'"
and
TRIM(IFNULL(`contact_name`,\") <> \""
and
TRIM(IFNULL(`contact_name`,"''") <> "''""
I am out of ideas.
but it doesn't like the single quotes.
Bash on my machine is fine with single quotes:
$ echo "SELECT id,contact_name from Profile where TRIM(IFNULL(contact_name,'') <> ''"
SELECT id,contact_name from Profile where TRIM(IFNULL(contact_name,'') <> ''
$
I am trying to store each value from the following sql select statement and store them in separate variables using bash.
#!/bin/bash
mysqlhost="thehost"
mysqldb="thedb"
mysqlun="theusername"
mysqlpw="thepassword"
mysqlconnection="--disable-column-names --host=$mysqlhost --user $mysqlun --password=$mysqlpw --database=$mysqldb"
declare -a pinIDs=$(mysql $mysqlconnection -e "SELECT pinID FROM somewhere WHERE something = something";)
I get the following result when I use code
echo $pinIDs
8 11 23 26
I need to store each of those values into their own variable.
Add brackets to put output in array pinIDs. Replace
declare -a pinIDs=$(mysql $mysqlconnection -e "SELECT pinID FROM somewhere WHERE something = something";)
by
declare -a pinIDs=( $(mysql $mysqlconnection -e "SELECT pinID FROM somewhere WHERE something = something";) )
Then see output of: declare -p pinIDs
For others that are using this to learn. What I did after fixing the brackets in order to put the result into an array such as (8 11 23 26) instead of 8 11 23 26, was this:
cnt=${#pinIDs[#]}
for (( i=0 ; i<cnt ; i++ ))
do
echo "pinId: ""${pinIDs[$i]}"
done
I have below function to read the data from mysql database:
get_all_server_ip_address(){
mysql -h$MYSQL_HOST -u$DB_USER_NAME -p$DB_PASSWORD << EOF
use $DB_SCHEMA;
select ip_address from server;
EOF
}
I am calling this function as below:
server_ip=get_all_server_ip_address
for ip in $server_ip
do
echo $ip
done
Output:
ip_address
54.206.76.6
I have only "54.206.76.6" in my database and it returns ip_address i.e column name
Database screenshot
use "--skip-column-names" option to login into mysql and then query , for example :
mysql -uroot probe --skip-column-names -e 'select ip_address from TABLE_NAME limit 1'
I have a linux script that gets a variable and I store it to var JOB_EXEC_ID
I am trying to pass the value of this to a MySQL query
Here is MySQL query set-up
print "JOB EXEC ID value for DataMeer Job ${LOADJOB} is : ${JobExecId} " |
tee -a ${LOGDIR}/${LOGFILE}
#Log on to MySQL to get the DataId
#Remove first the output file that would house the dataid
rm -f ${SCRDIR}/list_dataid.csv
mysql -u root -pmonday1 ${DAPDBNAME} < ${SCRDIR}/dataid_query.nosql
SQLRTN=$?
if [[ ${SQLRTN} != 0 ]]
then
print "Return code from sqlcall - DAP : ${SQLRTN}" |
tee -a ${LOGDIR}/${LOGFILE}
print "Exiting $Script - 55 " |
tee -a ${LOGDIR}/${LOGFILE}
exit 55
fi
The file dataid_query.nosql looks like this:
set #job_exec_id=10151
select d.id DataId
from data d inner join dap_job_configuration djc on d.dap_job_configuration__id = djc.id
left outer join dap_job_execution dje on djc.id = dje.dap_job_configuration__id and dje.created_data__id = d.id
where dje.id=#job_exec_id
into OUTFILE "/home/app1ebb/cs/list_dataid.csv"
I want to pass the value of JOB_EXEC_ID to the set command that is currently hardcoded right now with a value of 10151
in place of
mysql -u root -pmonday1 ${DAPDBNAME} < ${SCRDIR}/dataid_query.nosql
SQLRTN=$?
this lines
sed "1 s/[0-9]*$/${JOB_EXEC_ID}/" > /tmp/dataid_query.nosql
mysql -u root -pmonday1 ${DAPDBNAME} < /tmp/dataid_query.nosql
SQLRTN=$?
rm /tmp/dataid_query.nosql