I'm running a python script which returns several lines of text, which I use the following truncate command tr and want to add to my database.
I start here: this removes the unformatted line delimiters.
tr -d '\15\32' < long_text > unixfile.txt
I'm then left with an output which looks like this:
Happy Birthday Stackoverflow
Happy Birthday to you
Happy Birthday Stackoverflow
I use the following command to place this into a variable:
lyrics=$(cat unixfile.txt)
mysql --user=USER --password=PASSWORD --database='DB' --execute='INSERT INTO `song_lyrics` (`id`, `song_id`, `lyrics`, `info`) VALUES ('"'$i'"', '"'$i'"', '"$lyrics"', '0');'
ERROR 1064 (42000) at line 1:
It seems that MySQL is seeing my entire 3 line string (shown above) in three different commands, because the output of the unixfile.txt appears in the error output.
This problem can be solved using the following:
Ensure the long text string is being captured in a file, instead of a variable - then re-capture the file/variable.
lyrics=$(cat unixfile.txt)
mysql --user="$user" --password="$password" --database="$my_db" <<END
use my_db;
INSERT INTO song_lyrics (id, song_id, lyrics, info) VALUES ("$i", "$i", "$lyrics", '0');
END
this method ensures the variable is passed properly.
the only remaining issues would be with various types of punctuation
although this post does not describe this.
I'm trying to update multiple rows in a DB using a small script.
I need to update the rows based on some specific user_ids which I have in a list on Linux machine.
#! /bin/bash
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in ()";
As you see above, the user_ids are in a file, let's say /opt/test/user_ids_txt.
How can I import them into this command?
This really depends on the format of user_ids_txt. If we assume it just happens to be in the correct syntax for your SQL in statement, the following will work:
#! /bin/bash
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in ($(< /opt/test/user_ids_txt))";
The bash interpreter will substitute in the contents of the file. This can be dangerous for SQL queries, so I would echo out the command on the terminal to make sure it is correct before implementing it. You should be able to preview your SQL query by simply running the following on the command line:
echo "update device set in_use=0 where user_id in ($(< /opt/test/user_ids_txt))"
If your file is not in the SQL in syntax you will need to edit it (or a copy of it) before running your query. I would recommend something like sed for this.
Example
Let's say your file /opt/test/user_ids_txt is just a list of user_ids in the format:
aaa
bbb
ccc
You can use sed to edit this into the correct SQL syntax:
sed 's/^/\'/g; s/$/\'/g; 2,$s/^/,/g' /opt/test/user_ids_txt
The output of this command will be:
'aaa'
,'bbb'
,'ccc'
If you look at this sed command, you will see 3 separate commands separated by semicolons. The individual commands translate to:
1: Add ' to the beginning of every line
2: Add ' to the end of every line
3: Add , to the beginning of every line but the first
Note: If your ID's are strictly numeric, you only need the third command.
This would make your SQL query translate to:
update device set in_use=0 where user_id in ('aaa'
,'bbb'
,'ccc')
Rather than make a temporary file to store this, I would use a bash variable, and simply plug that into the query like this:
#! /bin/bash
in_statement="$(sed 's/^/\'/g; s/$/\'/g; 2,$s/^/,/g' /opt/test/user_ids_txt)"
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in (${in_statement})";
I am looking for a way to have a simple way to execute SQL commands. One attempt is to make a custom #! script but I am not sure how to do this either.
If I do something like:
#!/bin/cat -n
select
col1
from
table;
I get output like
1 #!/bin/cat -n
2 select
3 col1
4 from
5 table;
Which makes me think I could be close.
But when I create a script like runsql.sh
#!/bin/bash
cat -n
./some_sql.sh: line 2: syntax error near unexpected token `newline'
./some_sql.sh: line 2: `select'
This is my attempt at being able to execute sql files. Is there someway people are doing this that I am not doing?
Thank you
try it:
mysql -e "YOUR_SQL_COMMAND"
for large command try it (i dont test but need work):
sqlCommand=$(cat <<EOF
This is large
Sql command
This is line three.
EOF
)
mysql < $sqlCommand
When I try to convert the timestamp in the following query, using bash
docker exec compose_TSOwncloudMySQL_1 mysql -h localhost -udockerdev -pdocker owc -e "
SELECT DATE_FORMAT(FROM_UNIXTIME(`timestamp`), '%Y%m%d timestamp%h:%i:%s') AS 'date_formatted',
oc_ldap_user_mapping.ldap_dn,
oc_activity.subject,
oc_activity.file,
oc_activity.subjectparams
FROM oc_activity INNER JOIN oc_ldap_user_mapping ON oc_activity.user = oc_ldap_user_mapping.owncloud_name
ORDER BY oc_activity.timestamp;"> /home/dockerdmz/tsowncloud/log_owc/owc_$DATE.log`
I have this error:
ERROR 1582 (42000) at line 2: Incorrect parameter count in the call to native function 'FROM_UNIXTIME'
When I run this query in MySQL admin page, it works well.
SELECT DATE_FORMAT(FROM_UNIXTIME(`timestamp`), '%Y%m%d %h:%i:%s') AS 'date_formatted',
oc_ldap_user_mapping.ldap_dn,
oc_activity.subject,
oc_activity.file,
oc_activity.subjectparams
FROM oc_activity INNER JOIN oc_ldap_user_mapping ON oc_activity.user = oc_ldap_user_mapping.owncloud_name
ORDER BY oc_activity.timestamp;
When I run this bash code (without timestamp conversion), it works well
Docker exec compose_TSOwncloudMySQL_1 mysql -h localhost -udockerdev -pdocker owc -e "
SELECT oc_activity.timestamp,
oc_ldap_user_mapping.ldap_dn,
oc_activity.subject,
oc_activity.file,
oc_activity.subjectparams
FROM oc_activity INNER JOIN oc_ldap_user_mapping ON oc_activity.user = oc_ldap_user_mapping.owncloud_name
ORDER BY oc_activity.timestamp;"> /home/dockerdmz/tsowncloud/log_owc/owc_$DATE.log
Ugh. It's a bash backtick thing.
Try escaping the backticks with backslashes when you use bash, like so.
-e " SELECT DATE_FORMAT(FROM_UNIXTIME(\`timestamp\`), ...
Pro tip: Avoid using reserved words (such as timestamp in your case) for column or table names. That way you don't have to wrap them in backticks in your queries, and you can use the same queries in various contexts (bash, php, etc).
I have a script that essentially creates Devices, graphs, and trees as well as attempts to create a user for Cacti graphing software.
I receive the below SQL error when attempting to run the shell script. However copying and pasting the exact same input statement into MySQL directly it accepts the syntax.
ERROR 1064 (42000) at line 3: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''TestName','cf199661b212c55286cf81d9602ce63','0', 'TestFullName','on','on','' at line 1
The below script is how this is implemented
php add_device.php --description=$1 --ip=$2 --community=community --template=5 --ping_method=icmp
mysql -u root cacti -ppassword -e "select id from host where description='$1'" | grep -v id > tempID
php add_tree.php --type=node --node-type=host --tree-id=3 --parent-node=34 --host-group-style=2 --host-id=`cat tempID`
php add_graphs.php --host-id=`cat tempID` --graph-type=ds --graph-template-id=2 --snmp-query-id=1 --snmp-query-type-id=14 --snmp-field=ifOperStatus --snmp-value=Up
mysql -u root cacti -ppassword -e "select MAX(id) FROM user_auth" | grep -v id > tempUserID
Number=`cat tempUserID`
User_IDD= $(($Number + 1))
Host_ID=`cat tempID`
mysql -uroot -ppassword cacti << EOF
INSERT INTO user_auth (id,username,password,realm,full_name,must_change_password,show_tree,show_list,show_preview,graph_settings,login_opts,policy_graphs,policy_trees,policy_hosts,policy_graph_templates,enabled) V
ALUES($User_IDD,'$3','cf199661b212c55286cf81d9602ce630',0,'$4','on','on','on','on','on',1,2,2,2,2,'on');
INSERT INTO user_auth_perms (user_id,item_id,type) VALUES ($User_IDD,$Host_ID,3);
INSERT INTO user_auth_realm (realm_id,user_id) VALUES (7,$User_IDD);
EOF
rm TempUserID
rm TempID
~
The exact SQL insert statements pasted directly into the command line are as follows as an example that functions
INSERT INTO user_auth
(id, username, password, realm, full_name, must_change_password, show_tree, show_list, show_preview, graph_settings, login_opts, policy_graphs, policy_trees, policy_hosts, policy_graph_templates, enabled)
VALUES
( 38,
'Test',
'cf199661b212c55286cf81d9602ce63',
'0',
'TestUser',
'on',
'on',
'on',
'on',
'on',
1,
2,
2,
2,
2,
'on'
);
INSERT INTO user_auth_perms
(user_id, item_id, type)
VALUES (38, 285, 3);
INSERT INTO user_auth_realm
(realm_id, user_id)
VALUES (7, 38);
I've been bashing my head against this for a few days now and I can't quite find what's wrong in my syntax, does anyone see anything outright wrong?
It sounds like what's happening is that you're "running" the script in your head, finding that it works fine, and then getting stuck when the computer doesn't agree.
When debugging, you shouldn't run the script for the computer: don't copy-paste the SQL and then replace the variables with what you think they ought to be.
Instead, let the computer run the script: have it output the SQL and then observe if that matches what you expect.
You can do this by replacing mysql -uroot -ppassword cacti with cat, which will make the script spit out the result instead of executing it. You'll see it's something like this:
INSERT INTO user_auth (id,username,password,realm,full_name, [snip])
VALUES(,'baz','cf199661b212c55286cf81d9602ce630',0,'cow','on','on',[snip]);
And now the problem is more obvious: VALUES(,'baz' is missing its first value.
To solve that problem, you can pay careful attention to unexpected error messages. You'll see that one you're getting is something like:
42: command not found
You should not ignore these kinds of messages. When you copy-paste output to stackoverflow, you should include all output (if it's too long, you should reduce the size of your script and input data, rather than redacting information you don't think is relevant).
You'll find that the error occurs on this line:
User_IDD= $(($Number + 1))
And why is it saying "command not found" instead of assigning to the variable? Because of the space after the =. Assignments in bash can not have spaces around the assignment operator.
Remove the space and try again.
I'm not saying that's sufficient to solve all the problems with your script, but it's a good start.
PS: ShellCheck would have automatically pointed out the bad space.