I have a shell script
mysql --local-infile=1 -h myhost -P myport -u me < ./loader_file.sql
When I run this directly in command line, it works great. data gets loaded into the database.
but inside a scala code
val myScript = "mysql --local-infile=1 -h myhost -P myport -u me" #< "loader_file.sql"
myScript !
I get a error message saying the mysql commands in my shell, such as "use" and "load" are not found. Obviously, shell is interpreting the loader_file.sql as bash command rather then mysql commands.
But if I incorporate the whole string as one command
val myScript = "mysql --local-infile=1 -h myhost -P myport -u me < loader_file.sql"
Then I get a mysql dump with all its valid options. Apparently, mysql thinks I am feeding it illegal input parameters.
Anyone knows how to fix this?
Here's what you need to do it from scala code:
import scala.sys.process._
val cmd = Seq("bash","-c","mysql --local-infile=1 -h myhost -P myport -u me < loader_file.sql")
for( line <- cmd.lineStream_! ){
printf("%s\n",line)
}
That should work under linux, OSX, cygwin, or similar (sounds like you have one of those environments, since bash is present).
There are lots of variations on this particular theme, allowing you to set the default working directory, specify environment variables, separately process STDOUT and STDERR streams, and capture the return value, etc.
Here's a version that does everything except specify environment variables:
import scala.sys.process._
val cmd = Seq("bash","-c","command with args, blah, blah, blah")
val proc = Process( cmd, new java.io.File(".") )
val exitValue = proc ! ProcessLogger (
(out) => System.out.printf("stdout:%s\n",out)
,
(err) => System.err.printf("stderr:%s\n",err)
)
printf("exit value: %d\n",exitValue)
The documentation for scala.sys.process is pretty good, although it assumes a linux-type environment. A shell command line will sometimes work without being wrapped with Seq("bash","-c", ...), but it isn't as portable (Windows java doesn't pass your command line to bash, for example).
Related
I'm developing an InSpec control that runs CIS compliance commands.
While working on MySQL, I'm stuck here:
Execute the following SQL statement to determine the value of datadir:
show variables where variable_name = 'datadir';
I need to extract the output from the above command and reuse it in the next command:
ls -l <THE OUTPUT OF THE PREVIOUS COMMAND>/.. | egrep "^d[r|w|x]{3}------\s*.\s*mysql\s*mysql\s*\d*.*mysql"
The problem is that the first command is an SQL Request and the second command is a terminal command.
How can I put both of them (after getting the output of the first command and put it in the second one) in an InSpec control like the following:
control "mysql1" do
impact 1.0
title "Use dedicated Least Privileged Account for MySQL Daemon/Service"
desc "May reduce the impact of a MySQL-born vulnerability"
describe command ('ps -ef |e grep "^mysql.*$"') do
its('stdout') { should match ''}
end
end
Thank you for your help #Matt
I've read your answer and found it really helpful, except the last block of code : Does
egrep "^d[r|w|x]{3}------\s*.\s*mysql\s*mysql\s*\d*.*mysql"
mean
it { expect(subject).to_not be_owned_by 'mysql' }
it { expect(subject).to_not be_grouped_into 'mysql' }
it { expect(subject).to_not be_executable_by 'mysql' }
?
Plus I did try all of the blocks you wrote previously and none of them did work.. And yes, I'm using linux 16.04
You can extract the output of the SQL request with the following method:
command('mysql -u <user> -p -e "show variables where variable_name = \'datadir\'"').stdout.split(' ')
The mysql -u <user> -p -e part is necessary to execute the SQL query from a Linux command. If you are using Window, you will probably need to make use of sqlcmd instead. This allows the SQL query to execute successfully with the command method.
The reason the command method works here is because it is a custom RSpec type (implicitly therefore also a class constructor in the sense that Ruby has constructors) that will execute locally or remotely on the tested system. The .stdout method is a member of the class to capture the stdout of the command. .split will ensure the output variables are stored in a whitespace-delimited array.
Now we can use it in the next command like so:
# store array of variables
variables = command('mysql -u <user> -p -e "show variables where variable_name = \'datadir\'"').stdout.split(' ')
# use array in command
variables.each do |variable|
describe command("ls -l #{variable}/.. | egrep \"^d[r|w|x]{3}------\s*.\s*mysql\s*mysql\s*\d*.*mysql\"") do
its('stdout') { should match ''}
end
end
Above we iterate through the array of variables captured in the SQL query and test it in the describe command() RSpec test. A better way to execute this test would be to test the stdout of the command in the matcher and not the egrep. Doing that and cleaning up the match method:
# store array of variables
variables = command('mysql -u <user> -p -e "show variables where variable_name = \'datadir\'"').stdout.split(' ')
# use array in command
variables.each do |variable|
describe command("ls -l #{variable}/..") do
its('stdout') { should_not match(/^d[r|w|x]{3}------\s*.\s*mysql\s*mysql\s*\d*.*mysql/)}
end
end
Updating to non-deprecated RSpec matchers and fixing the invoking of the stdout method as a string instead of a symbol we arrive at:
# store array of variables
variables = command('mysql -u <user> -p -e "show variables where variable_name = \'datadir\'"').stdout.split(' ')
# use array in command
variables.each do |variable|
describe command("ls -l #{variable}/..") do
its(:stdout) { is_expected.to_not match(/^d[r|w|x]{3}------\s*.\s*mysql\s*mysql\s*\d*.*mysql/)}
end
end
Another improvement we can make is to use a better suited file type and the permissions matchers instead of raw commands. This helps for platform-independent testing:
# store array of variables
variables = command('mysql -u <user> -p -e "show variables where variable_name = \'datadir\'"').stdout.split(' ')
# use array in file type
variables.each do |variable|
describe file("#{variable}/..") do
# check permissions
it { expect(subject).to_not be_owned_by 'mysql' }
it { expect(subject).to_not be_grouped_into 'mysql' }
it { expect(subject).to_not be_executable_by 'mysql' }
end
end
I understand there was a good bit here to implement the functionality you are looking for and many fixes and improvements as well, so be sure to examine the code and explanations closely to understand everything I did here.
I've the following bash script to upgrade my database schema. Script reads hostname and database password from command line.
The problem is here that if the password is alphanumeric e.g r00t then script works. But if password contains special characters e.g pa**w0rd, then script does not work and directly exits. Please help me with this. Thanks.
#!/bin/bash
echo "Enter hostname."
read -p "Hostname [localhost]: " DB_HOST
DB_HOST=${DB_HOST:-localhost}
echo "Enter MySQL root password"
DB_PASS=
while [[ $DB_PASS = "" ]]; do
read -sp "Password: " DB_PASS
done
MYSQL="mysql --force --connect-timeout=90 --host=$DB_HOST -u root --password=${DB_PASS}"
# Apply schema updates. My DBName is "mydb"
# Upgrade schema file is stored in "mysql" folder
$MYSQL mydb -e exit > /dev/null 2>&1 && $MYSQL mydb < "../mysql/upgrade_schema_v.2.1.sql"
Logging into mysql using bash
For ubuntu or linux shell try to use command
mysql -u username -p'p#ssw()rD'
for remote host login use
mysql -h hostname -u user -p'password'
This is occurring because you are using shell GLOB (wildcard) characters in the password, and in Bash (or on Linux generally) wildcards are expanded by the shell.
The safest and most reliable solution is to not use shell wildcard characters or other characters interpreted by the shell in the password. You should also avoid spaces. There are plenty of other characters.
Here are the ones you should avoid:
" ' $ , [ ] * ? { } ~ # % \ < > | ^ ;
Here are the ones it is usually safe to use:
: # . , / + - ! =
To ensure the password is still secure, make it longer. As an example:
K#3amvv7l1wz1192sjqhym
This meets old-fashioned password complexity rules, because upper, lower, numbers and special characters are in the first four, the remainder is randomly generated but avoids any problematic characters.
However if you must use them, you can quote the password parameter with single quotes - though you will still run in to trouble if the password contains single quotes!
Try enclosing your password in single quotes.
If it's pa**w0rd, use 'pa**w0rd'
Variables are best used for data, not code. The layers of variables make it hard to protect the expansion when you want some parts of the expansion (i.e., you want your command line to be word split on the tokens you want), but don't want all the other effects. The solution is to not store the code in a string. Instead, use a function like:
do_mysql() {
host="$1"
pass="$2"
mysql --force --connect-timeout=90 --host="$host" -u root --password="$pass" "$#"
}
then you can run it with extra arguments like
do_mysql "$DB_HOST" "$DB_PASS" -e exit > /dev/null && do_mysql "$DB_HOST" "$DB_PASS" < "../mysql/upgrade_schema_v.2.1.sql"
Though it would also be better not to use upper case for your variables. Doing so makes it so you could collide with environment variables and accidentally change things you don't intend to change (as the number of people who accidentally reset PATH can attest).
As I follow from the documentation, The import command in its most basic form is:
mysql.exe < example.sql
It works when I run it from the command line in Windows. But it doesn't work when I start the process mysql.exe with < example.sql parameters. For example, creating a shortcut and setting its path to mysql.exe < example.sql doesn't work and it only prints the help info for mysql.exe.
As a side note, I first noticed this problem when trying to run the following C# code:
new Process
{
StartInfo = new ProcessStartInfo
{
FileName = "mysql.exe",
Arguments = "< example.sql",
}
}.Start();
The part < example.sql does not contitute parameters for mysql.exe; the < character denotes a redirection operator, so the content of file example.sql is redirected into mysql.exe.
I guess you have to change the file name to cmd.exe and the arguments to /C "mysql.exe < example.sql". Consider to specify full absolute paths to all of the files.
I don't work in windows but I assume you best fill in the full path + database hostname information:
C:\mysql\directory\bin\mysql -h {hostname} -u {username} -p {databasename} < example.sql
I'm writing a bash script to do some db stuff. New to MySQL. I'm on Mac and have MySQL installed via homebrew.
Am using username "root" right now and there isn't a pw set. I included the pw syntax below just to help others out that may have a pw.
My goal is to have mysql commands be as "clean" as possible in my bash script
Not a hige deal, but would like to do this if possible.
Example
# If I can do it without logging in (*ideal)
mysql CREATE DATABASE dbname;
# Or by logging in with - mysql -u root -pPassword
CREATE DATABASE dbname;
# Instead of
mysql -u root -pPassword -e"CREATE DATABASE dbname";
Tried to simplify it. I have a handful of things I gotta do, so would rather keep my code cleaner if possible. I tried logging in with the bash script, but the script stopped once logged into MySQL and didn't run any commands.
Another option I was considering (but don't really like) would be just to keep username and pw string in a var and call it for every commmand like so
# Set the login string variable
login_details="-u root -p password -e"
# example command
mysql $login_details"CREATE DATABASE dbname";
So any ideas?
Write a new bash script file and run this file after putting all your commands into it. Don't forget to give right username and password in your bash script.
For bash script:
#!/bin/bash
mysql -u root -pSeCrEt << EOF
use mysql;
show tables;
EOF
If you want to run single mysql command.
mysql -u [user] -p[pass] -e "[mysql commands]"
Example:
mysql -h 192.168.1.10 -u root -pSeCrEt -e "show databases"
To execute multiple mysql commands:
mysql -u $user -p$passsword -Bse "command1;command2;....;commandn"
Note: -B is for batch, print results using tab as the column separator, with each row on a new line. With this option, mysql does not use the history file. Batch mode results in nontabular output format and escaping of special characters. -s is silent mode. Produce less output. -e is to execute the statement and quit
I want to make an alias that is kept in my bashrc file to log into a remote MySQL db via SSH.
Assume that I can't add/alter any files on the remote machine that I'm SSHing into. Here's the relevant code.
function ssh_mysql {
echo "SSHing to $server"
ssh -t -t $suser#$server <<ENDSSH
eval "mysql -h "$host" -u $user -p $pass $db"
ENDSSH
}
alias wt_mysql=ssh_mysql
The Problem: Entering 'wt_mysql' into the terminal as an alias SSHs and logs into MySQL fine.. but when trying to enter any command/query/etc at the MySQL prompt, none of what I've submitted is executed/run. Including the 'exit' command. I have to ctrl C to get back to my local terminal. although its a bit out of my understanding I believe the problem is related to this topic, Terminating SSH session executed by bash script
How can I make sure that mysql and any subsequent commands are executed remotely?
Thanks!
I don't understand why you're using eval (or why you're passing the -t switch twice).
I would expect this ssh command to do what you want:
ssh -t $suser#$server "mysql -h '$host' -u $user -p $pass $db"