How to execute a MySQL query and avoid having the query or an alias in the output ? I tried "" (empty string) as an alias but I didn't get the results expected as I ended up having a blank line.
edit: added some code
SELECT
CONCAT("{\"counters\":{",
-- Total memory used calculation
"\"mysql.total_memory\":",
((##read_buffer_size + ##sort_buffer_size) * ##max_connections + ##key_buffer_size),",",
-- other monitored server status variables
GROUP_CONCAT(
CONCAT("\"mysql.",LCASE(VARIABLE_NAME),"\":",VARIABLE_VALUE)
)
,"}}")
FROM INFORMATION_SCHEMA.GLOBAL_STATUS
WHERE VARIABLE_NAME = "SLOW_QUERIES"
OR VARIABLE_NAME="Qcache_lowmem_prunes"
OR VARIABLE_NAME="SELECT_FULL_JOIN"
OR VARIABLE_NAME="SELECT_RANGE_CHECK"
OR VARIABLE_NAME="SELECT_SCAN"
OR VARIABLE_NAME="SELECT_RANGE";
I want to have a json format as output. I need this as an input for another software. This software doesn't accept a blank line among other things (compressed json format).
edit2: added output
CONCAT("{\"counters\":{",
"\"mysql.total_memory\":",
((##read_buffer_size + ##sort_buffer_size) * ##max_connections + ##key_buffer_size),",",
GROUP_CONCAT(
CONCAT("\"mysql.",LCASE(VARIABLE_NAME),"\":",VARIABLE_VALUE)
)
,"}}")
{"counters":{"mysql.total_memory":39108608,"mysql.qcache_lowmem_prunes":0,"mysql.select_full_join":0,"mysql.select_range":0,"mysql.select_range_check":0,"mysql.select_scan":84,"mysql.slow_queries":0}}
I want to remove the "CONCAT(...)" part and only have the result as output.
I think I know what you mean, you only want the outcome of the query printed without boxes and headers, therefore start your mysql client the following way: mysql -uroot -p -s -r -N.
This will supress output of the boxes arround the querys and also the column names. You can also use the -e Parameter to execute your query and then exit the mysql client after printing the results of your query to stdout, this is usefule when using it in scripts. Please see below (a simplified) example:
[root#db1 ~]# mysql -uroot -p******* -s -r -N -e "select 1+1"
2
[root#db1 ~]#
Every column in the resultset must be named. But you can name with whatever explicit (non-empty) alias you wish. As documented under SELECT Syntax:
A select_expr can be given an alias using AS alias_name.
Therefore, in your case:
SELECT CONCAT(
'{"counters":{',
-- Total memory used calculation
'"mysql.total_memory":', (
(##read_buffer_size + ##sort_buffer_size) * ##max_connections
+ ##key_buffer_size
),',',
-- other monitored server status variables
GROUP_CONCAT('"mysql.',LCASE(VARIABLE_NAME),'":',VARIABLE_VALUE),
'}}'
) AS my_column -- assign your chosen alias here <===================
FROM INFORMATION_SCHEMA.GLOBAL_STATUS
WHERE VARIABLE_NAME IN (
'SLOW_QUERIES',
'Qcache_lowmem_prunes',
'SELECT_FULL_JOIN',
'SELECT_RANGE_CHECK',
'SELECT_SCAN',
'SELECT_RANGE'
);
Related
I'm trying to update multiple rows in a DB using a small script.
I need to update the rows based on some specific user_ids which I have in a list on Linux machine.
#! /bin/bash
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in ()";
As you see above, the user_ids are in a file, let's say /opt/test/user_ids_txt.
How can I import them into this command?
This really depends on the format of user_ids_txt. If we assume it just happens to be in the correct syntax for your SQL in statement, the following will work:
#! /bin/bash
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in ($(< /opt/test/user_ids_txt))";
The bash interpreter will substitute in the contents of the file. This can be dangerous for SQL queries, so I would echo out the command on the terminal to make sure it is correct before implementing it. You should be able to preview your SQL query by simply running the following on the command line:
echo "update device set in_use=0 where user_id in ($(< /opt/test/user_ids_txt))"
If your file is not in the SQL in syntax you will need to edit it (or a copy of it) before running your query. I would recommend something like sed for this.
Example
Let's say your file /opt/test/user_ids_txt is just a list of user_ids in the format:
aaa
bbb
ccc
You can use sed to edit this into the correct SQL syntax:
sed 's/^/\'/g; s/$/\'/g; 2,$s/^/,/g' /opt/test/user_ids_txt
The output of this command will be:
'aaa'
,'bbb'
,'ccc'
If you look at this sed command, you will see 3 separate commands separated by semicolons. The individual commands translate to:
1: Add ' to the beginning of every line
2: Add ' to the end of every line
3: Add , to the beginning of every line but the first
Note: If your ID's are strictly numeric, you only need the third command.
This would make your SQL query translate to:
update device set in_use=0 where user_id in ('aaa'
,'bbb'
,'ccc')
Rather than make a temporary file to store this, I would use a bash variable, and simply plug that into the query like this:
#! /bin/bash
in_statement="$(sed 's/^/\'/g; s/$/\'/g; 2,$s/^/,/g' /opt/test/user_ids_txt)"
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in (${in_statement})";
All of the other variables that make this work are tested and working correctly so I'm obviously doing this wrong.
I have a bash script that first selects some mysql data and stores into a new variable.
Then it goes on to connect again and update the database.
title=$(mysql -u $user -p$pass -h $host dbname | SELECT post_title FROM wp_posts WHERE ID=$8);
mysql --host=$host --user=$user --password=$pass dbname <<EOF
UPDATE wp_my_music_lib SET title = "$title" WHERE track_id=${4}${6};
EOF
The title entry is always blank which says to me that the initial SELECT isn't working properly. It should also be noted that the data expected from the select result has white space and special chars in it ie :
Some Artist (10/10/13)
I thought quoting the var "$title" would fix any potential problems with gobbling but that isn't the issue here as I've tried selecting a single numerical object from a different column and that doesn't work either.
If I hard code the title var it works as expected.
1) Can you see what I'm doing wrong?
2) Is it possible to perform all of the above with one db connection instead as that would make more sense?
mysql | SELECT pipes the output of mysql to a command called SELECT, which is сertainly not what you want.
To execute a query via mysql and capture the output you can use this syntax:
title=$(mysql -B dbname <<< "SELECT post_title FROM wp_posts WHERE ID=$8")
You could also execute the SELECT in a subquery to avoid multiple calls to mysql:
mysql --host=$host --user=$user --password=$pass dbname <<EOF
UPDATE wp_my_music_lib SET title = (
SELECT post_title FROM wp_posts WHERE ID=$8)
WHERE track_id=${4}${6}
EOF
I am storing output of MySQL query in a varible using shell scripting. The output of SQL query is in multiple rows. When I checked the count of the variable (which I think is an array), it is giving 1. My code snippet is as follows:
sessionLogin=`mysql -ugtsdbadmin -pgtsdbadmin -h$MYSQL_HOST -P$MYSQLPORT CMDB -e " select distinct SessionID div 100000 as 'MemberID' from SessionLogin where ClientIPAddr like '10.104%' and LoginTimestamp > 1426291200000000000 order by 1;"`
echo "${#sessionLogin[#]}"
How can I store the MySQL query output in an array in shell scripting?
You can loop over the output from mysql and append to an existing array. For example, in Bash 3.1+, a while loop with process substitution is one way to do it (please replace the mysql parameters with your actual command)
output=()
while read -r output_line; do
output+=("$output_line")
done < <(mysql -u user -ppass -hhost DB -e "query")
echo "There are ${#output[#]} lines returned"
Also take a look at the always excellent BashFaq
I have a table that holds relative paths to real files on HDD. for example:
SELECT * FROM images -->
id | path
1 | /files/1.jpg
2 | /files/2.jpg
Can I create a query to select all records pointing to non-existent files? I need to check it by MySql server exactly, without using an iteration in PHP-client.
I would go with a query like this:
SELECT id, path, ISNULL(LOAD_FILE(path)) as not_exists
FROM images
HAVING not_exists = 1
The function LOAD_FILE tries to load the file as a string, and returns NULL when it fails.
Please notice that a failure in this case might be due to the fact that mysql simply cannot read that specific location, even if the file actually exists.
EDIT:
As #ostrokach pointed out in comments, this isn't standard SQL, even though MySQL allows it, to follow the standard it could be:
SELECT *
FROM images
WHERE LOAD_FILE(PATH) IS NULL
The MySQL LOAD_FILE command has very stringent requirements on the files that it can open. From the MySQL docs:
[LOAD_FILE] Reads the file and returns the file contents as a string. To use this function, the file must be located on the server host, you must specify the full path name to the file, and you must have the FILE privilege. The file must be readable by all and its size less than max_allowed_packet bytes. If the secure_file_priv system variable is set to a non-empty directory name, the file to be loaded must be located in that directory.
So if the file can't be reached by the mysql user or any of the other requirements are not satisfied, LOAD_FILE will return Null.
You can get a list of IDs that correspond to missing files using awk:
mysql db_name --batch -s -e "SELECT id, path FROM images" \
| awk '{if(system("[ -e " $2 " ]") == 1) {print $1}}' \
>> missing_ids.txt
or simply using bash:
mysql db_name --batch -s -e "SELECT id, path FROM images" \
| while read id path ; if [[ -e "$path" ]] ; then echo $id ; done
>> missing_ids.txt
This also has the advantage of being much faster than LOAD_FILE.
MYSQL only handles the Database so there is no way for you to fire an SQL Statement to check on the HDD if the file exists. You need to iterate over the rows and check it with PHP.
It's not possible using stock MySQL. However you can write UDF (user-defined function), probably in C, load it using CREATE FUNCTION statement and use it from MySQL as you would use any built-in function.
i have a view and want to extract its data into a file that has create table statement as well data.
i know that mysqldump doesn't work on views.
Obviously, there isn't an automated way to generate the CREATE TABLE statement of a table that does not exist. So you basically have two options:
Create an actual table, dump it and remove it afterwards.
Write a lot of code to analyse the view and underlying tables and generate the appropriate SQL.
First option is not optimal at all but it's easy to implement:
CREATE TABLE my_table AS
SELECT *
FROM my_view
You can now dump the table with mysqldump. When you're done:
DROP TABLE my_table
Second option can be as optimal as you need but it can get pretty complicate and it depends a lot on your actual needs and tool availability. However, if performance is an issue you can combine both approaches in a quick and dirty trick:
CREATE TABLE my_table AS
SELECT *
FROM my_view
LIMIT 1;
SHOW CREATE TABLE my_table;
Now, you use your favourite language to read values from my_view and build the appropriate INSERT INTO code. Finally:
DROP TABLE my_table;
In any case, feel free to explain why you need to obtain SQL code from views and we may be able to find better solutions.
Use SELECT ... INTO OUTFILE to create a dump of the data.
I have written a bash function to export the "structure" and data of a VIEW without creating a full copy of the data. I tested it with MySQL 5.6 on a CentOS 7 server. It properly takes into account columns with JSON values and strings like "O'Mally", though you may need to tweak it further for other special cases.
For the sake of brevity, I did not make it robust in terms of error checks or anything else.
function export_data_from_view
{
local DB_HOST=$1
local SCHEMA=$2
local VIEW=$3
local TMP_TABLE_NAME="view_as_table_$RANDOM"
local SQL1="
create table $TMP_TABLE_NAME as
(select * from $VIEW where 1=0);
show create table $TMP_TABLE_NAME \G
"
# Create an empty table with the structure of all columns in the VIEW.
# Display the structure. Delete lines not needed.
local STRUCT=$(
mysql -h $DB_HOST -BANnq -e "$SQL1" $SCHEMA |
egrep -v "\*\*\*.* row \*\*\*|^${TMP_TABLE_NAME}$" |
sed "s/$TMP_TABLE_NAME/$VIEW/"
)
echo
echo "$STRUCT;"
echo
local SQL2="
select concat( 'quote( ', column_name, ' ),' )
from information_schema.columns
where table_schema = '$SCHEMA'
and table_name = '$VIEW'
order by ORDINAL_POSITION
"
local COL_LIST=$(mysql -h $DB_HOST -BANnq -e "$SQL2")
# Remove the last comma from COL_LIST.
local COL_LIST=${COL_LIST%,}
local SQL3="select $COL_LIST from $VIEW"
local INSERT_STR="insert into $VIEW values "
# Fix quoting issues to produce executeable INSERT statements.
# \x27 is the single quote.
# \x5C is the back slash.
mysql -h $DB_HOST -BANnq -e "$SQL3" $SCHEMA |
sed '
s/\t/,/g; # Change each TAB to a comma.
s/\x5C\x5C\x27/\x5C\x27/g; # Change each back-back-single-quote to a back-single-quote.
s/\x27NULL\x27/NULL/g; # Remove quotes from around real NULL values.
s/\x27\x27{/\x27{/g; # Remove extra quotes from the beginning of a JSON value.
s/}\x27\x27/}\x27/g; # Remove extra quotes from the end of a JSON value.
' |
awk -v insert="$INSERT_STR" '{print insert "( " $0 " );"}'
local SQL4="drop table if exists $TMP_TABLE_NAME"
mysql -h $DB_HOST -BANnq -e "$SQL4" $SCHEMA
echo
}