I use the following construct in a shell script (using dash, but I think in that case there is no difference to bash):
for FIELD in `echo "select column_with_varchar_type from table" | mysql MYDATABASE`
do
echo "FIELD: $FIELD"
done
This works fine for normal strings, but it fails for an entry where star ("*") is in column_with_varchar_type, then dash expands this to use all files in the current working directory which is of course completely wrong.
Is there a way to prevent the shell expansion or replace the star with something else in the mysql-query?
Clarification: I do NOT want to select all columns "select *", but I want to select all entries of one column and one of these entries is a string that is a star. The input to the query does not contain any stars and is fine, the problem lies in the output of the query.
For example, if the table contains the strings "aaa", "bbb" and "*", a typical output may be:
FIELD: aaa
FIELD: bbb
FIELD: Makefile
FIELD: src
FIELD: some_other_file_that_happens_to_be_in_the_working_directory.txt
So I get the strings "aaa" and "bbb" (correct), but "*" is missing and is replaced with whatever garbage is in the working directory.
Updated
As discussed in comments, this can be an approach:
while read -r field
do
echo "FIELD: $field"
done < <(echo "select column_with_varchar_type from table" | mysql MYDATABASE)
Because the problem was with the data coming from the query, not from the query itself. Hence, it is necessary to handle the strings with * that are coming from such query and while -r solves it.
The problem is that * gets expanded by the shell when it is within double quotes.
You can use single quotes instead:
for FIELD in $(echo 'select * from table' | mysql MYDATABASE)
do
do_something
done
Note you could also use -e option:
mysql MYDATABASE -e 'select * from table'
try this:
for FIELD in `echo 'select \* from table' | mysql MYDATABASE`
do
do_something
done
The solution is to add quotes in the MySQL-query:
for FIELD in `mysql MYDATABASE -e 'select concat(concat("\"", column_with_varchar_type), "\"") from table'`
do
echo "FIELD: $FIELD"
done
Related
I intend to execute a SQL as following and get the result outputted to terminal:
echo "select * from tbl limit 10" | mysql -hxxx -uxxx -pxxx -Dxxx
there's one field's content having single-quote in it, which is a JSON. I'd like the result being "{\'a\': 1}", yet what it yields is "{'a': 1}".
OUTFILE could not be applied in current scenario. Could anyone help out? Thanks!
I am trying to get a quoted csv output from beeline. My query looks like:
beeline -u 'jdbc:hive2://localhost:10000/' --outputformat=csv2 -e 'set system:disable.quoting.for.sv=false; select 1 as a, 2 as b'
I expect as output
"a","b"
"1","2"
But only get the non-quoted version.
According to the documentation:
The quoting can be disabled by setting the disable.quoting.for.sv system variable to true.
I suppose I am doing something wrong here. How can I set this variable to false? Having it in my query does not seem to do the trick, having it on hive-site.xml neither.
Note: hive 1.2.1 on hdp 2.5.
Quoting fields in csv is only necessary/useful when the field contains a separator or a newline.
So if you try to write 1 2,3, the last value should be quoted: 1,"2,3".
Quoting a field does not mean it should be treated as a string. The "2,3" is still a number (if the , is the decimal separator in this case).
Some csv writers add quotes around every field, which is totally unnecessary and only makes the file larger without any benefit.
The documentation seems to be clear about this too:
If quoting is not disabled, double quotes are added around a value if
it contains special characters (such as the delimiter or double quote
character) or spans multiple lines.
Embedded double quotes are escaped with a preceding double quote.
You can try:
beeline --disableQuotingForSV=true --verbose=true -u 'jdbc:hive2://localhost:10000/' --outputformat=csv2 -e 'select 1 as a, 2 as b'
this will work:
env HADOOP_CLIENT_OPTS="-Ddisable.quoting.for.sv=false" beeline -u 'jdbc:hive2://localhost:10000/' --outputformat=csv2 -e 'select 1 as a, 2 as b'
I was excited to see how easy it is to write a bash script to interact with MySQL.
But trying this:
#!/bin/bash
res=`mysql -u $USER -p$PASS students <<EOF | tail -n +2
SELECT name FROM table WHERE age = 20 limit 1;
EOF`
for d in $res;
do
echo Result : $d
done
If the result is "John Smith" I get:
Result: John
Result: Smith
How can I get around this issue with the space?
It seems like it treats it as 2 values while it is a single column.
One way to do what you ask is adding this before loop:
IFS=$'\n'
This will change default bash internal field separator (IFS), which by default works with spaces, tabs and new lines.
My example will only work with new lines, as probably this is what you're looking for.
I have the following select statement, for which the result is sent to a command line parameter:
"SELECT show.file \
FROM show, schedule \
WHERE channel = 1 AND start_time <= UNIX_TIMESTAMP()"
However, if the result returned has spaces in it, it will cause the command to fail. How can I escape out any spaces in the result? Note that this select statement will only ever return one result.
SELECT REPLACE(show.file, " ", "\ ")
FROM show, schedule
WHERE channel = 1 AND start_time <= UNIX_TIMESTAMP()
Should do the trick. If you have another escape character replace \ with the respective one.
See also
http://dev.mysql.com/doc/refman/5.1/de/string-functions.html
If Markus' solution didn't work, try using the 'xargs' command.
xargs YOUR_COMMAND YOUR_QUERY_RESULT_AS_PARAMETER
depending on the command you're gonna use, it may or may not work.
I need to set linesize to some queries I perform which are quite important. Does anyone know how?
If you are looking for an equivalent output formatting in MySQL to Oracle's linesize & pagesize, I would suggest using raw output mode. If you need line wrapping at a certain length rather than the full unwrapped line, you can pass it through a command line utility such as Unix fmt:
echo "SELECT col1, col2 FROM tab1;" | mysql --raw --batch -uuser -p dbname | fmt --width=80