Error with JSON and Shell Script - json

I have tried to make a slack notification with a shell script.
The JSON parameters are formed by variables what they are obtained by MySql querys.
#!/bin/sh
#MySQL RO Access
host='mysqlserver.com'
userdb='slackro'
password='password'
db='db'
#Slack information
hook='https://hook.slack'
user='slackusr'
channel='o_channel'
emoji='slackusr'
#Query
id=`mysql -D $db -u $userdb -p$password -e 'SELECT id FROM ticket WHERE tn ='$1'' -h $host | sed -e '1d'`
tn=`mysql -D $db -u $userdb -p$password -e 'SELECT tn FROM ticket WHERE tn ='$1'' -h $host | sed -e '1d'`
title=`mysql -D $db -u $userdb -p$password -e 'SELECT title FROM ticket WHERE tn ='$1'' -h $host | sed -e '1d' | sed "s/'/ /g" | sed "s/°//g" | sed "s/ /_/g" `
customer=`mysql -D $db -u $userdb -p$password -e 'SELECT customer_id FROM ticket WHERE tn ='$1'' -h $host | sed -e '1d'`
msj=`mysql -D $db -u $userdb -p$password -e 'SELECT a_body FROM article WHERE ticket_id ='$id' ORDER BY id DESC LIMIT 1' -h $host | sed -e '1d'`
url='http://iiabox.infra.ultra.sur.top/otrs/index.pl?Action=AgentTicketZoom;TicketID'$1
#Message
curl -X POST -H 'Content-type: application/json' --data '{"username": "slackusr","icon_emoji": ":slackusr:","attachments": [{"fallback": "New Ticket","pretext": "New ticket from '$customer'","title": "'$title'","title_link": "'$url'","text": "'$msj'","color": "#006495"}]}' $hook
When I execute this script I obtain something like that
curl -X POST -H 'Content-type: application/json' --data '{"username": "OTRS","icon_emoji": ":slackusr:","attachments": [{"fallback": "New Ticket","pretext": "New ticket from my#email.com","title": "Prueba' de Notificación '6","title_link": "http://site/otrs/index.pl?Action=AgentTicketZoom;TicketID2016110472000067","text": "Cerrado","color": "#006495"}]}' https://hooks.slack.com/
curl: (6) Could not resolve host: de
curl: (6) Could not resolve host: xn--notificacin-zeb
curl: (3) [globbing] unmatched close brace/bracket in column 152
I don't understand why the result of the variable $title shows that "Prueba' de Notificación '6"
If I print $title variable with echo I obtain: "Prueba de Notificación 6" without simple quotes before the first space and after the last space.
What can I do?

First: This code is, as a whole, broken beyond repair. Do not use it in production. Rewrite it in a language you're actually good in (and that has SQL libraries that support bind variables so you can fix your security bugs, and JSON libraries that will ensure that content is always correctly quoted), not shell.
That said, as for your immediate problem --
Whenever you do this in a single-quoted context:
"title": "'$title'",
...you're expanding $customer unquoted, meaning that spaces inside the expanded value are used for word-splitting and glob-expansion by the shell.
Instead, make it:
"title": "'"$title"'"
...opening a double-quoted context after terminating the single-quoted context.

Related

Bash script, get data from a DB and echo it

I have a script that search in a DB for a single column and store them in an array, then I use that column values in another query to get multiples columns, and store them in a variable.
Im trying to echo the variable message but when I run the script it doesnt echo anything.
#!/bin/bash
mapfile result < <(mysql -ugroot -p<pass> -D <db> -h <host-ip> -P 3306 -s -N -e "SELECT <query> ;")
for row in "${result[#]}";do
message=`mysql -ugroot -p<pass> -D <db> -h <host-ip> -P 3306 -s -N -e "SELECT <query> AND tg2.nombre = '${row}';"`
echo $message
done
Unless you explicitly need the result array I think you can use:
for row `mysql -ugroot -p<pass> -D <db> -h <host-ip> -P 3306 -s -N -e "SELECT <query> ;"` ; do
message=`mysql -ugroot -p<pass> -D <db> -h <host-ip> -P 3306 -s -N -e "SELECT <query> AND tg2.nombre = '${row}';"`
echo $message
done
BR

Bash script Request body is un-json-able

Im trying to push a data to an API,
for which I get the below error:
("Request body is not unjson()-able: %s" % body)
DirectException: Request body is not unjson()-able: '{"action":"EventsRouter","method":"add_event","data":[{"summary":"mes","device":"hos","message":"message","component":"sev","severity":"ap","evclasskey":"nxlog","evclass":"nxlog","monitor":"localhost"}],"type":"rpc","tid":2}'
current bash file:
#!/bin/bash
message=$1
hostname=$2
appname=$3
severity=$4
data='{"action":"EventsRouter","method":"add_event","data":[{"summary":"'$message'","device":"'$hostname'","message":"message","component":"'$appname'","severity":"'$severity'","evclasskey":"nxlog","evclass":"nxlog","monitor":"localhost"}],"type":"rpc","tid":2}'
echo "Total number of args : $#"
echo "message = $message"
echo "hostname = $hostname"
echo "appname = $appname"
echo "data = $data"
curl -u "Eve:eve#123" -k "https://myurl.com/zport/dmd/evconsole_router" -d "'$data'" -H "Content-Type:application/json"
First, What is the reason and how can i correct this. I am only sending simple values while testing:
sh tcp.sh mes hos sev ap
However, when i curl a data directly, it is working.
curl -u Eve:eve#123 -k https://myurl.com/zport/dmd/evconsole_router -d '{"action":"EventsRouter", "method":"add_event","data":[{"summary":"test","device":"test","message":"msg","component":"testhost","severity":"5", "evclasskey":"nxlog", "evclass":"/nxlog/perf","monitor":"localhost"}],"type":"rpc","tid":2}' -H "Content-Type:application/json"
{"uuid": "1654584c-5f86-489e-a5e7-35d45e462066", "action": "EventsRouter", "result": {"msg": "Created event", "success": true},"tid": 2, "type": "rpc", "method": "add_event"}
Secondly, This is about to be automated to read multiple log messages, if the log message has special characters such as ',",\,/, etc, will it affect my bash inputs?
If yes, how to I eliminate it?
can you please try this syntax and check the execution
data=$(cat <<EOF
{"action":"EventsRouter","method":"add_event","data":[{"summary":"$message","device":"$hostname","message":"message","component":"$appname","severity":"$severity","evclasskey":"nxlog","evclass":"nxlog","monitor":"localhost"}],"type":"rpc","tid":2}
EOF
)
Curl command syntax like
curl -u "Eve:eve#123" -k "https://myurl.com/zport/dmd/evconsole_router" -d "$data" -H "Content-Type:application/json"

Extract token from a curl request and use in another shell command

I've started picking up bash scripting and I'm stuck at something I'm trying to wrap my head around.
I have a curl command that outputs a token and I need to use it in the following command:
curl -k 'https://server:port/session' -X POST -H 'Content-Type: application/json' -d '{"username":"admin","password":"password"}'
It then outputs a token here:
{"token":"ac07098ad59ca6f3fccea0e2a2f6cb080df55c9a52fc9d65"}
I then need to use it in the follow up command
curl https://server:port/ -k -X POST -H 'Content-Type: application/json' -H 'X-Cookie:token=token' -d '
I was thinking I could output the token to a file, then have a sed command write the token to a file, then the new command use a variable where token=$token
Thanks!
This is where a JSON parsing tool comes in handy (such as jq):
$ echo '{"token":"ac07098ad59ca6f3fccea0e2a2f6cb080df55c9a52fc9d65"}' | jq -r .token
ac07098ad59ca6f3fccea0e2a2f6cb080df55c9a52fc9d65
So
json=$( curl -k 'https://server:port/session' -X POST -H 'Content-Type: application/json' -d '{"username":"admin","password":"password"}' )
token=$( jq -r ".token" <<<"$json" )
curl https://server:port/ -k -X POST -H "X-Cookie:token=$token" ...
With no further tool than a bash (tested Centos/Rhel7/GitBash) :
json=$(curl -k 'https://server:port/session' \
-X POST -H 'Content-Type: application/json' \
-d '{"username":"admin","password":"password"}') \
&& token=$(echo $json | sed "s/{.*\"token\":\"\([^\"]*\).*}/\1/g") \
&& echo "token = $token"
then use your authentication needing commands like that :
curl https://server:port/ -k -X POST \
-H 'Content-Type: application/json' \
-H 'X-Cookie:token=$token' -d ...'
If Python is installed, and hopefully it is on modern systems, you can do something like:
OUTPUT="$(curl -k 'https://server:port/session' -X POST -H 'Content-Type: application/json' -d '{"username":"admin","password":"password"}' | python -c "import sys, json; print json.load(sys.stdin)['token']")"
This will give you:
echo $OUTPUT
ec2e99a1d294fd4bc0a04da852ecbdeed3b55671c08cc09f
Use the `` syntax:
cmd1result=$(command1 | cut -d ':' -f 2 | grep -Po "[a-z0-9-]+")
command2 $cmd1result

Getting Google Drive APK access_token programatically

I have a linux OS based custom embedded system. There are cameras connected to my system and i want to use google drive as cloud storage for records that were taken from my cameras.
For that purpose i used almost all google drive api functions successfully by following google drive api rest documentation in
https://developers.google.com/drive/v3/web/about-sdk
Restful api functions require an access_token for authentication purpose in accordance with OAuth2 protocol in HTTP requests. Getting this token requires some one time manual operations.
1- When signed in to google with my account, i had to create a client_id and a client_secure using google developer console.
2- Then i use these credentials to get access_code. In a response to a http request google sends me a url and a grant code.
3- Then i visit the url using a browser, typing grant code and allowing for authentication manually. Then i get access_token and refresh_token with another HTTP request.
4- After that point i can use any api function successfully with given access_token. (and i refresh it if it expires using refresh_token)
My problem is one time manual operations that are made to get tokens. I am making these operation in my computer so making them in my embedded system looks very difficult. But i want to make all the steps (signing in, client_id and client_secure, visiting url, typing grant code and allowing for authentication) in my embedded system.
So, I wonder if it is possible doing all this one time manual process to get access_code without using developer console and a browser? Can i make them programatically?
This is what i tried following oauth2 google official documentation.
After creating client_id and client_Secret i simply used curl for http operations. I wrote following bash code for testing purpose. So It is not self confident and requires some files which can be empty at first and named "access_token", "folder_id", "refresh_token", "myFile" files in the same directory.
#!/bin/bash
# Google Drive API
get_file_id() {
RESPONSE=`curl --silent -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files?q=title+contains+\'$1\'\&fields=items%2Fid`
FILE_ID=`echo $RESPONSE | python -mjson.tool | grep -oP 'id"\s*:\s*"\K(.*)"' | sed 's/"//'`
}
set -e
CLIENT_SECRET="my client_secret"
CLIENT_ID="my_client_id"
BOUNDARY="foo_bar_baz"
SCOPE="https://docs.google.com/feeds"
MIME_TYPE="application/octet-stream"
ACCESS_TOKEN=`cat access_token`
REFRESH_TOKEN=`cat refresh_token`
FOLDER_ID=`cat folder_id`
if [ "$1" == "create_token" ]; then # Usage: <"create_token">
RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/device/code" -d "client_id=$CLIENT_ID&scope=$SCOPE"`
DEVICE_CODE=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'device_code"\s*:\s*"\K(.*)"' | sed 's/"//'`
USER_CODE=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'user_code"\s*:\s*"\K(.*)"' | sed 's/"//'`
URL=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'verification_url"\s*:\s*"\K(.*)"' | sed 's/"//'`
echo -n "Go to $URL and enter $USER_CODE to grant access to this application. Hit enter when done..."
read
RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/token" -d "client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&code=$DEVICE_CODE&grant_type=http://oauth.net/grant_type/device/1.0"`
ACCESS_TOKEN=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'access_token"\s*:\s*"\K(.*)"' | sed 's/"//'`
REFRESH_TOKEN=`echo "$RESPONSE" | python -mjson.tool | grep -oP 'refresh_token"\s*:\s*"\K(.*)"' | sed 's/"//'`
echo "Access Token: $ACCESS_TOKEN"
echo "Refresh Token: $REFRESH_TOKEN"
echo "$ACCESS_TOKEN" > access_token
echo "$REFRESH_TOKEN" > refresh_token
elif [ "$1" == "refresh_token" ]; then # Usage: <"refresh_token">
RESPONSE=`curl --silent "https://accounts.google.com/o/oauth2/token" --data "client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&refresh_token=$REFRESH_TOKEN&grant_type=refresh_token"`
ACCESS_TOKEN=`echo $RESPONSE | python -mjson.tool | grep -oP 'access_token"\s*:\s*"\K(.*)"' | sed 's/"//'`
echo "Access Token: $ACCESS_TOKEN"
echo "$ACCESS_TOKEN" > access_token
elif [ "$1" == "create_folder" ]; then # Usage: <"create_folder">
FOLDER_NAME=`date "+%F-%T"`
( echo -en "{ \"title\": \"$FOLDER_NAME\", \"mimeType\": \"application/vnd.google-apps.folder\" }\n" ) \
| curl -H 'GData-Version: 3.0' -v "https://www.googleapis.com/drive/v2/files" \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--header "Content-Type: application/json" \
--data-binary "#-"
#save FILE_ID to filde
get_file_id $FOLDER_NAME
echo "$FILE_ID" > folder_id
elif [ "$1" == "upload_file" ]; then # Usage: <"upload_file"> <file name>
( echo -en "--$BOUNDARY\nContent-Type: application/json; charset=UTF-8\n\n{ \"title\": \"$2\", \"parents\": [ { \"id\": \"$FOLDER_ID\" } ] }\n\n--$BOUNDARY\nContent-Type: $MIME_TYPE\n\n" \
&& cat $2 && echo -en "\n\n--$BOUNDARY--\n" ) \
| curl -H 'GData-Version: 3.0' -v "https://www.googleapis.com/upload/drive/v2/files/?uploadType=multipart" \
--header "Authorization: Bearer $ACCESS_TOKEN" \
--header "Content-Type: multipart/related; boundary=\"$BOUNDARY\"" \
--data-binary "#-"
elif [ "$1" == "list_files" ]; then # Usage: <"list_files"> <number of files>
curl -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files?maxResults=$2
elif [ "$1" == "download_file" ]; then # Usage: <"download_file"> <file name>
get_file_id $2
curl -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID?alt=media
elif [ "$1" == "get_file" ]; then # Usage: <"get_file"> <file name>
get_file_id $2
curl -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID
elif [ "$1" == "delete_file" ]; then # Usage: <"delete_file"> <file name>
get_file_id $2
curl -X Delete -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID
elif [ "$1" == "trash_file" ]; then # Usage: <"trash_file"> <file name>
get_file_id $2
curl -d -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID/trash
elif [ "$1" == "untrash_file" ]; then # Usage: <"untrash_file"> <file name>
get_file_id $2
curl -d -H 'GData-Version: 3.0' -H "Authorization: Bearer $ACCESS_TOKEN" \
https://www.googleapis.com/drive/v2/files/$FILE_ID/untrash
fi
exit 0
Regards
One way to do this would be to use Domain-Wide Delegation of Authority, a service account. This method works with a JWT for the authentication. The process is explained here: Using OAuth 2.0 for Server to Server Applications
The app would use the service account to authenticate as the id that you want t use to store the drive files.

Catch Mysql Error in bash

I have a bash script that queries mysql and I would like to catch the error and echo it . Here is my code
mysql --silent -h ${sql_server} -P 3306 -u${USER} -p${MYSQL_PASS} -D${DATABASE} -BNe "$sql_str" | while read -r line
do
query="$(echo "$line" | cut -f1)"
database_out="$(echo "$line" | cut -f2)"
outfile="$(echo "$line" | cut -f3)"
directory_out="$(echo "$line" | cut -f4)"
type="$(echo "$line" | cut -f5)"
host="$(echo "$line" | cut -f6)"
username="$(echo "$line" | cut -f7)"
directory_scp="$(echo "$line" | cut -f8)"
#switch over format types
case "$type" in
csv)
file_out=$directory_out$outfile"."`date +%Y%m%d`.csv
mysql -h ${sql_server} -P 3306 -u${USER} -p${MYSQL_PASS} -D${database_out} -e "$query" | sed 's/\t/","/g;s/^/"/;s/$/"/;s/\n//g' > $file_out
;;
xml)
file_out=$directory_out$outfile`date +%Y%m%d`.xml
mysql -h ${sql_server} -P 3306 -u${USER} -p${MYSQL_PASS} -D${database_out} -e "$query" -X > $file_out
;;
*)
echo "["$(timestamp)"] : {ERROR} Invalid output format. Currently only csv and xml output supported."
;;
esac
str=$file_out" "$username#$host:$directory_scp
echo "["$(timestamp)"] : {NOTIFICATION} Attempting to scp "$outfile" to "$username#$host:$directory_scp" - [Started]"
scp $str
echo "["$(timestamp)"] : {NOTIFICATION} scp "$outfile" to "$username#$host:$directory_scp" - [Complete]"
done
Any ideas. I would like to catch error and format the error output.
result=$(mysql --silent -h ${sql_server} -P 3306 -u${USER} -p${MYSQL_PASS} -D${DATABASE} -BNe "$sql_str")
if [ $? -ne "0" ]; then
echo "["$(timestamp)"] : {ERROR} MySQL error thrown in EOD DUMP"
else
while read -r line
...do something