I am new to shell scripting and MQTT.
I need to publish a JSON file using MQTT. We can do it by storing the JSON contents in a shell variable. But it is not working for me.
my shell script:
#!/bin/sh
var1='{"apiVersion":"2.1","data":{"id":"4TSJhIZmL0A","uploaded":"2008-07-15T18:11:59.000Z","updated":"2013-05-01T21:01:49.000Z","uploader":"burloandbardsey","category":"News","title":"bbc news start up theme","description":"bbc","thumbnail":{"sqDefault":"http://i.ytimg.com/vi/4TSJhIZmL0A/default.jpg","hqDefault":"http://i.ytimg.com/vi/4TSJhIZmL0A/hqdefault.jpg"},"player":{"default":"http://www.youtube.com/watch?v=4TSJhIZmL0A&feature=youtube_gdata_player","mobile":"http://m.youtube.com/details?v=4TSJhIZmL0A"},"content":{"5":"http://www.youtube.com/v/4TSJhIZmL0A?version=3&f=videos&app=youtube_gdata","1":"rtsp://v5.cache7.c.youtube.com/CiILENy73wIaGQlAL2aGhIk04RMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp","6":"rtsp://v5.cache7.c.youtube.com/CiILENy73wIaGQlAL2aGhIk04RMYESARFEgGUgZ2aWRlb3MM/0/0/0/video.3gp"},"duration":15,"aspectRatio":"widescreen","rating":4.6683936,"likeCount":"354","ratingCount":386,"viewCount":341066,"favoriteCount":0,"commentCount":155,"accessControl":{"comment":"allowed","commentVote":"allowed","videoRespond":"allowed","rate":"allowed","embed":"allowed","list":"allowed","autoPlay":"allowed","syndicate":"allowed"}}}'
mosquitto_pub -h localhost -t test -m "$var1"
echo "$var1"
my Mosquitto commands:
Publisher: `mosquitto_pub -h localhost -t "test" -m "{"Contents":$var1}"
Subscriber: mosquitto_sub -h localhost -t "test"
Output I got:
{"Contents":}
Expected Output:
{"Contents":{"name":"Harini", "age":24, "city":"NewYork", "message":"Hello world"}}
I can get the output only at the terminal because of echo. But I want to publish and subscribe to the contents of the shell variable(var1)
Please help me out to get the output. Whether I need to add some more code in the shell script. I don't know how to proceed. Or can you suggest any other method.
The following works just fine, it's all about which quotes you use where:
#!/bin/sh
var1='{"name":"Harini", "age":24, "city":"NewYork","message":"Hello world"}'
echo $var1
mosquitto_pub -t test -m "{\"Content\": $var1}"
You need to wrap the -m argument in quotes because it contains spaces, which in turn means you need to escape the " round Content.
Wrapping the content of var1 in single quotes means you don't need to escape the double quotes in it.
Related
To my eyes the following JSON looks valid.
{
"DescribeDBLogFiles": [
{
"LogFileName": "error/postgresql.log.2022-09-14-00",
"LastWritten": 1663199972348,
"Size": 3032193
}
]
}
A) But, jq, json_pp, and Python json.tool module deem it invalid:
# jq 1.6
> echo "$logfiles" | jq
parse error: Invalid numeric literal at line 1, column 2
# json_pp 4.02
> echo "$logfiles" | json_pp
malformed JSON string, neither array, object, number, string or atom,
at character offset 0 (before "\x{1b}[?1h\x{1b}=\r{...") at /usr/bin/json_pp line 51
> python3 -m json.tool <<< "$logfiles"
Expecting value: line 1 column 1 (char 0)
B) But on the other hand, if the above JSON is copy & pasted into an online validator, both 1 and 2, deem it valid.
As hinted by json_pp's error above, hexdump <<< "$logfiles" indeed shows additional, surrounding characters. Here's the prefix: 5b1b 313f 1b68 0d3d 1b7b ...., where 7b is {.
The JSON is output to a logfiles variable by this command:
logfiles=$(aws rds describe-db-log-files \
--db-instance-identifier somedb \
--filename-contains 2022-09-14)
# where `aws` is
alias aws='docker run --rm -it -v ~/.aws:/root/.aws amazon/aws-cli:2.7.31'
> bash --version
GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
Have perused this GitHub issue, yet can't figure out the cause. I suspect that double quotes get mangled somehow when using echo - some reported that printf "worked" for them.
The use of docker run --rm -it -v command to produce the JSON, added some additional unprintable characters to the start of the JSON data. That makes the resulting file $logfiles invalid.
The -t option allocations a tty and the -i creates an interactive shell. In this case the -t is allowing the shell to read login scripts (e.g. .bashrc). Something in your start up scripts is outputting ansi escape codes. Often this will to clear the screen, set up other things for the interactive shell, or make the output more visually appealing by colorizing portions of the data.
Add bash variables value to json file
I am trying to get latest zip file from nexus using below curl command.
Ouput of curl comes like this : 1.0.0.0-20190205.195251-396
In the json field i need this value(1.0.0.0-20190205.195251-396) to be updated like this: developer-service-1.0.0.0-20190205.195251-396.zip
ERROR: [2019-02-06T16:19:17-08:00] WARN: remote_file[/var/chef/cache/developer-service-.zip] cannot be downloaded from https://nexus.gnc.net/nexus/content/repositories/CO-Snapshots/com/GNC/platform/developer/developer-service/1.0.9.9-SNAPSHOT/developer-service-.zip: 404 "Not Found"
#!/bin/bash
latest=`curl -s http://nexus.gnc.net/nexus/content/repositories/CO-Snapshots/com/gnc/platform/developer/developer-service/1.0.9.9-SNAPSHOT/maven-metadata.xml | grep -i value | head -1 | cut -d ">" -f 2 | cut -d "<" -f 1`
echo $latest
sudo bash -c 'cat << EOF > /etc/chef/deploy_service.json
{
"portal" : {
"nexus_snapshot_version":"developer-service-${latest}.zip"
}
}
EOF'
The problem is that when you use "${latest}", it's inside single-quotes, and hence not treated as a variable reference, just some literal text; it's passed to a subshell (the bash -c, and that will parse it as a variable reference and replace it with the value of the variable latest, but that variable is only defined in the parent shell, not in the subshell. You could probably export the variable (so it gets inherited by subprocesses) and use sudo -E to prevent sudo from cleaning the environment (hence removing the variable)... But this whole thing is an overcomplicated mess; there's a much simpler way, using the standard sudo tee trick:
sudo tee ./deploy_service.json >/dev/null <<EOF
{
"portal" : {
"nexus_snapshot_version":"developer-service-${latest}.zip"
}
}
EOF
This way there's not single-quoted string, no subshell, etc. The variable reference is now just in a plain here-document (that's interpreted by the shell that knows $latest), and gets expanded normally.
I'm trying to write a sample script where I'm generating names like 'student-101...student-160'. I need to post JSON data and when I do, I get a JSON parse error.
Here's my script:
name="student-10"
for i in {1..1}
do
r_name=$name$i
echo $r_name
curl -i -H 'Authorization: token <token>' -d '{"name": $r_name, "private": true}' "<URL>" >> create_repos_1.txt
echo created $r_name
done
I always get a "Problems parsing JSON" error. I've tried various combination of quotes, etc but nothing seems to work!
What am I doing wrong?
First, your name property is a string, so you need to add double quotes to it in your json.
Second, using single quotes, bash won't do variable expansion: it won't replace $r_name with the variable content (see Expansion of variable inside single quotes in a command in bash shell script for more information).
In summary, use:
-d '{"name": "'"$r_name"'", "private": true}'
Another option is to use printf to create the data string:
printf -v data '{"name": "%s", "private": true}' "$r_name"
curl -i -H 'Authorization: token <token>' -d "$data" "$url" >> create_repos_1.txt
Don't; use jq (or something similar) to build correctly quoted JSON using variable inputs.
name="student-10"
for i in {1..1}
do
r_name=$name$i
jq -n --arg r_name "$r_name" '{name: $r_name, private: true}' |
curl -i -H 'Authorization: token <token>' -d #- "<URL>" >> create_repos_1.txt
echo created $r_name
done
The #- argument tells curl to read data from standard input (via the pipe from jq) to use for -d.
Something like "{\"name\": \"$r_name\", \"private\": true}" may work, but it is ugly and will also fail if r_name contains any character which needs to be quoted in the resulting JSON, such as double quotes or ASCII control characters.
How can I change this curl command to make it work
It is something about using the $# param that github starts complaining
function create_repo(){
curl -u 'USER' https://api.github.com/user/repos -d '{"name":$#}'
}
It works if I hardcode the param as a string
Your command uses a singly-quoted string, inside which variables are usually not interpolated (though you haven't specified a particular shell).
Try this instead:
function create_repo(){
curl -u 'USER' https://api.github.com/user/repos -d "{\"name\":\"$#\"}"
}
Note that we use \" instead of ' for our inner quotes because JSON requires double quotes.
I'm having an issue passing variables to a Bash script using QSub.
Assume I have a Bash script named example. The format of example is the following:
#!/bin/bash
# (assume other variables have been set)
echo $1 $2 $3 $4
So, executing "bash example.sh this is a test" on Terminal (I am using Ubuntu 12.04.3 LTS, if that helps) produces the output "this is a test".
However, when I enter "qsub -v this,is,a,test example.sh", I get no output. I checked the output file that QSub produces, but the line "this is a test" is nowhere to be found.
Any help would be appreciated.
Thank you.
Using PBSPro or SGE, arguments can simply be placed after the script name as may seem intuitive.
qsub example.sh hello world
In Torque, command line arguments can be submitted using the -F option. Your example.sh will look something like this:
#!/bin/bash
echo "$1 $2"
and your command like so:
qsub -F "hello world" example.sh
Alternatively, environment variables can be set using -v with a comma-separated list of variables.
#!/bin/bash
echo "$FOO $BAR"
and your command like so:
qsub -v FOO="hello",BAR="world" example.sh
(This may be better phrased as a comment on #William Hay's answer, but I don't have the reputation to do so.)
Not sure which batch scheduler you are using but on PBSPro or SGE then submitting with qsub example.sh this is a test should do what you want.
The Torque batch scheduler doesn't (AFAIK) allow passing command line arguments to the script this way. You would need to create a script looking something like this.
#!/bin/bash
echo $FOO
Then submit it with a command like:
qsub -v FOO="This is a test" example.sh