I'm trying to use a variable to identify mxf or mov file extensions. The following works where I explicitly name the file extensions with a regular expression.
${FSWATCH_PATH} -0 \
-e ".*" --include ".*\.[ mxf|mov ]" \
--event Updated --event Renamed --event MovedTo -l $LATENCY \
$LOCAL_WATCHFOLDER_PATH \
| while read -d "" event
do
<code here>
done
How can I use a variable for the file extensions, where the variable name is FileTriggerExtensions? The code below doesn't work:
FileTriggerExtensions=mov|mxf
${FSWATCH_PATH} -0 \
-e ".*" --include ".*\.[ $FileTriggerExtensions ]" \
--event Updated --event Renamed --event MovedTo -l $LATENCY \
$LOCAL_WATCHFOLDER_PATH \
| while read -d "" event
do
done
I guess you use Bash or a similar shell?
FileTriggerExtensions=mov|mxf
-bash: mxf: command not found
Use quotes or escape the pipe symbol.
Related
I am following this tutorial from Vault about creating your own certificate authority. I'd like to separate the response (change the output to API call using cURL to see the response) into two distinct files, one file possessing the certificate and issuing_ca attributes, the other file containing the private_key. The tutorial is using jq to parse JSON objects, but my unfamiliarity with jq isn't helpful here, and most searches are returning info on how to merge JSON using jq.
I've tried running something like
vault write -format=json pki_int/issue/example-dot-com \
common_name="test.example.com" \
ttl="24h" \
format=pem \
jq -r '.data.certificate, .data.issuing_ca > test.cert.pem \
jq -r '.data.private_key' > test.key.pem
or
vault write -format=json pki_int/issue/example-dot-com \
common_name="test.example.com" \
ttl="24h" \
format=pem \
| jq -r '.data.certificate, .data.issuing_ca > test.cert.pem \
| jq -r '.data.private_key' > test.key.pem
but no dice.
It is not an issue with jq invocation, but the way the output files get written. Per your usage indicated, after writing the file test.cert.pem, the contents over the read end of the pipe (JSON output) is no longer available to extract the private_key contents.
To duplicate the contents over at the write end of pipe, use tee along with process substitution. The following should work on bash/zsh or ksh93 and not on POSIX bourne shell sh
vault write -format=json pki_int/issue/example-dot-com \
common_name="test.example.com" \
ttl="24h" \
format=pem \
| tee >( jq -r '.data.certificate, .data.issuing_ca' > test.cert.pem) \
>(jq -r '.data.private_key' > test.key.pem) \
>/dev/null
See this in action
jq -n '{data:{certificate: "foo", issuing_ca: "bar", private_key: "zoo"}}' \
| tee >( jq -r '.data.certificate, .data.issuing_ca' > test.cert.pem) \
>(jq -r '.data.private_key' > test.key.pem) \
>/dev/null
and now observe the contents of both the files.
You could abuse jq's ability to write to standard error (version 1.6 or later) separately from standard output.
vault write -format=json pki_int/issue/example-dot-com \
common_name="test.example.com" \
ttl="24h" \
format=pem \
| jq -r '.data as $f | ($f.private_key | stderr) | ($f.certificate, $f.issuing_ca)' > test.cert.pem 2> test.key.pem
There's a general technique for this type of problem that is worth mentioning
because it has minimal prerequisites (just jq and awk), and because
it scales well with the number of files. Furthermore it is quite efficient in that only one invocation each of jq and awk is needed. The idea is to setup a pipeline of the form: jq ... | awk ...
There are many variants
of the technique but in the present case, the following would suffice:
jq -rc '
.data
| "test.cert.pem",
"\t\(.certificate)",
"\t\(.issuing_ca)",
"test.key.pem",
"\t\(.private_key)"
' | awk -F\\t 'NF == 1 {fn=$1; next} {print $2 > fn}'
Notice that this works even if the items of interest are strings with embedded tabs.
I've got a text param (from jenkins job dsl plugin) in jenkins configuration which allows you to enter a multi line comment. I'm using that variable for the body value when posting a release to a github repository from a shell script. I'm getting this error that says problem parsing json and I can't find a workaround. I'll try to give you an example below. Please help.
PERSONAL_ACCESS_TOKEN="random"
TAG_NAME="12.0.0"
VERSION_BUMP="major"
MIGRATION_DOCUMENT="This is first line
This is second line"
curl -i \
-H "Authorization: token ${PERSONAL_ACCESS_TOKEN}" \
-d '{"tag_name": "'"${TAG_NAME}"'", "name": "'"${VERSION_BUMP}"'", \
"body": "'"${MIGRATION_DOCUMENT}"'"}' \
https://github.deere.com/api/v3/repos/randomOrg/testRepo/releases
This
{ "a": "b
c" }
is invalid JSON because a string must not contain control characters such as newlines.
If you have a string containing newlines, you can convert them to \n using shell parameter expansion:
$ var='a
b'
$ echo "$var"
a
b
$ echo "${var//$'\n'/'\n'}"
a\nb
So, to feed your string into your JSON object, use
"body": "'"${MIGRATION_DOCUMENT//$'\n'/'\n'}"'"
at the end of your JSON object.
Also, if you use line continuation in single quotes such as
var='abc \
def'
then the backslash and the linebreak are literal:
$ echo "$var"
abc \
def
Don't use line continuation like that in single quoted strings.
All in all:
curl -i \
-H "Authorization: token ${PERSONAL_ACCESS_TOKEN}" \
-d '{"tag_name": "'"${TAG_NAME}"'", "name": "'"${VERSION_BUMP}"'", "body": "'"${MIGRATION_DOCUMENT//$'\n'/'\n'}"'"}' \
https://github.deere.com/api/v3/repos/randomOrg/testRepo/releases
If you really want to, you can still use line continuation, but it has to be in a double quoted context:
curl -i \
-H "Authorization: token ${PERSONAL_ACCESS_TOKEN}" \
-d '{"tag_name": "'"${TAG_NAME}"'", "name": "'"${VERSION_BUMP}"'", '"\
"'"body": "'"${MIGRATION_DOCUMENT//$'\n'/'\n'}"'"}' \
https://github.deere.com/api/v3/repos/randomOrg/testRepo/releases
As a side note, you shouldn't use all uppercase names for variables; those are reserved for environment variables, see the POSIX spec (fourth paragraph).
I tried the following command to capture the output of a command (grep as an example) with color. But the result is shown as ^[[01;31m^[[Ka^[[m^[[K.
grep --color=always a <<< a |
a2ps -=book -B -q --medium=A4dj --borders=no -o out1.ps &&
gs \
-sDEVICE=png16m \
-dNOPAUSE -dBATCH -dSAFER \
-dTextAlphaBits=4 -q \
-r300x300 \
-sOutputFile=out2.png out1.ps
Is there a way to capture the color in the image? Thanks.
Let's take the following example:
curl -i -X POST \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "Player.Open", "params":{"item":false}}' \
http://example.com/jsonrpc
Now I want to have the boolean value of "item" be set in a shell script variable such as:
PRIVATE=false
read -p "Is this a private? (y/[n]) " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then
PRIVATE=true
fi
And I want to pass in the value of PRIVATE to item. I have tried every which way but no luck. Can anyone shed some light?
You can do it this way:
curl -i -X POST \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "Player.Open", "params":{"item":'"$PRIVATE"'}}' \
http://example.com/jsonrpc
Instead of your existing -d ... line above, you could try the following:
-d "{\"jsonrpc\": \"2.0\", \"method\": \"Player.Open\", \"params\":{\"item\":$PRIVATE}}" \
That is: when using double quote speechmarks ("), bash substitutes values for variables referenced $LIKE_THIS (not the case for single quotes you were using). The downside is that you then need to escape any double-quotes in the string itself (using the backslash, as above).
This abomination works too.
$ npm run script -- madrid
# script
json='{"city":"'"$1"'"}'
curl -X POST -d $json http://localhost:5678/api/v1/weather
I'm now testing some restful APIs using bash shell script.
I want to read url from file then make a json data string with the url in file.
For the test, below codes work fine. It's not reading from file.
#!/bin/bash
URL=http://test.com/test.jpg
curl -X POST \
-H "Content-Type:application/json" \
-H "accept:application/json" \
--data '{"url":"'"$URL"'"}' \
http://api.test.com/test
But, it returns some error when I'm using the codes like below.
#!/bin/bash
FILE=./url.txt
cat $FILE | while read line; do
echo $line # or whaterver you want to do with the $line variable
curl -X POST \
-H "Content-Type:application/json" \
-H "accept:application/json" \
--data '{"url":"'"$line"'"}' \
http://api.test.com/test
done
But, it returns error when I use the string from reading file.
This is error message.
Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value
at [Source: org.apache.catalina.connector.CoyoteInputStream#27eb679c; line: 1, column: 237]
How to solve this issue?
Why it returns error when I use the string from file reading?
It appears that your file is in dos format with \n\r line terminators. Try running dos2unix on it to strip the \rs. Additionally, no need to cat the file, use redirection, like so
while read -r line; do
echo $line # or whaterver you want to do with the $line variable
curl -X POST \
-H "Content-Type:application/json" \
-H "accept:application/json" \
--data '{"url":"'"$line"'"}' \
http://api.test.com/test
done < "$FILE"
Also, pass -r to read to prevent backslash escaping