How to insert a bash variable into a json string? - json

I have a bash script that does a post request to a GraphQL endpoint. The post body is JSON. Basically within that JSON, I would like to replace a value of one key/value pair with a value from a bash variable. I've tried different combinations but all of them resulted to errors. Here's my code:
#!/bin/bash
utc_format=$(date -v-2d -v-23H -u +"%Y-%m-%dT%H:%M:%SZ")
curl -v \
-X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Email: email#email.com" \
-H "X-Auth-key: $API_KEY" \
--data '{"query": "query { viewer { zones ( filter: { zoneTag_in: [ \"abcde12345\" ] } ) { firewallEventsAdaptive ( filter: { source: \"waf\" datetime_gt: "'"\"$utc_format\""'" ruleId: \"100000\" action: \"simulate\" } orderBy: [ datetime_DESC ] limit: 100 ) { clientIP edgeResponseStatus metadata { key value } } } } }"}' \
https://api.api.com/graphql \
| python -m json.tool >> curl_results
If you look at my code, the "' at the end of datetime_gt is to close the '{" in --data. Then before rule_id, I opened it up again with '". What am I doing wrong?? I probably checked this a hundred times. Every time I try a different formatting, a new error pops up. Currently (with the code pasted here), the error that I get is:
{
"data": null,
"errors": [
{
"extensions": {
"timestamp": "2020-12-21T03:32:11.090255834Z"
},
"message": "failed to recognize JSON request: 'invalid character '\"' after object key:value pair'",
"path": null
}
]
}

Check if it works. At least you gonna have a valid JSON as output.
#!/bin/bash
utc_format=$(date -v-2d -v-23H -u +"%Y-%m-%dT%H:%M:%SZ")
curl -v \
-X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Email: email#email.com" \
-H "X-Auth-key: $API_KEY" \
--data '{"query": "query { viewer { zones ( filter: { zoneTag_in: [ \"abcde12345\" ] } ) { firewallEventsAdaptive ( filter: { source: \"waf\" datetime_gt: \"'$utc_format'\" ruleId: \"100000\" action: \"simulate\" } orderBy: [ datetime_DESC ] limit: 100 ) { clientIP edgeResponseStatus metadata { key value } } } } }"}' \
https://api.api.com/graphql \
| python -m json.tool >> curl_results

Related

FIWARE Orion, NGSIv2 subscription in attributes with structured values

I'm working with Orion Contex Broker and I need to receive notifications when a parameter in a structured attribute changes its value. An example:
Subscription:
curl -iX POST \
--url 'http://localhost:1026/v2/subscriptions' \
--header 'content-type: application/json' \
--data '{
"description":"Notify me of Store changes in street Address",
"subject":{
"entities":[
{
"idPattern":".*",
"type":"Store"
}
],
"condition":{
"attrs":[
"address.streetAddress"
]
}
},
"notification":{
"http":{
"url":"http://localhost:3000/subscription/store-change"
}
}
}'
Create entity:
curl -iX POST \
--url 'http://localhost:1026/v2/op/update' \
-H 'Content-Type: application/json' \
-d '{
"actionType":"append",
"entities":[
{
"type":"Store",
"id":"urn:ngsi-ld:Store:001",
"address":{
"type":"PostalAddress",
"value":{
"streetAddress":"Old",
"addressRegion":"Berlin"
}
},
"name":{
"type":"Text",
"value":"Bösebrücke Einkauf"
}
}
]
}'
Update the entity:
curl -iX PATCH \
--url 'http://localhost:1026/v2/entities/urn:ngsi-ld:Store:001/attrs' \
-H 'Content-Type: application/json' \
-d '{
"address":{
"type":"PostalAddress",
"value":{
"streetAddress":"Bornholmer"
}
}
}'
The expected result would be to receive a notification when the entity was created and update. Another possibility could be the "condition expressions". However one of kind: "q": "address.streetAddress!=${previousValue}" is not implemented yet.
Attributes within NGSI are usually numbers or strings - this typically leads to a very flat data model. In this case when the attribute value changes the subscription would be fired.
JSON objects (such as address above) are also supported, but the change occurs whenever the Object's value change and is not specifically bound to a sub attribute Hence
"attrs":[
"address.streetAddress"
]
Would need to be:
"attrs":[
"address"
]
However, the q parameter could be used to filter against a specific sub-attribute e.g. q=address.streetAddress!="Old" - and the listening interface could amend the subscription after it has fired.

Uploading linked Revit models to Autodesk Forge

I've been trying to use the Post references (https://developer.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-references-POST/) to set up the reference between two files in forge but although I get a message "success" as result when I try it on the forge viewer I still see the files separately even after I translate the models. Has someone been through the same issue?
Without seeing you code it is hard to tell what is happening. Below I copied my bash script code which references/translate an obj with material and texture.
Au.obj
+- Au.mtl
+- Au.jpg
After upload, I got these
idObj="urn:adsk.objects:os.object:cyrillejcrja/Au.obj"
idMtl="urn:adsk.objects:os.object:cyrillejcrja/Au.mtl"
idJpg="urn:adsk.objects:os.object:cyrillejcrja/Au.jpg"
the code to set references, now
urn=$(xbase64encode $idObj)
job='{
"urn": "'${idObj}'",
"filename": "Au.obj",
"references": [{
"urn": "'${idMtl}'",
"relativePath": "./Au.mtl",
"filename": "Au.mtl",
"references": [{
"urn": "'${idJpg}'",
"relativePath": "./Au.jpg"
}]
}]
}'
response=$(curl -H "Content-Type: application/json" \
-H "Authorization: ${bearer}" \
-X POST ${ForgeHost}/modelderivative/v2/designdata/${urn}/references \
-k -s -d "${job}")
Here is got a reply like below which only means that the references are registered.
{
"result": "success"
}
Now, I do this to translate the obj and use the references
urn=$(xbase64encode $idObj)
job='{
"input": {
"urn": "'${urn}'",
"checkReferences": true
},
"output": {
"formats": [
{
"type": "svf",
"views": [
"2d",
"3d"
]
}
]
}
}'
response=$(curl -H "Content-Type: application/json" \
-H "Authorization: ${bearer}" \
-H "x-ads-force: true" \
-X POST ${ForgeHost}/modelderivative/v2/designdata/job \
-k -s -d "${job}")
Note the "checkReferences": true, as documented here.
Now, I can wait the translation to complete and see the result in the Viewer.
For reference the xbase64safeencode function used above
function xbase64safeencode () { local id64=$(echo -ne $1 | base64 $wrap_arg | tr -d '=' | tr '+/' '-_'); echo $id64; }
#Cyrille, this is my request:
curl -X 'POST' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsI' -H 'Content-Type: application/json' -v 'https://developer.api.autodesk.com/modelderivative/v2/designdata/{urn}/references' -d
'{
"urn": "urn:adsk.objects:os.object:bucket/non-existent.rvt",
"filename": "",
"references": [
{
"urn": "urn:adsk.objects:os.object:bucket/non-existent.rvt",
"relativePath": "",
"filename": ""
}
]
}'
I got as result:
'{
"result": "success"
}'
The point is I'm getting success as a result even when I do not have the specified file on the server, so I'd suggest few server-side validations, for example, when a model has been translated once we can't set as reference right so it should at least return an error. Thank you and I hope this helps.

Pass boolean value in a curl request [duplicate]

This question already has answers here:
Passing a shell variable to a JSON request to curl?
(3 answers)
Closed 5 years ago.
My goal is to have an attribute isAdmin in my users schema that is a boolean value, defaulted to false, and can be sent a true value in a CURL request.
I started off with this in my schema:
isAdmin: {
type: Boolean,
default: false,
required: true
}
and have a working CURL request and the isAdmin returns false as expected.
API="http://localhost:4741"
URL_PATH="/sign-up"
EMAIL="a2#admin.com"
PASSWORD="test"
curl "${API}${URL_PATH}" \
--include \
--request POST \
--header "Content-Type: application/json" \
--data '{
"credentials": {
"email": "'"${EMAIL}"'",
"password": "'"${PASSWORD}"'",
"password_confirmation": "'"${PASSWORD}"'"
}
}'
I then tried to take the default value out:
isAdmin: {
type: Boolean,
required: true
}
..and try passing T or F in the CURL request and it's not working. The error says that isAdmin is required. Not sure what is wrong with my CURL request.
API="http://localhost:4741"
URL_PATH="/sign-up"
EMAIL="a2#admin.com"
PASSWORD="test"
ADMIN="True"
curl "${API}${URL_PATH}" \
--include \
--request POST \
--header "Content-Type: application/json" \
--data '{
"credentials": {
"email": "'"${EMAIL}"'",
"isAdmin": "'"${ADMIN}"'",
"password": "'"${PASSWORD}"'",
"password_confirmation": "'"${PASSWORD}"'"
}
}'
Any ideas as to what I am doing wrong? And/Or how I can have a default value of false and POST a new user with a true isAdmin attribute? Thank you in advance for any advice.
Clarification: Using Mongoose and ExpressAPI.
Since you are enclosing the value in double quotes, it seems to be treated as a string, rather than boolean. Try this:
ADMIN=true # may be it is safer to use all lowercase for true/false
curl "${API}${URL_PATH}" \
--include \
--request POST \
--header "Content-Type: application/json" \
--data '{
"credentials": {
"email": "'"${EMAIL}"'",
"isAdmin": '${ADMIN}',
"password": "'"${PASSWORD}"'",
"password_confirmation": "'"${PASSWORD}"'"
}
}'

Using Global Instance of Keystone

Is it possible to use the Global Instance of Keystone to retrieve registered user profile info?
According to these references: https://github.com/telefonicaid/fiware-pep-steelskin#keystone and Keystone create user and permissions by api, it seems possible if I wish to install an instance by my own. However, what if I wish to use the Global Instance, instead. Is it possible?
For example, I have tested te retrieve some data as indicated below without success:
curl -s -H "X-Auth-Token:cXylpiNyh74V6J9YOlqN2GTzYSmGQa" http://cloud.lab.fiware.org:4730/v2.0/tokens | python -mjson.tool
curl -s -H "X-Auth-Token:cXylpiNyh74V6J9YOlqN2GTzYSmGQa" http://cloud.lab.fiware.org:4730/v3/users/ | python -mjson.tool
curl http://cloud.lab.fiware.org:4730/v3/auth/tokens -H "Content-Type: application/json" -d ' { "auth": { "identity": { "methods": [ "password" ], "password": { "user": { "domain": { "name": "matest" }, "name": "pep_proxy_99c59...", "password": "e3025a286dab..." } } } } }'
Note: I have tried both port: 5000 and 4730.
Any hint will be appreciated.
Users doesn't have permissions to see other users information or to create new users using the API.
However, you can issue tokens from the global keystone using both v2.0 and v3 protocols:
curl -X POST http://130.206.84.8:4730/v2.0/tokens \
-H 'Content-Type: application/json' \
-d "{\"auth\": {\"tenantName\": \"${OS_TENANT_NAME}\",
\"passwordCredentials\": {
\"username\": \"$OS_USERNAME\",
\"password\": \"$OS_PASSWORD\"}}}" | \
jq -r '.access.token.id'
Or issue a token in v3:
curl -v -H "Content-Type: application/json" -d "
{ \"auth\": {
\"identity\": {
\"methods\": [\"password\"],
\"password\": {
\"user\": {
\"name\": \"$OS_USERNAME\",
\"domain\": { \"id\": \"default\" },
\"password\": \"$OS_PASSWORD\"
}
}
}
}
}" http://cloud.lab.fiware.org:4730/v3/auth/tokens 2>&1 \
| grep -i "X-Subject-Token"
There are few things you can do with Keystone itself using these tokens if you are not the admin user (Non admin users obviously have few permissions). However, you coud, for instance query the endpoints:
curl -s -H "X-Auth-Token: $TOKEN_ID" http://130.206.84.8:4730/v3/endpoints
The domains:
curl -s -H "X-Auth-Token: $TOKEN_ID" http://130.206.84.8:4730/v3/domains

Create JSON Gist (conflict)

I want to create a Gist containing a JSON (valid one, I checked) with curl command as described here.
I tried first this script :
configText=$(cat jsonFile.json)
generate_post_data()
{
cat <<EOF
{
"description": "the description for this gist",
"public": true,
"files": {
"file1.txt": {
"content": $configText
}
}
}
EOF
}
curlBody="$(generate_post_data)"
curlResponse=$(curl -H "Content-Type: application/json" -X POST -d '$curlBody' https://api.github.com/gists)
Which gave me the error Problems parsing JSON, so I tried passing the file directly in the command:
curl -H "Content-Type:application/json" -data-binary #jsonFile.json https://api.github.com/gists
But I'm getting the same error. I know that this must be a conflict between the JSON body of the POST request and the JSON of my file (quotes, brackets...).
How can I send a clean JSON file to Gist ?
For the issues in your script :
in your curl request, you use single quotes around your bash variable in POST -d '$curlBody', use double quote to expand it : POST -d "$curlBody"
content is a text field : "content": $configText to "content": "$configText"
configText can have new lines and unescaped double quotes " which break your content JSON data. You could use the following to escape quotes and remove new lines :
configText=$(cat test.json | sed 's/\"/\\\"/g' | tr -d '\n')
The following example build your gist request with jq JSON parser/builder, not that this example will not preserve new lines in your input :
#!/bin/bash
ACCESS_TOKEN="YOUR_ACCESSS_TOKEN"
description="the description for this gist"
filename="file1.txt"
curlBody=$(jq --arg desc "$description" --arg filename "$filename" '.|
{ "description": $desc,
"public": true,
"files": {
($filename) : {
"content": tostring
}
}
}' jsonFile.json)
curl -v -H "Content-Type: application/json" \
-H "Authorization: Token $ACCESS_TOKEN" \
-X POST -d "$curlBody" https://api.github.com/gists
The following will preserve new lines in your json input by replacing new lines with \\n :
#!/bin/bash
ACCESS_TOKEN="YOUR_ACCESSS_TOKEN"
description="the description for this gist. There are also some quotes 'here' and \"here\" in that description"
public="true"
filename="file1.txt"
desc=$(echo "$description" | sed 's/"/\\"/g' | sed ':a;N;$!ba;s/\n/\\n/g')
json=$(cat test.json | sed 's/"/\\"/g' | sed ':a;N;$!ba;s/\n/\\n/g')
curl -v -H "Content-Type: text/json; charset=utf-8" \
-H "Authorization: Token $ACCESS_TOKEN" \
-X POST https://api.github.com/gists -d #- << EOF
{
"description": "$desc",
"public": "$public",
"files": {
"$filename" : {
"content": "$json"
}
}
}
EOF
Note that your access token must have the gist scope