I have a shell variable which contains json as follows
{
"test" :"test",
"temp":"temp"
}
This is a part of json that I got when parsed from jq, which is a json parser tool.But when I pass this to curl as a part of post request body, it is getting converted to
'{'
'"test"' ':' 'test,'
'"temp"' ':' 'temp'
'}'
But I want it as
'{
"test" : "test",
"temp" : "temp"
}'
VAL=$(echo "$RET" | jq ".pending[$j].value")
VAL is the variable that will contain the json
I need to pass this VAL as request body to curl
curl -X POST -H "$CONTENT_HEADER" -H "$AUTH_HEADER" http://localhost:8080/testApi -d $VAL
The shell is interpreting it as several arguments. Either quote the expansion (as in -d "$VAL") or pipe it instead of saving it to a variable jq '...' | curl ...
Related
how to fix error in escape sequence which has backslash it self as a special character.
error in parsing escape sequence characters through bash script.
we have below json data in a file test.json
"specialties": {
"type": "array",
"description": "list of the provider's specialties",
"items": {
"type": "string",
"description": "A specialty of the provider provider's",
"nullable": true,
"example": "[\"Psychiatry\"]"
}
}
we are trying parse this data into a string with below code.
data=$(cat test.json )
data=${data//\"/\\\"}
data=$(echo \"$data\" | tr -d ' ')
echo $data
above code output string
\"specialties\": { \"type\": \"array\", \"description\": \"list of the provider's specialties\", \"items\": { \"type\": \"string\", \"description\": \"A specialty of the provider provider's\", \"nullable\": true, \"example\": \"[\\"Psychiatry\\"]\" } }
this output string used with below curl request is throwing below error
curl -s -H "Accept: application/json" -H "Content-Type: application/json" --location --request POST "${HOST}/api/v1/projects" --header "Authorization: Bearer "$token"" -d '{"name":"'${PROJECT_NAME}'","openText": '${data}',"source": "API"}'
{"timestamp":"2022-12-10T14:00:32.384+0000","status":400,"error":"Bad Request","message":"JSON parse error: Unexpected character ('P' (code 80)): was expecting comma to separate Object entries; nested exception is com.fasterxml.jackson.core.JsonParseException: Unexpected character ('P' (code 80)): was expecting comma to separate Object entries\n at [Source: (PushbackInputStream); line: 1, column: 34273]","path":"/api/v1/projects"}
we use this (https://jsontostring.com/) site to convert above json data to string and it produces below output string which is working with our java app.
"\"specialties\":{\"type\":\"array\",\"description\":\"listoftheprovider'sspecialties\",\"items\":{\"type\":\"string\",\"description\":\"Aspecialtyoftheproviderprovider's\",\"nullable\":true,\"example\":\"[\\\"Psychiatry\\\"]\"}}"
difference between two output strings is one extra backslash\ character at the beginning and end of the word "Psychiatry"
#not working string part
\"[\\"Psychiatry\\"]\"
#working string part
\"[\\\"Psychiatry\\\"]\"
why is that extra backslash character is producing in this case and how to get that output.
after adding below code and using jq tool, it is fixed now. thanks fravadona.
data=$(cat test.json )
openText=$(echo $data | jq . -R | tr -d ' ')
echo $openText
#above code output string
"\"specialties\":{\"type\":\"array\",\"description\":\"listoftheprovider'sspecialties\",\"items\":{\"type\":\"string\",\"description\":\"Aspecialtyoftheproviderprovider's\",\"nullable\":true,\"example\":\"[\\\"Psychiatry\\\"]\"}}"
In terraform,I am trying to make a PUT call via curl command using null_resource and executing the command in local_exec provisioner. The body of the request is json array.
The command expects the data to be in string format. I have the data as tuple in locals. I am using jsonencode to serialize the tupple to a string and pass to curl. But on jsonencode, the string formed has an additional \ prefixed before each " in json string.
eg. expected json string is:
"[{"key-1" : "value-1", "key-2": "value-2"}]"
But after jsonencode, the formatted string is as follows:
"[{\"key-1\" : \"value-1\", \"key-2\": \"value-2\"}]"
additional backslash is added along with quotes.
Because of the malformed request body, the API returns Bad Request response.
How can we correctly serialize the tupple/list to correct json string in Terraform and pass to curl command?
This is my null_resource:
resource "null_resource" "call_API" {
for_each = { for x in local.array: x.name => {
name = x.name
vars = jsonencode(x.vars)
} }
provisioner "local-exec" {
command = <<EOF
curl -X PUT ${var.url} -d ${each.value.vars} -H 'Content-Type:application/json' -H 'Authorization:${local.bearer_token}'
EOF
}
I think I understand that you want to pass this variable
"[{"key-1" : "value-1", "key-2": "value-2"}]"
You tried to use EOF?
You can do something like that
variable "body" {
type = "string"
}
variable = <<EOF
"[{"key-1" : "value-1", "key-2": "value-2"}]"
EOF
Or with EOT, honestly I don't remember which one work
EDIT:
If the json is not static and you receive it from other module or others, you can try to expand the variable inside the EOFs
body = <<EOF
${var.json}
EOF
My Bash script has lots of variable assignments which are put in a JSON file:
var='Hello
"world".
This is \anything \or \nothing
'
echo "{ \"var\"= \"$var\" }" > output.json
When validating the file output.json jq says:
$ cat output.json | jq .
parse error: Invalid numeric literal at line 1, column 9
How can I make a valid JSON file out of it? Any \ and " should be preserved.
The correct JSON string would be
{ "var": "Hello
\"world\".
This is \\anything \\or \\nothing
" }
I cannot modify the variable assignments but the JSON creation.
Pass $var as a raw string argument and JQ will automatically convert it to a valid JSON string.
$ jq -n --arg var "$var" '{$var}'
{
"var": "Hello\n \"world\". \n This is \\anything \\or \\nothing\n"
}
"JSON strings can not contain line feeds", as oguz ismail already mentioned, so it's better to let dedicated tools like xidel (or jq) convert the line feeds to a proper escape sequence and to valid JSON.
Stdin:
$ var='Hello
"world".
This is \anything \or \nothing
'
$ xidel - -se '{"var":$raw}' <<< "$var"
{
"var": "Hello\n \"world\". \n This is \\anything \\or \\nothing\n"
}
Environment variable:
$ export var='Hello
"world".
This is \anything \or \nothing
'
$ xidel -se '{"var":environment-variable("var")}'
{
"var": "Hello\n \"world\". \n This is \\anything \\or \\nothing\n"
}
I want to use curl to PUT data. This works:
curl -X PUT --data '{ "xy": [0.6476, 0.2727] }' "http://"
I have a couple of bash variables that I'd like to use in place of the literals:
$key="xy"
$value="[0.6476, 0.2727]"
I've tried replacing the literals with quoted vars, but I get an error 'parameter not available'. I've tried combinations of nested and escaped quotes, but I am not getting anywhere.
This works for me with Node:
key="xy"
value="[0.6476, 0.2727]"
jsonData=$(node -e "console.log(JSON.stringify({$key: $value}))")
#jsonData output is: {"xy":[0.6476,0.2727]}
curl -X PUT --data $jsonData "http://"
But this only works for Node.js Developer or you have Node environment installed.
Well this works for me :
$ key='foo'
$ value='[ bar, bar ]'
$ echo "{ \"$key\": $value }"
{ "foo": [ bar, bar ] }
So this should work for you as well :
curl -X PUT --data "{ \"$key\": $value }" "http://google.com.uy"
Let me know if it helps!
Using jq (1.6 or later), use the --jsonargs option to take multiple JSON-encoded values as a JSON array accessible via $ARGS.positional:
$ key=xy
$ value=(0.6476 0.2727)
$ jq -n '{($k): $ARGS.positional}' --arg k "$key" --jsonargs "${value[#]}"
{
"xy": [
0.6476,
0.2727
]
}
--jsonargs must come last, so that all remaining arguments are treated as values in the desired array. We use --jsonargs rather than --args so that you get an array of floating-point values, rather than an array of strings.
Once you have jq working, you pipe its output to curl, which uses - as the argument to --data to read the data from the pipe.
jq -n '{($k): $ARGS.positional}' --arg k "$key" --jsonargs "${value[#]}" |
curl -X PUT --data - "http://"
I am trying to invoke a curl command in powershell and pass some JSON information.
Here is my command:
curl -X POST -u username:password -H "Content-Type: application/json" -d "{ "fields": { "project": { "key": "key" }, "summary": "summary", "description": "description - here", "type": { "name": "Task" }}}"
I was getting globbing errors and "unmatched braces" and host could not be resolved, etc.
Then I tried prefixing the double quotes in the string with the backtick character, but it could not recognize the - character in the description json field
thanks
EDIT 1:
When I wrote the curl command in a regular batch file, I used double quotes and no single quotes. Also, in the -d string, I escaped all the double quotes with \ and the command worked.
In this case, my curl is actually pointing to curl.exe. I specified the path, just didn't list it here. Also I tried adding single quotes around -d and I got:
curl: option -: is unknown curl: try 'curl --help' or 'curl --manual' for more information
Seems like it cannot recognize the - character in the JSON
Pipe the data into curl.exe, instead of trying to escape it.
$data = #{
fields = #{
project = #{
key = "key"
}
summary = "summary"
description = "description - here"
type = #{
name = "Task"
}
}
}
$data | ConvertTo-Json -Compress | curl.exe -X POST -u username:password -H "Content-Type: application/json" -d "#-"
curl.exe reads stdin if you use #- as your data parameter.
P.S.: I strongly suggest you use a proper data structure and ConvertTo-Json, as shown, instead of building the JSON string manually.
Easy way (for simple testing):
curl -X POST -H "Content-Type: application/json" -d '{ \"field\": \"value\"}'