I am running the following command in Windows prompt:
curl -XPUT http://127.0.0.1:9200/test-index/test-type/_mapping?pretty=true -d '{"test-type": {"properties": {"name": {"index": "analyzed", "term_vector": "with_positions_offsets", "boost": 1.0, "store": "yes", "type": "string"}}}}'
I get the following error:
{
"error" : "ElasticsearchParseException[Failed to parse content to map]; nested: JsonParseException[Unexpected character (''' (code 39)): expected a
valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.elasticsearch.common.compress.lzf.LZFCompressedStreamInput#45
4ed1d2; line: 1, column: 2]]; ",
"status" : 400
}
I searched for solutions and found alternatives such as put json data in files, but I cannot use it for some reasons.
Thanks!
Windows's cmd doesn't support strings with single quotes. Use " and escape the inner ones with \".
"I searched for solutions and found alternatives such as put json data in files, but I cannot use it for some reasons"
This should work, with hello.json in temp. The # is requried.
c:\temp>curl -v -X PUT \
--data "#hello.json" \
-H "Content-Type:application/json" \
http://localhost:8080/api/myresource
Related
So I'm writing a script that needs to create a json object, and post it with curl.
This is working:
curl --header "Content-Type: application/json" --request POST --data '{ "_type": "_test", "_device": "123.123.123.123", "_system": "web-services", "result": "success", "_time": "123", "error": "" }' $data_pipeline
$data_pipeline contains the URL for the post request
if $json_string contains the string including single quotes:
'{ "_type": "_test", "_device": "123.123.123.123", "_system": "web-services", "result": "success", "_time": "123", "error": "" }'
This should work but it doesn't:
curl --header "Content-Type: application/json" --request POST --data $json_string $data_pipeline
First I was creating the $json_object without the single quotes, and tried to add them on the command line for CURL. If I don't escape the single quotes, $json_string is sent as a literal, instead of expanding the variable. I escaped the single quotes, and it did not help, I even tried a double escape in case that was needed, and still it is not working. It only works if I put the entire json string by hand, but not if I put it in a variable. How can I fix this? The json is dynamically created by the script, using jq, and the json is valid as I can successfully run the post by hand, I just need it to work with a variable. It won't work without the single quotes, but the single quotes don't work when I use a variable to hold the json, it doesn't matter if I put the single quotes in the variable, or try to do it outside of the variable... How do I fix this?
The json object is built with this code:
json_string=\'$( jq -n \
--arg _type "$_type" \
--arg _device "$_device" \
--arg _system "$_system" \
--arg result "$result" \
--arg _time "$_time" \
--arg error "$error" \
'{_type: $_type, _device: $_device, _system: $_system, result: $result, _time: $_time, error: $error}' )\'
Originally I was creating the json_string without the ' but I added that in attempt to get the single quotes wrapped around the json.
Thanks!
#!/bin/bash
_type="hello"
# let's have some fun
_device="single'quote"
_system='double"quote'
error='multi
line'
encode()
{
printf "%s" "$1" | sed 's/"/%22/' | tr '\n' ' '
}
# you don't need encode() if your strings are not json-offensive
json_string="{
_type: \"$_type\"
_device: \"$_device\"
_system: \"$(encode "$_system")\"
error: \"$(encode "$error")\"
}"
set -x
curl \
-X POST \
-d "$json_string" \
-H 'content-type: application/json' \
http://httpbin.org/anything
Output:
+ curl -X POST -d '{
_type: "hello"
_device: "single'\''quote"
_system: "double%22quote"
error: "multi line"
}' -H 'content-type: application/json' http://httpbin.org/anything
{
"args": {},
"data": "{\n\t_type: \"hello\"\n\t_device: \"single'quote\"\n\t_system: \"double%22quote\"\n\terror: \"multi line\"\n}",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Content-Length": "92",
"Content-Type": "application/json",
"Host": "httpbin.org",
"User-Agent": "curl/7.58.0",
},
"json": null,
"method": "POST",
"url": "http://httpbin.org/anything"
}
I need to concatenate 2 variables, but when i execute the bash code i get this error filters.api.malformed_request_body.
I need to get the current IPs, to be able to add a new IP to the filter, that's because i need to use two variables, $a is the current IP that is in my firewall rule, and $b is the new IP that i will add.
From Cloudflare
To preserve existing values, issue a GET request and based on the response, determine which fields (and respective values) to include in your PUT request and that way, avoid any undesired overwrites.
Code:
a=122.16.89.10
b=137.77.77.77
curl -X PUT \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: KEY" \
-H "Content-Type: application/json" \
-d '[
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {'$a'" "'$b'})",
"description": "Block IP"
}
]' "https://api.cloudflare.com/client/v4/zones/ZONE/filters"
I also tried: "(ip.src in {'$a $b'})" and:
new_filter="$a $b"
...
...
"(ip.src in {'$new_filter'})"
If i echo $new_filter it shows the correct result:
new_filter="$a $b"
echo $new_filter
#122.16.89.10 137.77.77.77
When i use the variable $new_filter it also show this error curl: (3) [globbing] unmatched close brace/bracket in column 13 line 13 is this one -H "Content-Type: application/json" \.
None worked, why? I get this error:
{
"result": null,
"success": false,
"errors": [
{
"code": 10014,
"message": "filters.api.malformed_request_body"
}
],
"messages": []
}
This works: "(ip.src in {'$a'})".
Well, let's take your first example and modify it to print the JSON body:
export a=122.16.89.10
export b=137.77.77.77
echo '[
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {'$a'" "'$b'})",
"description": "Block IP"
}
]'
The output is this:
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {122.16.89.10" "137.77.77.77})",
"description": "Block IP"
}
]
You can see that JSON is not valid. The double quotes are unbalanced in expression.
Try "expression": "(ip.src in {'$a' '$b'})", instead -- that will produce valid JSON.
This is a small command line script that is used to post a json body in an http server. I am finding difficulties to pass the first command line argument $1 to the json body.
#!/bin/bash
curl -X POST -d '{ "game": 16, "id": $(($1)) }' http://localhost:10000/
The command does not fail, however the http body contains exactly
{ "game": 16, "id": $(($1)) }
I want to run the script ./script 123 and send the json body
{ "game": 16, "id": 3 }
How can I do this using bash?
You can also use single quotes so you don't have to escape double quotes like this:
#!/bin/bash
curl -X POST -d '{ "game": 16, "id": '$1' }' http://localhost:10000/
Using single quotes will print literal characters. You need to use double quotes for string interpolation. Try:
curl -X POST -d "{ \"game\": 16, \"id\": $1 }" http://localhost:10000/
I'm trying to load simple JSON data into BigQuery table the following way:
$ bq load \
--apilog \
--source_format=NEWLINE_DELIMITED_JSON \
my_dataset.my_table \
./input.json ./schema.json
but get the following error message:
Upload complete.
Waiting on bqjob_xxxx_xxx ... (3s) Current status: DONE
BigQuery error in load operation: Error processing job 'my_project_id:bqjob_xxxx_xxx': CSV table encountered too many errors, giving up. Rows: 1; errors: 1.
Failure details:
- file-00000000: Error detected while parsing row starting at
position: 0. Error: Data between close double quote (") and field
separator.
It complains about some CSV error, but I'm trying to load JSON (--source_format=NEWLINE_DELIMITED_JSON)
My input.json contains this data:
{"domain":"stackoverflow.com","key":"hello","value":"world"}
My schema.json is the following:
[
{
"name": "domain",
"type": "string",
"mode": "nullable"
},
{
"name": "key",
"type": "string",
"mode": "nullable"
},
{
"name": "value",
"type": "string",
"mode": "nullable"
}
]
bq version 2.0.25:
$ gcloud version | grep ^bq
bq 2.0.25
The problem here is that the flag apilog expects a string as input. This command should work for you:
bq load \
--apilog '' \
--source_format=NEWLINE_DELIMITED_JSON \
my_dataset.my_table \
./input.json ./schema.json
Empty string sends output to stdout. If you want to save the log to a local file then you can just send a non-empty string, such as --apilog 'localfile_name'.
BQ command says:
USAGE: bq.py [--global_flags] <command> [--command_flags] [args]
As you see there are global_flags and command_flags
For the global_flags that have values you need to use the equal sign:
--flag=value
The command_flags are either boolean:
--[no]replace
Or they take arguments that must follow the flag:
--source_format NEWLINE_DELIMITED_JSON
Also do not mix global and command flags: apilog is a global flag.
I would rewrite your command into:
$ bq --apilog load \
--source_format NEWLINE_DELIMITED_JSON \
my_dataset.my_table \
./input.json ./schema.json
When publishing avro payload to kafka-rest service, if one of the the field in value_schema defined with a default value, and in records omit this field. Seems kafka-rest still insist to have it inside record. What is the right way to do so?
eg.
curl -X POST -H "Content-Type: application/vnd.kafka.avro.v1+json" \
--data '{"value_schema": "{\"type\": \"record\", \"name\": \"User\", \"fields\": [{\"name\": \"name\", \"type\": \"string\"}, {\"name\":\"age\",\"type\":\"int\",\"default\":18}, {\"name\":\"id\",\"type\":\"string\"}]}", "records": [{"value": {"name": "testUser", "id": "001"}}]}' \
"http://localhost:8082/topics/avrotest"
Output
{"error_code":42203,"message":"Conversion of JSON to Avro failed: Failed to convert JSON to Avro: Expected field name not found: age"}