When publishing avro payload to kafka-rest service, if one of the the field in value_schema defined with a default value, and in records omit this field. Seems kafka-rest still insist to have it inside record. What is the right way to do so?
eg.
curl -X POST -H "Content-Type: application/vnd.kafka.avro.v1+json" \
--data '{"value_schema": "{\"type\": \"record\", \"name\": \"User\", \"fields\": [{\"name\": \"name\", \"type\": \"string\"}, {\"name\":\"age\",\"type\":\"int\",\"default\":18}, {\"name\":\"id\",\"type\":\"string\"}]}", "records": [{"value": {"name": "testUser", "id": "001"}}]}' \
"http://localhost:8082/topics/avrotest"
Output
{"error_code":42203,"message":"Conversion of JSON to Avro failed: Failed to convert JSON to Avro: Expected field name not found: age"}
Related
How to update a field with a new value in a dynamic variable using bash?
I am storing some curl output in a variable below is the output response in it.
data=$(curl -s --location --request GET "${HOST}/api/v1/envs/projects/${PROJECT_ID}?page=0&pageSize=25" --header "Accept: application/json" --header "Content-Type: application/json" --header "Authorization: Bearer "$token"" | jq -r '.data[]')
echo "$data"
{
"id": "8aa0809085f46e000185f4c28b7a0b52",
"createdBy": "8aa0808e82fd01fb0182fd0b05820007",
"createdDate": "2023-01-27T19:44:28.538+0000",
"modifiedBy": "8aa0808e82fd01fb0182fd0b05820007",
"modifiedDate": "2023-01-27T19:45:14.811+0000",
"version": null,
"inactive": false,
"projectId": "8aa0809085f46e000185f4c28b760b51",
"name": "Master",
"refId": "Master",
"description": null,
"baseUrl": "http://petstore.swagger.io/v1",
}
Now I want to update the baseUrl field in that data variable dynamically and store that update complete JSON object either in same or different variable to be able use it in a different curl request operation.
How can we do that in bash scripting?
You are already using jq to extract part of the data returned from the API. You can extend this jq command to modify the baseURL field on the fly. The following will update baseURL in the first object in the data array, then print it out:
jq -r '.data[0].baseUrl = "http://example.com" | .data[0]'
This question already has answers here:
Curl command issue in bash script because of string variable containing quotes [duplicate]
(2 answers)
Closed 3 years ago.
I am trying to insert a variable into my json payload (working in shell script) but I am unsure of how to escape the characters properly
I've tried many different escape methods but I'm a total noob at it and I either return the literal string or it doesn't run
SLACK_ALERT_WEBHOOK=desiredurl
curl -X POST -H 'Content-type: application/json' --data '{"text": "*Daily Webhook Verification*", "attachments": [
{
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "Slack post failed for webhook, please investigate: $SLACK_ALERT_WEBHOOK"
}
}
]
}
]}' "$SLACK_ALERT_WEBHOOK"
I just want to insert the value of SLACK_ALERT_WEBHOOK into this portion of the code "text": "Slack post failed for webhook, please investigate: $SLACK_ALERT_WEBHOOK but it either doesn't run or returns the literal string. I have the bottom "$SLACK_ALERT_WEBHOOK" working successfully at the bottom to send to my desired slack channel so I'm not worried about that.
I've got it working thanks to tripleee:
curl -X POST -H 'Content-type: application/json' --data "{\"text\": \"*Verification*\", \"attachments\": [{\"blocks\": [{\"type\": \"section\",\"text\": {\"type\": \"mrkdwn\",\"text\": \"$SLACK_ALERT_WEBHOOK\"}}]}]}" $SLACK_ALERT_WEBHOOK
Try this:
SLACK_ALERT_WEBHOOK=desiredurl
$(curl -X POST -H 'Content-type: application/json' --data '{"text": "*Daily Webhook Verification*", "attachments": [
{
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "Slack post failed for webhook, please investigate: `echo $SLACK_ALERT_WEBHOOK`"
}
}
]
}
]}' "`echo $SLACK_ALERT_WEBHOOK`")
Using the echo <<variable>> it should just run that as a command. I tested it on my own server, but might work differently on yours. Hope it helps.
In the Watson API documentation, the following is given for adding query to training data using CURL:
curl -X POST -u "{username}":"{password}" -H "Content-Type: application/json" -d
'{
"natural_language_query": "who is keyser soze",
"filter": "text:criminology",
"examples": [
{
"document_id": "adaf50f1-2526-4fad-b670-7d6e8a42e6e6",
"relevance": 2
},
{
"document_id": "63919442-7d5b-4cae-ab7e-56f58b1390fe",
"cross_reference": "my_id_field:14",
"relevance": 4
}
]
}' "https://gateway.watsonplatform.net/discovery/api/v1/environments/{environment_id}/collections/{collection_id}/training_data?version=2018-03-05"
However much I try or use multiple libraries, it does not seem to have the correct JSON format (even when JSON validators have confirmed).
Has anyone tried using this method to add queries?
Im trying to index a simple json data in solr using curl. When i use the command, it says
"curl -X POST -H 'Content-Type:application/json'-d http://localhost:8983/solr/informationretrieval/update/json/docs '[{"id":"1","title":"Doc 1"},{"id":"2","title":"Doc 2"}]'
{"responseHeader":{"status":0,"QTime":1}}
curl: (3) [globbing] bad range specification in column 3"
I have tried removing quotes, tried -g and --globoff etc but every time there are errors like illegal port number. Can anyone please help?
Adding Multiple JSON Documents
curl -X POST -H 'Content-Type: application/json' 'http://localhost:8983/solr/informationretrieval/update' --data-binary '
[
{
"id": "1",
"title": "Doc 1"
},
{
"id": "2",
"title": "Doc 2"
}
]'
I am running the following command in Windows prompt:
curl -XPUT http://127.0.0.1:9200/test-index/test-type/_mapping?pretty=true -d '{"test-type": {"properties": {"name": {"index": "analyzed", "term_vector": "with_positions_offsets", "boost": 1.0, "store": "yes", "type": "string"}}}}'
I get the following error:
{
"error" : "ElasticsearchParseException[Failed to parse content to map]; nested: JsonParseException[Unexpected character (''' (code 39)): expected a
valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.elasticsearch.common.compress.lzf.LZFCompressedStreamInput#45
4ed1d2; line: 1, column: 2]]; ",
"status" : 400
}
I searched for solutions and found alternatives such as put json data in files, but I cannot use it for some reasons.
Thanks!
Windows's cmd doesn't support strings with single quotes. Use " and escape the inner ones with \".
"I searched for solutions and found alternatives such as put json data in files, but I cannot use it for some reasons"
This should work, with hello.json in temp. The # is requried.
c:\temp>curl -v -X PUT \
--data "#hello.json" \
-H "Content-Type:application/json" \
http://localhost:8080/api/myresource