Cocatenate 2 variables to put inside json is not working - json

I need to concatenate 2 variables, but when i execute the bash code i get this error filters.api.malformed_request_body.
I need to get the current IPs, to be able to add a new IP to the filter, that's because i need to use two variables, $a is the current IP that is in my firewall rule, and $b is the new IP that i will add.
From Cloudflare
To preserve existing values, issue a GET request and based on the response, determine which fields (and respective values) to include in your PUT request and that way, avoid any undesired overwrites.
Code:
a=122.16.89.10
b=137.77.77.77
curl -X PUT \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: KEY" \
-H "Content-Type: application/json" \
-d '[
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {'$a'" "'$b'})",
"description": "Block IP"
}
]' "https://api.cloudflare.com/client/v4/zones/ZONE/filters"
I also tried: "(ip.src in {'$a $b'})" and:
new_filter="$a $b"
...
...
"(ip.src in {'$new_filter'})"
If i echo $new_filter it shows the correct result:
new_filter="$a $b"
echo $new_filter
#122.16.89.10 137.77.77.77
When i use the variable $new_filter it also show this error curl: (3) [globbing] unmatched close brace/bracket in column 13 line 13 is this one -H "Content-Type: application/json" \.
None worked, why? I get this error:
{
"result": null,
"success": false,
"errors": [
{
"code": 10014,
"message": "filters.api.malformed_request_body"
}
],
"messages": []
}
This works: "(ip.src in {'$a'})".

Well, let's take your first example and modify it to print the JSON body:
export a=122.16.89.10
export b=137.77.77.77
echo '[
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {'$a'" "'$b'})",
"description": "Block IP"
}
]'
The output is this:
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {122.16.89.10" "137.77.77.77})",
"description": "Block IP"
}
]
You can see that JSON is not valid. The double quotes are unbalanced in expression.
Try "expression": "(ip.src in {'$a' '$b'})", instead -- that will produce valid JSON.

Related

curl custom slack notifications from jenkins

I am trying to send an custom slack notification to my slack channel using curl. Below my payload . Using curl to post notifications to Slack channel so that Team members could be able to see the Verison, s3 link and Directly they can access directly. Any suggestions or inputs would be greatly appreciated.
version=1.2.4
bundleversion=1.3.4.5
SLACK_MSG="Version=$version bundleversion=$bundleversion s3link:Random "
curl -H "Content-type: application/json" -X POST --data-urlencode -d
"payload='{
"username": "Kalyan",
"attachments": [
{
"color": "danger",
"fields": [
{
"title": "Danger Event",
"text": "$SLACK_MSG",
"short": false
}
]
},
{
"color": "warning",
"fields": [
{
"title": "Warning Event",
"value": "This is a warning",
"short": false
}
]
},
{
"color": "good",
"fields": [
{
"title": "Good Event",
"value": "This is good",
"short": false
}
]
}
]
}'" https://hooks.slack.com/services/XXXXXX/XXXXXXX/XXXXXXXXXX
Below Jenkins Error
curl: (3) [globbing] nested brace in column 51
curl: (3) Illegal characters found in URL
curl: (6) Could not resolve host: bundleversion=1.3.4.5
curl: (3) Port number ended with 'R'
curl: (3) [globbing] unmatched close brace/bracket in column 52
curl: (3) Illegal characters found in URL
curl: (6) Could not resolve host: is
curl: (6) Could not resolve host: a
curl: (3) Illegal characters found in URL
curl: (6) Could not resolve host: is
curl: (3) [globbing] unmatched close brace/bracket in column 56
Any inputs Greatly Appreciated.
Your first danger attachment has a field with an invalid property called text specified - change this property to value to make it into a valid Slack field.
You can troubleshoot Slack message payloads via the Slack Message Formatting page to see if they are valid: here's the corrected message payload.
I will also make it simple:
Please note: There should not be any space after "content-type:" and "application/json" people do that mistake.
and take care about backslash. You can also try this on terminal/cmd. I should work
curl -X POST -H "Content-type:application/json" --data '{\"text\":\"here_is_your_message\"}' YOUR_WEBHOOK_URL
try this: payload and curl for custom slack notification.
I used that in Gitlab-CI.yml file for sending Job artifacts and reports to Slack. It works fine.
The code below you can modify because I wrote that like it appears Green/Red when job Pass or Fail.
- 'curl -H "Content-Type:application/json" -X POST --data "{
\"attachments\": [
{
\"mrkdwn_in\": [\"text\"],
\"color\": \"#36a64f\",
\"author_name\": \"<https://$Gitlab_Home_URL/${GITLAB_USER_LOGIN}|${GITLAB_USER_NAME}>($GITLAB_USER_LOGIN)\",
\"text\": \"*Job <https://$REPO_URL/-/jobs/${CI_JOB_ID}|TESTING> was SUCCESSFUL in pipeline <https://$REPO_URL/pipelines/${CI_PIPELINE_ID}|${CI_PIPELINE_ID}>*\",
\"fields\": [
{
\"title\": \"Trigger source\",
\"value\": \"$CI_PIPELINE_SOURCE\",
\"short\": true
},
{
\"title\": \"Branch\",
\"value\": \"<https://$REPO_URL/tree/$CI_COMMIT_REF_NAME|$CI_COMMIT_REF_NAME>\",
\"short\": true
},
{
\"title\": \"Commit message\",
\"value\": \"<https://$REPO_URL/commit/${CI_COMMIT_SHA}|$CI_COMMIT_TITLE>\",
\"short\": true
}
],
\"footer\": \"<https://$REPO_URL|$CI_PROJECT_NAME>\",
\"footer_icon\": \"https://www.stickpng.com/assets/images/5847f997cef1014c0b5e48c1.png\",
}
]
}" YOUR_SLACK_WEBHOOK'

Send data with curl in Bash script: positional argument not working

Total noob in BASH. Trying to learn.
I have the following bash script to make an API request:
#!/bin/bash
if [ $1 = "new_event" ]; then
a='https://www.googleapis.com/calendar/v3/calendars/'
b=$2
c='/events?access_token='
d=$3
path=$a$b$c$d
echo $4
OUTPUT="$(curl -s -H "Content-Type: application/json" $path -d $4 )"
echo "${OUTPUT}"
fi
Positional arguments are 'new_event', calendarId, access token and json string.
If I run the script I get:
first echo is the json string I pass as parameter in the call:
' {"guestsCanSeeOtherGuests": false, "location": "", "description": "TEST", "reminders": {"useDefault": false}, "start": {"dateTime": "2017-07-06T14:00:00", "timeZone": "America/Sao_Paulo"}, "end": {"dateTime": "2017-07-06T15:00:00", "timeZone": "America/Sao_Paulo"}, "guestsCanInviteOthers": false, "summary": "TEST", "status": "tentative", "attendees": []} '
second echo gives me parsing error.
BUT, if I copy the echoed json string and replace $4 for it, everything works.
OUTPUT="$(curl -s -H "Content-Type: application/json" $path -d ' {"guestsCanSeeOtherGuests": false, "location": "", "description": "TEST", "reminders": {"useDefault": false}, "start": {"dateTime": "2017-07-06T14:00:00", "timeZone": "America/Sao_Paulo"}, "end": {"dateTime": "2017-07-06T15:00:00", "timeZone": "America/Sao_Paulo"}, "guestsCanInviteOthers": false, "summary": "TEST", "status": "tentative", "attendees": []} ' )"
Any hint why is it not working with the positional argument while it works if I paste it's content?
Thanks!
When you pass the JSON string as a parameter, it assigns the content contained inside the single quote as a whole string but when you pass it to curl it was subjected to word splitting.
To show what it looks like here's a sample script to demonstrate it.
This script will receive the string and pass it to the second script.
#!/bin/bash
./params $1
This second script simulates what curl sees. It will print the number of parameters it receives.
#!/bin/bash
echo $#
Guess what the output is:
27
To fix your issue and make it simpler, drop the outermost quote and quote everything inside the $().
OUTPUT=$(curl -s -H "Content-Type: application/json" "$path" -d "$4" )

curl: (3) [globbing] bad range specification in column 3

Im trying to index a simple json data in solr using curl. When i use the command, it says
"curl -X POST -H 'Content-Type:application/json'-d http://localhost:8983/solr/informationretrieval/update/json/docs '[{"id":"1","title":"Doc 1"},{"id":"2","title":"Doc 2"}]'
{"responseHeader":{"status":0,"QTime":1}}
curl: (3) [globbing] bad range specification in column 3"
I have tried removing quotes, tried -g and --globoff etc but every time there are errors like illegal port number. Can anyone please help?
Adding Multiple JSON Documents
curl -X POST -H 'Content-Type: application/json' 'http://localhost:8983/solr/informationretrieval/update' --data-binary '
[
{
"id": "1",
"title": "Doc 1"
},
{
"id": "2",
"title": "Doc 2"
}
]'

Is there any solution to do the "NOT IN" functionality in Elasticsearch?

Here is a simple but difficult question.
I want to do an aggregation for a query results that should be use "NOT IN" functionality like any RDBMS' SQL.
For example, I want to do a job something like below.
curl -XGET http://localhost:9200/my_index/my_type/_search?pretty -d '{
"query": {
"filtered": {
"filter": {
!!! Documents whose 'user_id' field value is 'NOT IN' distinct user_ids where the 'action' field value is 'signup' !!!
}
}
},
"aggregations": {
"distinct_users":{
"cardinality": {
"field": "user_id",
"precision_threshold": 1000000
}
}
}
}'
Edit
Here is an example data.
curl -s -XPOST 'localhost:9200/my_index/my_type/1' -d'{ "user_id": 1234, "action": "signup" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/2' -d'{ "user_id": 1234, "action": "visit" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/3' -d'{ "user_id": 1234, "action": "visit" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/4' -d'{ "user_id": 5678, "action": "visit" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/5' -d'{ "user_id": 5678, "action": "visit" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/6' -d'{ "user_id": 9012, "action": "signup" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/7' -d'{ "user_id": 9012, "action": "visit" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/8' -d'{ "user_id": 9012, "action": "visit" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/9' -d'{ "user_id": 3456, "action": "visit" }'
curl -s -XPOST 'localhost:9200/my_index/my_type/10' -d'{ "user_id": 3456, "action": "visit" }'
What I really want to get is the "Documents whose user_id DOES NOT signed up based on these log data".
So, documents [4, 5, 9, 10] are the final results what I want to get.
Is it possible to get the results what I want in Elasticsearch?
Thanks in advance.
No, elasticsearch does not do joins and what you are asking for is a variation of a join.
As pointed out above, it does have parent child relations and nested objects though which might help you somewhat, depending on your domain.
Elasticsearch also does not have a distinct feature which you need as well. But you can sort of fake it with a terms aggregation.
However, that does not help you here since you really need a join for this. So, the only solution here is to do the join outside of elasticsearch. Depending on your data size, that might be expensive. Also see application side joins.
if you use the not filter Elasticsearch will check each document in turn - the following will return all documents that have an action where action is anything but signup.
curl -XGET http://localhost:9200/myindex/my_type/_search?pretty -d '{
"query": {
"filtered": {
"filter": {
"not" : {
"term" : { "action" : "signup" }
}
}
}
}
}'
To meet the requirement where a userid is returned if it doesn't have any instances of action=signup then you'll need to set up a parent child relationship.
in this case, Userid would be unique across all user type documents. Each user document would have one or more action type children.
The following query checks the action child and returns the user document.
curl -XGET 'http://localhost:9200/myindex/my_type/_search?pretty' -d '{
"query": {
"filtered": {
"filter": {
"not" : {
"has_child": { "type": "my_action", "term" : { "action" : "signup" }}
}
}
}
}
}'

cURL Response JSON RestAPI

I need to delete some wrong data, inserted in a lot of processes, and I need to figure if this is possible with cURL and rest API, with a script in sh, batch or something like this:
curl -u admin:admin -i -H "Accept: application/json" -X GET "http://json_bpm.com/wle/v1/service/6112449?action=getData&fields=context"
First I just need the result map.
Output:
{"status":"200","data":{"result":"{\"context\":{\"name\":\"xxx\" (...)
"resultMap":{"context":{"name\":"xxx\" (...) }}}
Because I need to remove the userDelete array (see below) for thousands of processes, and set this again using curl. If you know how to remove arrays from JSON too, you're the man. :)
{
"context": {
"name": "Change Process",
"startUser": {
"user": "0001"
},
"endUser": {
"user": "0001"
},
"userDelete": {
"user": "0002"
},
"origin": "GUI",
"userAction": "Change Process"
}
}