How to pass json parameters for Jenkins using CURL? - json

I'm trying to build a job by passing JSON parameter for Jenkins through Linux CLI. But I'm not able to pass the parameters in JSON. I have used -g to turn off globbing. Still the issue persists. Any help would be appreciated.
curl -X POST -u username:password -g JENKINS_URL/job/Bulk_Job/buildWithParameters?serviceBranchJson="{"key1":"value1","key2":"value2"}"
Error message
Processing provided DSL script
{key1:value1,key2:value2}
groovy.json.JsonException: expecting '}' or ',' but got current char 'k' with an int value of 107
The current character read is 'k' with an int value of 107
expecting '}' or ',' but got current char 'k' with an int value of 107
line number 1
index number 1
{key1:value1,key2:value2}
Code snippet
import groovy.json.JsonSlurper
def serviceBranchJson = serviceBranchJson
println(serviceBranchJson)
Map servicesMap = new JsonSlurper().parseText(serviceBranchJson)

The problem is that your JSON is missing quotes around the keys and values. From your error message, the json is {key1:value1,key2:value2} and it should be: {"key1":"value1","key2":"value2"} to work properly.
Try to escape the quotes in your curl command, for example:
curl -u user:token "JENKINS_URL/job/Bulk_Job/buildWithParameters?serviceBranchJson={\"key1\":\"value1\",\"key2\":\"value2\"}"
Then it should work correctly with your pipeline code (I added println servicesMap to the end of the code):
Started by remote host ...
[Pipeline] Start of Pipeline
[Pipeline] echo
{"key1":"value1","key2":"value2"}
[Pipeline] echo
{key1=value1, key2=value2}
[Pipeline] End of Pipeline
Finished: SUCCESS

Related

Zabbix - triggering on text, displaying only part of the text

I'm monitoring a web page that displays the status of a few hundred items. The page looks like this:
{"arrisId":"a000098","status":"Running","startTime":"2018-05-10T08:02:19.563Z"},{"arrisId":"a000101","status":"Running","startTime":"2018-05-10T08:02:19.892Z"},{"arrisId":"a000107","status":"Running","startTime":"2018-05-10T08:02:28.556Z"},...
What I want to do is trigger when 1 of the things is "Not Running", but I would like to display only the item that is not working and not the entire page. Hope that makes sense. I could use web.page.regexp and send a message that something is not running, but if I use web.page.get, is there a way to configure a trigger to display the not running and the 25 or so characters in front of that?
I hope this question makes sense.
Your best course of action is to use Low Level Discovery.
Your LLD rule will run a script to ingest your main status page, then parse it and use the fields to create your items according to the "Item prototypes" you define.
The item prototype themselves will need a script as well to get their respective information (unless you are willing to use the Zabbix in beta)
I've done a simple setup, using mock json from here:
LLD Script: will parse the mock json and convert it into a Zabbix LLD compliant format:
import requests
import json
jsonSource = "https://jsonplaceholder.typicode.com/users"
lld = {}
data = []
lld['data'] = data
session = requests.Session()
response = session.get(jsonSource)
for jsonObject in response.json():
data.append ( {
'{#NAME}': jsonObject['name'],
'{#ID}': jsonObject['id'],
'{#URL}': jsonSource + '/' + str(jsonObject['id'])
} )
print json.dumps(lld)
Item GET Script: get a specific field of a specific item (will become obsolete with http agent item from Zabbix 4.0):
import requests
import json
import sys, argparse
parser = argparse.ArgumentParser()
parser.add_argument('-i', required=True, metavar='User ID')
parser.add_argument('-f', required=True, metavar='\"Requested JSON Field\"')
args = parser.parse_args()
jsonSource = "https://jsonplaceholder.typicode.com/users/" + args.i
session = requests.Session()
response = session.get(jsonSource)
print (response.json()[args.f])
Command line usage sample:
$ jsonLLD.py
{"data": [{"{#ID}": 1, "{#URL}": "https://jsonplaceholder.typicode.com/users/1", "{#NAME}": "Leanne Graham"}, {"{#ID}": 2, "{#URL}": "https://jsonplaceholder.typicode.com/users/2", "{#NAME}": "Ervin Howell"},
[cut]
$ jsonGet.py -i 10 -f phone
024-648-3804
$ jsonGet.py -i 10 -f name
Clementina DuBuque
Then you have to set it up into Zabbix:
create a new template
create a Discovery rule of "Zabbix agent" type and set it to run system.run[/usr/bin/jsonLLD.py] (mind the path!)
create an item prototype for each json field you want to work on (ie: Item name: {#NAME} telephone number, Item key system.run[/usr/bin/jsonGet.py -i {#ID} -f phone] )
create trigger prototypes accordingly
associate an host to the template
In your situation I'd use the Zabbix server itself as host, and install the scripts in its /usr/bin.
Watch the Zabbix Agent's log to see the discovery and item gathering process:
1972:20180519:121849.052 Executing command '/usr/bin/jsonGet.py -i 1 -f phone'
1971:20180519:121850.054 Executing command '/usr/bin/jsonGet.py -i 2 -f phone'
1974:20180519:121851.055 Executing command '/usr/bin/jsonGet.py -i 3 -f phone'
1974:20180519:121852.073 Executing command '/usr/bin/jsonGet.py -i 4 -f phone'
1974:20180519:121853.076 Executing command '/usr/bin/jsonGet.py -i 5 -f phone'
1973:20180519:121854.077 Executing command '/usr/bin/jsonGet.py -i 6 -f phone'
1972:20180519:121855.079 Executing command '/usr/bin/jsonGet.py -i 7 -f phone'
[cut]

Error while Passing Json string in scala using Curl

I am trying to post a json string using curl in scala. My curl command works fine if executed from linux box but throes an error(("message": "Must provide query string.",) always from scala.
my working curl command in linux:
curl http://laptpad1811:5000/graphql -H "Content-Type: application/json"
-X POST -d '{"query":"mutation
CreateFileReceivedEvent($createFileReceivedEventInput:
CreateFleReceivedEventInput!) { createFileReceivedEvent(input:
$createFileReceivedEventInput) { clientMutationId }}","variables":
{"createFileReceivedEventInput":
{"clientMutationId":"Test","fileReceivedEvent":{"file":
{"fileTrackingId":"83a86c44-66a5-4de0-9b7f-
c6995877279d","name":"textfile_2017-08-21T15:58:45Z","fileType":
{"code":"textfile"}},"eventTimestamp":"2017-08-
21T15:59:30Z"}}},"operationName":"CreateFileReceivedEvent"}'
My scala code:
step1: copying the entire json string(pay load) to txt file
'{"query":"mutation CreateFileReceivedEvent($createFileReceivedEventInput:
CreateFleReceivedEventInput!) { createFileReceivedEvent(input:
$createFileReceivedEventInput) { clientMutationId }}","variables":
{"createFileReceivedEventInput":
{"clientMutationId":"Test","fileReceivedEvent":{"file":
{"fileTrackingId":"83a86c44-66a5-4de0-9b7f-
c6995877279d","name":"textfile_2017-08-21T15:58:45Z","fileType":
{"code":"textfile"}},"eventTimestamp":"2017-08-
21T15:59:30Z"}}},"operationName":"CreateFileReceivedEvent"}'
step2:
val data=fromFile("/usr/test/data.txt").getLines.mkString
step3:
val cmd = Seq("curl", "http://laptpad1811:5000/graphql", "-H",
"'Content-Type:application/json'" ,"-X", "POST", "-d" , data)
step4:
cmd.!!
I get the below error
String =
"{
"errors": [
{
"message": "Must provide query string.",
"stack": "BadRequestError: Must provide query string.\n
I have tried to change " to ' and mutiple combinations of the json string but I always get the same error.
I suspect that your issue is that sys.process doesn't pass commands through the shell (e.g. bash), so quotes that are necessary in the shell become unnecessary in Scala (and get passed through to the command which in the case of Unix-style utilities will probably result in unexpected behavior).
So try:
val cmd = Seq("curl", "http://laptpad1811:5000/graphql", "-H", "Content-Type: application/json", "-X", "POST", "-d", data)
Likewise remove the single quote wrapping from your text file.
I would however, counsel against spawning a curl in Scala and advise using one of the existing http client libraries (I personally like Gigahorse).

How do I get field from HTTP GET JSON result to file?

I am trying to make a HTTP GET request to an API service and push one of the returned fields in the JSON result to a txt file.
Based on this previously asked question: (Getting JSON value from cURL in Linux Bash)
...I have a bash script as follows...
TOKEN_FILE="/myhome/project/resources/auto_token.txt"
AUTH_RESULT=$(curl -i -H "Content-Type: application/json" "https://access.mywebservice.com/access/oauth/token?grant_type=client_credentials&client_id=123456&client_secret=MySecretPassword");
RESULT_FIELDS=$( cat <<EOF | json_reformat | \
sed -rne '/:/s#^\s+"(\w+)":\s+"([^"]+)",?#json_\1="\2"#gp'
[$AUTH_RESULT]
EOF
)
if [ -f "$TOKEN_FILE" ]
then
echo "$RESULT_FIELDS" > "$TOKEN_FILE"
fi
The expected JSON result looks like this (copied from Postman):
{
"access_token": "eyJ5bGciOiJSUzI1NiJ6.eyJzY29wZSI6WyJDUl7iLCJNQVAiLCJQVFkiLCJ8R1QiLCJTVFMiLCJUVEwiXSwiaXNzIjoiaHR0cHM6Ly9hY2Nlc3MtdWF0LWFwaS5jb3JlbG9naWMuYXNpYSIsImVudl9hx2Nlc3NfcmVzdHJpY3QiOmZhbHNlLCJleHAiOjE0NjcyODMwODcsImNsaWVudF9pZCI6IjhhOTY4OGJjIn0.F2iQfVsi9zntOxKYrNRukSIwuQ_LGSi_WMIXKII2A3GOEaqs-WmFTi7az9rvvfDsOl9rHy_s_66A6PiCpPftyw21Fl0aZZRoFcKv2H_zDUHuxOEs8V36jHeLghV7pjHwYI_nG68CIGvfuRWFNzQuiMFWc_i8oB3n5noSd8fQqa4",
"token_type": "bearer",
"expires_in": 43199,
"scope": "PROD1 PROD2 PROD3",
"iss": "https://access.mywebservice.com",
"env_access_restrict": false
}
I get the following errors returned...
bash-4.1$ ./token_renewal_test_05.sh
: command not foundt_05.sh: line 2:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
115 576 0 576 0 0 2266 0 --:--:-- --:--:-- --:--:-- 30315
: command not foundt_05.sh: line 3:
: command not foundt_05.sh: line 4:
./token_renewal_test_05.sh: line 14: warning: here-document at line 10 delimited by end-of-file (wanted `EOF')
./token_renewal_test_05.sh: line 13: warning: here-document at line 9 delimited by end-of-file (wanted `EOF')
: command not foundt_05.sh: line 13:
lexical error: invalid char in json text.
sed -rne '/:/s#^\s+"(\w+)":\s+"
(right here) ------^
: command not foundt_05.sh: line 10:
./token_renewal_test_05.sh: line 16: syntax error: unexpected end of file
I'm a bit new to bash and despite what appears to be a direct pointer to the issue am having problems resolving this one (note this is version 5)!
Can anyone offer any assistance with this one?
PS: I do not have jq either.
Thanks!
Regards,
Chris
Caveat emptor as per this comment on Parsing JSON with UNIX tools.
A working solution for your format:
eval $(cat <<EOF | \
sed -re 's/(,|\{|\})//g' | \
sed -re 's/"(\w+)":\s*"?([^"]*)"?$/json_\1='\''\2'\''/'
$JSON
EOF
)
set | grep '^json_'
json_access_token=eyJ5bGciOiJSUzI1NiJ6.eyJzY29wZSI6WyJDUl7iLCJNQVAiLCJQVFkiLCJ8R1QiLCJTVFMiLCJUVEwiXSwiaXNzIjoiaHR0cHM6Ly9hY2Nlc3MtdWF0LWFwaS5jb3JlbG9naWMuYXNpYSIsImVudl9hx2Nlc3NfcmVzdHJpY3QiOmZhbHNlLCJleHAiOjE0NjcyODMwODcsImNsaWVudF9pZCI6IjhhOTY4OGJjIn0.F2iQfVsi9zntOxKYrNRukSIwuQ_LGSi_WMIXKII2A3GOEaqs-WmFTi7az9rvvfDsOl9rHy_s_66A6PiCpPftyw21Fl0aZZRoFcKv2H_zDUHuxOEs8V36jHeLghV7pjHwYI_nG68CIGvfuRWFNzQuiMFWc_i8oB3n5noSd8fQqa4
json_env_access_restrict=false
json_expires_in=43199
json_iss=https://access.mywebservice.com
json_scope='PROD1 PROD2 PROD3'
json_token_type=bearer
Thanks again Chepner and Drew
I was having too many issues with Sed (probably due to my lack of exprience). As it turns out, I tried using a lookbehind. Sed doesn't have this but grep does so knowing the strcuture of my JSON response will never chance, I was able to get my token extracted using the following with grep instead...
grep -o -P '(?<="access_token":").*(?=","token_type")'

Error using Octave 4.0.2 to submit Coursera assignments

This is the error:
curl: (1) Protocol "https" not supported or disabled in libcurl
!! Submission failed: unexpected error: input file does not exist
!! Please try again later.
I am using Windows 10.
I see a possibly relevant answer here, but I don't know where this code would be added within Octave.
The URL is changed. Use the new one in submissionUrl() function in lib/submitWithConfiguration.m file.
function submissionUrl = submissionUrl()
%submissionUrl = 'https://www-origin.coursera.org/api/onDemandProgrammingImmediateFormSubmissions.v1';
submissionUrl = 'https://www.coursera.org/api/onDemandProgrammingImmediateFormSubmissions.v1';
end
For check URL you can use curl in terminal.
curl -k 'https://www.coursera.org/api/onDemandProgrammingImmediateFormSubmissions.v1'
You must get something like {"message":"","statusCode":404}
With wrong URL you dose't get anything.
Try to use the patch that changes following lines in the response function of submitWithConfiguration.m:
params = {'jsonBody', body};
%responseBody = urlread(submissionUrl, 'post', params); OLD CODE
[code, responseBody] = system(sprintf('echo jsonBody=%s | curl -k -X POST -d #- %s', body, submissionUrl));
d #- takes data in a file on the current stdin (the echo fills in).
-k allows curl to perform "insecure" SSL
(see curl --help)
HTH
==================
your code is the one i have, but i'm W7.
Do another try by setting quotes around the url in :
function submissionUrl = submissionUrl()
submissionUrl =
'"https://www-origin.coursera.org/api/onDemandProgrammingImmediateFormSubmissions.v1"';
end
(caution use : ' " and " ' that will quote the "https://.." on the command line.)
If it doesn't work, do a direct call to coursera with a command line (cmd) :
curl -k "https://www-origin.coursera.org/api/onDemandProgrammingImmediateFormSubmissions.v1"
This will call coursera and, as there is no sent form , the site will respond with an html page with near the end ... Action not found ....
if this works, the pb is probably not inside curl, but somewhere else. let us know.
There was a typo in Answer #1, which was corrected in Answer #2.
The change is:
In the function,
function response = submitParts(conf, email, token, parts) Apply the following changes
Comment the line responseBody = urlread(submissionUrl, 'post', params);
Type the following in place of it,
[code, responseBody] = system(sprintf('echo jsonBody=%s | curl -k -XPOST -d #- %s', body, submissionUrl));
So the final code of the function looks like
function response = submitParts(conf, email, token, parts)
body = makePostBody(conf, email, token, parts);
submissionUrl = submissionUrl();
params = {'jsonBody', body};
#responseBody = urlread(submissionUrl, 'post', params);
[code, responseBody] = system(sprintf('echo jsonBody=%s | curl -k -XPOST -d #- %s', body, submissionUrl));
response = loadjson(responseBody);
end
Change the following in submitWithConfiguration.m:
curl -k -X POST
to
curl -k -XPOST
and try again.
I just ran into this issue on Windows 10 today. In my case the request was performing correctly but the curl command was outputting timing information by default which was throwing off the validation logic in the submission script.
The submission was succeeding, but if I printed the response string, it looked something like this:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 1562 100 548 100 1014 548 1014 0:00:01 --:--:-- 0:00:01 2082
100 1562 100 548 100 1014 548 1014 0:00:01 --:--:-- 0:00:01 2082
{"id":"Blablablabla","courseId":"Blablabla","itemId":"Blabla",...}
I noticed that it was using the curl command to make the request, so I added the --silent flag to the code that creates the curl command to execute in submitWithConfiguration.m (in my case on line 134).
% use urlread or curl to send submit results to the grader and get a response
function response = getResponse(url, body)
% try using urlread() and a secure connection
params = {'jsonBody', body};
[response, success] = urlread(url, 'post', params);
if (success == 0)
% urlread didn't work, try curl & the peer certificate patch
if ispc
% testing note: use 'jsonBody =' for a test case
json_command = sprintf('echo jsonBody=%s | curl --silent -k -X POST -d #- %s', body, url);
% ^^^^^^^^ this right here!!
else
% it's linux/OS X, so use the other form
json_command = sprintf('echo ''jsonBody=%s'' | curl --silent -k -X POST -d #- %s', body, url);
end
% get the response body for the peer certificate patch method
[code, response] = system(json_command);
% test the success code
if (code ~= 0)
fprintf('[error] submission with curl() was not successful\n');
end
end
end
Now the response looked like a more reasonable:
{"id":"Blablablabla","courseId":"Blablabla","itemId":"Blabla",...}
And the submittion completed successfully.
I was having the same problem. All I did to overcome this problem is, I changed the path to ex2 folder and it worked for me.

SNMP OUTPUT OPTIONS - How do I get the OID response value only?

I have to go through and collect a few OIDs from some SNMP enabled network printers with a BASH script I have been working on.
My Request:
snmpget -v2c -c public 192.168.0.77
.1.3.6.1.2.1.1.1
.1.3.6.1.2.1.1.2
My Actual Response:
.1.3.6.1.2.1.1.1 = Counter32: 1974
.1.3.6.1.2.1.1.2 = Counter32: 633940
The Desired Response:
1974
633940314
(just the oid values only)
I looked and tested several options using the resource from the site below:
http://www.netsnmp.org/docs/man/snmpcmd.html#lbAF
-Oq removes '=' so running
snmpget -v2c -c public -Oq 10.15.105.133
.1.3.6.1.2.1.1.1
.1.3.6.1.2.1.1.2
returns
.1.3.6.1.2.1.1.1 Counter32: 1974
.1.3.6.1.2.1.1.2 Counter 32: 633940314
so I know I am phrasing my request properly.
I am taking the values and writing them to a MYSQL dB, I set the data types in my tale schema, the request is consistent so I know the definition of the OID, so I do not need all the information I am getting back, just the value of the OID itself, so I can write it to my dB without manipulating the the response. I probably can manipulate the response pulling the information to the right of ":" and writing the value of the OID.
I am relatively new to SNMP (http://www.net-snmp.org/), but I can not see why this is not a more commonly asked question because I have been searching everywhere for an answer and this post is my last recourse...
You can tune the output with the -O argument:
snmpgetnext -Oqv -v 2c -c public 192.168.0.77 .1
2
See the --help:
q: quick print for easier parsing
v: print values only (not OID = value)
You can postprocess the output with a simple Awk or sed script, or even just grep (provided you have grep -P).
snmpget -v2c -c public 192.168.0.77 <<'____HERE' | awk '{ print $4 }'
.1.3.6.1.2.1.1.1
.1.3.6.1.2.1.1.2
____HERE
or
.... | sed 's/.*: //'
or
.... | grep -oP ':\K[0-9]+'