Jenkins Build Time Trend API does not yield output using curl API - json

I got this link to get the Build Time Trend along with other Data in jenkins
https://jenkins:8080/view/<view-name>/job/<job-name>/<buildnumber>/api/json
This works well in a web browser but this does not seem to work with curl, does not give any result when I run along with curl command
This is what I tried
curl -u user:api_token -s -k "https://jenkins:8080/view/<view-name>/job/<job-name>/<buildnumber>/api/json"
This syntax worked with other API's.
Not sure what is wrong here.
curl -u userid:api_token -s -k "https://jenkins:8080/view/<view-name>/job/<job-name>/<buildnumber>/api/json" | jq.'causes[]|{result}'
jq.causes[]|{result}: command not found

You need a space between jq and its arguments (and probably not a period).
... | jq 'causes[]|{result}'
^
space here

Related

Can't execute some of the curl commands in the quickstart guide

I am working through the Digital Asset quickstart guide.
I am able to run:
curl -X GET http://localhost:8080/iou
And:
curl -X GET http://localhost:8080/iou/0
Without a problem. However, I am having trouble running:
curl -X PUT -d '{"issuer":"Alice","owner":"Alice","currency":"AliceCoin","amount":1.0,"observers":[]}' http://localhost:8080/iou
And:
curl -X POST -d '{ "newOwner":"Bob" }' http://localhost:8080/iou/ID/transfer
I get an output of
<html><body><h2>500 Internal Server Error</h2></body></html>
Is there a log somewhere that allows me to see what error occured? How can I debug this issue?
First I stopped the mvn, navigator and sandbox processes. Then I re-ran
da run damlc -- package daml/Main.daml target/daml/iou
Then I restarted sandbox. and re-entered
mvn clean compile exec:java
Now it works fine...

Stress test API using multiple JSON files

I am trying to fire 40000 requests towards an API using 40000 different JSON files.
Normally I could do something like this:
for file in /dir/*.json
do
#ab -p $file -T application/json -c1 -n1 <url>
curl -X POST -d#"$file" <url> -H "Content-Type: application/json"
done;
My problem is that I want to run simultaneous requests, e.g. 100 and I want the total time it took to send all requests etc. recorded. I can't use the -c 100 -n 40000 in ab since its the same URL with different files.
The files/requests all look something like
{"source":"000000000000","type":"A"}
{"source":"000000000001","type":"A"}
{"source":"000000000003","type":"A"}
I was not able to find any tool that supports this out of the box (e.g. Apache Benchmark - ab).
I came across this example here on SO (modded for this question).
Not sure I understand why that example would "cat /tmp" when mkfifo tmp is a file and not a dir though. Might work?
mkfifo tmp
counter=0
for file in /dir/*.json
do
if [ $counter -lt 100 ]; then
curl -X POST -H "Content-Type: application/json" -d#"$file" <url> &
let $[counter++];
else
read x < tmp
curl -X POST -H "Content-Type: application/json" -d#"$file" <url> &
fi
done;
cat /tmp > /dev/null
rm tmp
How should I go about achieving this in perl, ksh, bash or similar or does anyone know any tools that supports this out of the box?
Thanks!
If your request is just to time the total time take for sending these 40000 curl requests with different JSON each time, you can use good use of GNU parallel. The tool has great ways achieve job concurrency by making use of multiple cores on your machine.
The download procedure is quite simple. Follow How to install GNU parallel (noarc.rpm) on CentOS 7 for quick and easy list of steps. The tool has a lot more complicated flags to solve multiple use-cases. For your requirement though, just go the folder containing these JSON files and do
parallel --dry-run -j10 curl -X POST -H "Content-Type: application/json" -d#{} <url> ::: *.json
The above command tries to dry run your command, in terms of how parallel sets up the flags and processes its arguments and starts running your command. Here {} represents your JSON file. We've specified here to run 10 jobs at a time and increase the number depending on how fast it runs on your machine and by checking the number of cores on your machine. There are also flags to limit the overall CPU to be allowed use by parallel, so that it doesn't totally choke your system.
Remove --dry-run to run your actual command. And to clock the time taken for the process to complete, use the time command just prefix it before the actual command as time parallel ...

Docker API can’t apply json filters

According to the https://docs.docker.com/engine/reference/api/docker_remote_api_v1.24/#/list-tasks, filter can be only used to get running containers with a particular service name. For some reason, I am getting a full list of all tasks regardless of their names or desired states. I can't find any proper examples of using curl with JSON requests with Docker API.
I'm using the following command:
A)
curl -X GET -H "Content-Type: application/json" -d '{"filters":[{ "service":"demo", "desired-state":"running" }]}' https://HOSTNAME:2376/tasks --cert ~/.docker/cert.pem --key ~/.docker/key.pem --cacert ~/.docker/ca.pem
Returns everything
B)
trying to get something working from Docker Remote API Filter Exited
curl https://HOSTNAME:2376/containers/json?all=1&filters={%22status%22:[%22exited%22]} --cert ~/.docker/cert.pem --key ~/.docker/key.pem --cacert ~/.docker/ca.pem
This one returns "curl: (60) Peer's Certificate issuer is not recognized.", so I guess that curl request is malformed.
I have asked on Docker forums and they helped a little. I'm amazed that there are no proper documentation anywhere on the internet on how to use Docker API with curl or is it so obvious and I don't understand something?
I should prefix this with the fact that I have never seen curl erroneously report a certificate error when in fact there was some sort of other issue in play, but I will trust your assertion that this is in fact not a certificate problem.
I thought at first that your argument to filters was incorrect, because
according to the API reference, the filters parameter is...
a JSON encoded value of the filters (a map[string][]string) to process on the containers list.
I wasn't exactly sure how to interpret map[string][]string, so I set up a logging proxy between my Docker client and server and ran docker ps -f status=exited, which produced the following request:
GET /v1.24/containers/json?filters=%7B%22status%22%3A%7B%22exited%22%3Atrue%7D%7D HTTP/1.1\r
If we decode the argument to filters, we see that it is:
{"status":{"exited":true}}
Whereas you are passing:
{"status":["exited"]}
So that's different, obviously, and I was assuming that was the source of the problem...but when trying to verify that, I ran into a curious problem. I can't even run your curl command line as written, because curl tries to perform some globbing behavior due to the braces:
$ curl http://localhost:2376/containers/json'?filters={%22status%22:[%22exited%22]}'
curl: (3) [globbing] nested brace in column 67
If I correctly quote your arguments to filter:
$ python -c 'import urllib; print urllib.quote("""{"status":["exited"]}""")'
%7B%22status%22%3A%5B%22exited%22%5D%7D
It seems to work just fine:
$ curl http://localhost:2376/containers/json'?filters=%7B%22status%22%3A%5B%22exited%22%5D%7D'
[{"Id":...
I can get the same behavior if I use your original expression and pass -g (aka --globoff) to disable the brace expansion:
$ curl -g http://localhost:2376/containers/json'?filters={%22status%22:[%22exited%22]}'
[{"Id":...
One thing I would like to emphasize is the utility of sticking a proxy between the docker client and server. If you ever find yourself asking, "how do I use this API?", an excellent answer is to see exactly what the Docker client is doing in the same situation.
You can create a logging proxy using socat. Here is an example.
docker run -v /var/run/docker.sock:/var/run/docker.sock -p 127.0.0.1:1234:1234 bobrik/socat -v TCP-LISTEN:1234,fork UNIX-CONNECT:/var/run/docker.sock
Then run a command like so in another window.
docker -H localhost:1234 run --rm -p 2222:2222 hello-world
This example uses docker on ubuntu.
A docker REST proxy can be simple like this:
https://github.com/laoshanxi/app-mesh/blob/main/src/sdk/docker/docker-rest.go
Then you can curl like this:
curl -g http://127.0.0.1:6058/containers/json'?filters={%22name%22:[%22jenkins%22]}'

I'm using cURL -s and getting back a progress meter [duplicate]

I'm trying to use cURL in a script and get it to not show the progress bar.
I've tried the -s, -silent, -S, and -quiet options, but none of them work.
Here's a typical command I've tried:
curl -s http://google.com > temp.html
I only get the progress bar when pushing it to a file, so curl -s http://google.com doesn't have a progress bar, but curl -s http://google.com > temp.html does.
curl -s http://google.com > temp.html
works for curl version 7.19.5 on Ubuntu 9.10 (no progress bar). But if for some reason that does not work on your platform, you could always redirect stderr to /dev/null:
curl http://google.com 2>/dev/null > temp.html
In curl version 7.22.0 on Ubuntu and 7.24.0 on OSX the solution to not show progress but to show errors is to use both -s (--silent) and -S (--show-error) like so:
curl -sS http://google.com > temp.html
This works for both redirected output > /some/file, piped output | less and outputting directly to the terminal for me.
Update: Since curl 7.67.0 there is a new option --no-progress-meter which does precisely this and nothing else, see clonejo's answer for more details.
I found that with curl 7.18.2 the download progress bar is not hidden with:
curl -s http://google.com > temp.html
but it is with:
curl -ss http://google.com > temp.html
Since curl 7.67.0 (2019-11-06) there is --no-progress-meter, which does exactly this, and nothing else. From the man page:
--no-progress-meter
Option to switch off the progress meter output without muting or
otherwise affecting warning and informational messages like -s,
--silent does.
Note that this is the negated option name documented. You can
thus use --progress-meter to enable the progress meter again.
See also -v, --verbose and -s, --silent. Added in 7.67.0.
It's available in Ubuntu ≥20.04 and Debian ≥11 (Bullseye).
For a bit of history on curl's verbosity options, you can read Daniel Stenberg's blog post.
Not sure why it's doing that. Try -s with the -o option to set the output file instead of >.
this could help..
curl 'http://example.com' > /dev/null
On macOS 10.13.6 (High Sierra), the -sS option works. It is especially useful inside Perl, in a command like curl -sS --get {someURL}, which frankly is a whole lot more simple than any of the LWP or HTTP wrappers, for just getting a website or web page's contents.

zabbix_sender syntax for discovery rules

I have successfully installed a zabbix environment. Now I want to use zabbix_sender, to send data from a third party program to zabbix. I created a host “api_test”, and an item “test item ” with the key “java.test.item”. Sending
zabbix_sender -z localhost -p 10051 -s "api_test" -k java.test.item -o 1234
from the linux server works perfectly and adds a dataset as expected.
The problem is, that I would like to use a discovery item, and I cannot find the right syntax for zabbix_sender. Here is how I configured the discovery rule:
And this is the Item Prototype:
I expected the following query to add an Item based on the item prototype, but nothing happens:
zabbix_sender -z localhost -p 10051 -s "api_test" -k java.th.discovery -o '{"data":[{"{#THNAME}:"test_thread"}]}'
I also tried with different quotations marks (single, double, without), but nothing seems to work.
Consequently, the following query I tried afterwards fails:
zabbix_sender -z localhost -p 10051 -s "api_test" -k java.th.ex["test_thread"] -o 98765
Question is: where am I mistaken? I guess it is the discovery rule, or the zabbix_sender syntax for discovery, but I cannot find anything in the documentation.
Any help is appreciated!
Steffen, your configuration in Zabbix frontend is correct.
However, there is a mistake in JSON syntax that you use in the command line. Double quotes after {#THNAME} are missing:
{"data":[{"{#THNAME}:"test_thread"}]}
You should see the error message about it in discovery list:
It should work after that issue is fixed. If not, please provide details about your "#Thread for discovery" macro and the error message that you get.