I have a Nexus Repository server where my artifacts are stored. I want to write a shell script to download artifacts from here. When using the curl request curl --user username:password -X GET "http://your_ip:your_port/service/rest/v1/search?repository=your_repository" -H "accept: application/json" I get a list of the items in my repository which looks like this:
{
"items": [
{
"id": "dGVzdC1hcHA6ZDM1MTBiN2FkMThkODJjZGU1NjNhMWVlMWFmOWIwMGQ",
"repository": "test-app",
"format": "maven2",
"group": "no.ahj",
"name": "test-app",
"version": "1.0-20190715.130341-2",
"assets": [
{
"downloadUrl": "http://192.168.56.2:8081/repository/test-app/no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom",
"path": "no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom",
"id": "dGVzdC1hcHA6Yzc3MDE2OWMwYjJlM2VkODU0MGMyOGEwOWQ0Njk4ZTQ",
"repository": "test-app",
"format": "maven2",
"checksum": {
"sha1": "5fd032774dd3ae6fbbd6484b3dc6ef2582d9b397",
"md5": "3a6aa8e295a734fdb8a8df782c0a14d5"
}
},
I would like my shell script to run this curl request, extract the value from the downloadURL field, store it in some variable and then use wget with this variable to download the file. So my question is this: How can I take the URL from downloadURL and store/use it in my shell script?
A way to parse it with likely no external dependencies (as python is installed by default on most Linux distributions) is just to use python:
user#host ~ % JSON=$(curl --user username:password -X GET "http://your_ip:your_port/service/rest/v1/search?repository=your_repository" -H "accept: application/json")
user#host ~ % echo $JSON | python -c 'import sys, json; print(json.load(sys.stdin)["items"][0]["assets"][0]["downloadUrl"])'
http://192.168.56.2:8081/repository/test-app/no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom
If you are going to do a lot of JSON parsing in this script, it may be worth considering writing the entire script in Python, too, instead of shell script.
Related
I have a windows batch script to perform POST request using curl and reads the data from JSON file, it works fine with only a single object in the file and it looks like this.
curl -u username#password -H "Content-Type: application/json" -d #file.json http://apiurl.com
and the json file is this:
{
"name": "Empty name",
"properties": {
"active": "True",
"subcity_zone": "East Hararge",
"woreda": "Meta"
}
}
But now I want to send the request each object in the array by iterating each item. So, How do I iterate each JSON object from the file?
Here is what the new JSON array file looks like:
[{
"name": "test facility I",
"properties": {
"active": "True",
"city": "",
"subcity_zone": "East Hararge",
"woreda": "Meta"
}
},
{
"name": "test facility II",
"properties": {
"active": "True",
"subcity_zone": "East Hararge",
"woreda": "Girawa"
}
}]
Using jq:
jq -c '.[]' file | while read js; do
curl -u username#password -H "Content-Type: application/json" -d #<(echo "$js") http://apiurl.com
done
The jq command extracts the each object in one line that is read by read command into the $js variable.
The <(echo "$js") creates a temporary file that is passed to curl.
In the Watson API documentation, the following is given for adding query to training data using CURL:
curl -X POST -u "{username}":"{password}" -H "Content-Type: application/json" -d
'{
"natural_language_query": "who is keyser soze",
"filter": "text:criminology",
"examples": [
{
"document_id": "adaf50f1-2526-4fad-b670-7d6e8a42e6e6",
"relevance": 2
},
{
"document_id": "63919442-7d5b-4cae-ab7e-56f58b1390fe",
"cross_reference": "my_id_field:14",
"relevance": 4
}
]
}' "https://gateway.watsonplatform.net/discovery/api/v1/environments/{environment_id}/collections/{collection_id}/training_data?version=2018-03-05"
However much I try or use multiple libraries, it does not seem to have the correct JSON format (even when JSON validators have confirmed).
Has anyone tried using this method to add queries?
I have a JSON file in the following format:
"rows": [
{
"key": [
null,
null,
"dco_test_group",
"3d3ce6270fdfuashge12e1d41af93179",
"test_djougou"
],
"value": {
"lat": "31.538208354844658",
"long": "91.98762580927113"
}
},
{
"key": [
null,
null,
"dco_test_group",
"4cda7jhsadgfs6123hsdaa9069ade2",
"test_ouake"
],
"value": {
"lat": "59.696798503352547",
"long": "11.6626995307082464"
}
},
I want to import the file such that each object inside rows becomes a couchdb document. Right now, I have the following code:
curl -X PUT --data-binary #"C:\Users\me\Dropbox (Personal)\Research\Folder\location.json" http://127.0.0.1:5984/db/document_name
This adds all the data inside document_name.
If I try:
curl -X PUT --data-binary #"C:\Users\me\Dropbox (Personal)\Research\Folder\location.json" http://127.0.0.1:5984/db
a new db is created but no data gets added. How do I edit the code to get the desired output?
UPDATE 1
Does it matter if all the data is in record? Are there any rules analogous to 5 normal forms of RDB?
Use the bulk-document-api for this. Here is an example from the docs >> https://wiki.apache.org/couchdb/HTTP_Bulk_Document_API#Modify_Multiple_Documents_With_a_Single_Request
$ DB="http://127.0.0.1:5984/mydb"
$ curl -H "Content-type:application/json" -d '{"docs":[{"key":"baz","name":"bazzel"},{"key":"bar","name":"barry"}]}' -X POST $DB/_bulk_docs
$ curl -H "Content-type:application/json" -d #your_file.json -X POST $DB/_bulk_docs
Note, that all docs are items within a 'docs' array.
I need to delete some wrong data, inserted in a lot of processes, and I need to figure if this is possible with cURL and rest API, with a script in sh, batch or something like this:
curl -u admin:admin -i -H "Accept: application/json" -X GET "http://json_bpm.com/wle/v1/service/6112449?action=getData&fields=context"
First I just need the result map.
Output:
{"status":"200","data":{"result":"{\"context\":{\"name\":\"xxx\" (...)
"resultMap":{"context":{"name\":"xxx\" (...) }}}
Because I need to remove the userDelete array (see below) for thousands of processes, and set this again using curl. If you know how to remove arrays from JSON too, you're the man. :)
{
"context": {
"name": "Change Process",
"startUser": {
"user": "0001"
},
"endUser": {
"user": "0001"
},
"userDelete": {
"user": "0002"
},
"origin": "GUI",
"userAction": "Change Process"
}
}
I am trying to post the following JSON to a URL using cURL in Terminal:
[
{
"token": "ABCDEF",
"templateId": "{1234-5678-9}",
"senders": "null",
"viewers": "null",
"peoples": "null",
"fields": {
"Matter Name": "My test matter name",
"Matter Number": "ABC123"
}
}
]
This is how I POST it in Terminal:
curl -v -k -X POST -H "Content-Type: application/json" -d docfile=#test.json https://myWebsite.com/extension/extension/extension
The Terminal output clearly says that something was posted but the part that confuses me is this excerpt from the output: upload completely sent off: 18 out of 18 bytes
Only 18 bytes were sent? My file is 218 bytes...Why is this file not being POSTed? What is being POSTed?
The problem is with your -d switch. From the documentation:
-d
Sends the specified data in a POST request to the HTTP server... If you start the data with the letter #, the rest should be a file name to read the data from.
What you are passing to the -d switch does not begin with "#", so it is being interpreted as actual data. You'll notice docfile=#test.json IS actually 18 bytes.
You need to change it from -d docfile=#test.json to -d #test.json.