Importing json into couchdb such that each object becomes a document - json

I have a JSON file in the following format:
"rows": [
{
"key": [
null,
null,
"dco_test_group",
"3d3ce6270fdfuashge12e1d41af93179",
"test_djougou"
],
"value": {
"lat": "31.538208354844658",
"long": "91.98762580927113"
}
},
{
"key": [
null,
null,
"dco_test_group",
"4cda7jhsadgfs6123hsdaa9069ade2",
"test_ouake"
],
"value": {
"lat": "59.696798503352547",
"long": "11.6626995307082464"
}
},
I want to import the file such that each object inside rows becomes a couchdb document. Right now, I have the following code:
curl -X PUT --data-binary #"C:\Users\me\Dropbox (Personal)\Research\Folder\location.json" http://127.0.0.1:5984/db/document_name
This adds all the data inside document_name.
If I try:
curl -X PUT --data-binary #"C:\Users\me\Dropbox (Personal)\Research\Folder\location.json" http://127.0.0.1:5984/db
a new db is created but no data gets added. How do I edit the code to get the desired output?
UPDATE 1
Does it matter if all the data is in record? Are there any rules analogous to 5 normal forms of RDB?

Use the bulk-document-api for this. Here is an example from the docs >> https://wiki.apache.org/couchdb/HTTP_Bulk_Document_API#Modify_Multiple_Documents_With_a_Single_Request
$ DB="http://127.0.0.1:5984/mydb"
$ curl -H "Content-type:application/json" -d '{"docs":[{"key":"baz","name":"bazzel"},{"key":"bar","name":"barry"}]}' -X POST $DB/_bulk_docs
$ curl -H "Content-type:application/json" -d #your_file.json -X POST $DB/_bulk_docs
Note, that all docs are items within a 'docs' array.

Related

How to update a field with a new value in dynamic variable using bash?

How to update a field with a new value in a dynamic variable using bash?
I am storing some curl output in a variable below is the output response in it.
data=$(curl -s --location --request GET "${HOST}/api/v1/envs/projects/${PROJECT_ID}?page=0&pageSize=25" --header "Accept: application/json" --header "Content-Type: application/json" --header "Authorization: Bearer "$token"" | jq -r '.data[]')
echo "$data"
{
"id": "8aa0809085f46e000185f4c28b7a0b52",
"createdBy": "8aa0808e82fd01fb0182fd0b05820007",
"createdDate": "2023-01-27T19:44:28.538+0000",
"modifiedBy": "8aa0808e82fd01fb0182fd0b05820007",
"modifiedDate": "2023-01-27T19:45:14.811+0000",
"version": null,
"inactive": false,
"projectId": "8aa0809085f46e000185f4c28b760b51",
"name": "Master",
"refId": "Master",
"description": null,
"baseUrl": "http://petstore.swagger.io/v1",
}
Now I want to update the baseUrl field in that data variable dynamically and store that update complete JSON object either in same or different variable to be able use it in a different curl request operation.
How can we do that in bash scripting?
You are already using jq to extract part of the data returned from the API. You can extend this jq command to modify the baseURL field on the fly. The following will update baseURL in the first object in the data array, then print it out:
jq -r '.data[0].baseUrl = "http://example.com" | .data[0]'

Get JSON value from curl request

I have a Nexus Repository server where my artifacts are stored. I want to write a shell script to download artifacts from here. When using the curl request curl --user username:password -X GET "http://your_ip:your_port/service/rest/v1/search?repository=your_repository" -H "accept: application/json" I get a list of the items in my repository which looks like this:
{
"items": [
{
"id": "dGVzdC1hcHA6ZDM1MTBiN2FkMThkODJjZGU1NjNhMWVlMWFmOWIwMGQ",
"repository": "test-app",
"format": "maven2",
"group": "no.ahj",
"name": "test-app",
"version": "1.0-20190715.130341-2",
"assets": [
{
"downloadUrl": "http://192.168.56.2:8081/repository/test-app/no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom",
"path": "no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom",
"id": "dGVzdC1hcHA6Yzc3MDE2OWMwYjJlM2VkODU0MGMyOGEwOWQ0Njk4ZTQ",
"repository": "test-app",
"format": "maven2",
"checksum": {
"sha1": "5fd032774dd3ae6fbbd6484b3dc6ef2582d9b397",
"md5": "3a6aa8e295a734fdb8a8df782c0a14d5"
}
},
I would like my shell script to run this curl request, extract the value from the downloadURL field, store it in some variable and then use wget with this variable to download the file. So my question is this: How can I take the URL from downloadURL and store/use it in my shell script?
A way to parse it with likely no external dependencies (as python is installed by default on most Linux distributions) is just to use python:
user#host ~ % JSON=$(curl --user username:password -X GET "http://your_ip:your_port/service/rest/v1/search?repository=your_repository" -H "accept: application/json")
user#host ~ % echo $JSON | python -c 'import sys, json; print(json.load(sys.stdin)["items"][0]["assets"][0]["downloadUrl"])'
http://192.168.56.2:8081/repository/test-app/no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom
If you are going to do a lot of JSON parsing in this script, it may be worth considering writing the entire script in Python, too, instead of shell script.

Add query to training data using CURL

In the Watson API documentation, the following is given for adding query to training data using CURL:
curl -X POST -u "{username}":"{password}" -H "Content-Type: application/json" -d
'{
"natural_language_query": "who is keyser soze",
"filter": "text:criminology",
"examples": [
{
"document_id": "adaf50f1-2526-4fad-b670-7d6e8a42e6e6",
"relevance": 2
},
{
"document_id": "63919442-7d5b-4cae-ab7e-56f58b1390fe",
"cross_reference": "my_id_field:14",
"relevance": 4
}
]
}' "https://gateway.watsonplatform.net/discovery/api/v1/environments/{environment_id}/collections/{collection_id}/training_data?version=2018-03-05"
However much I try or use multiple libraries, it does not seem to have the correct JSON format (even when JSON validators have confirmed).
Has anyone tried using this method to add queries?

Uploading linked Revit models to Autodesk Forge

I've been trying to use the Post references (https://developer.autodesk.com/en/docs/model-derivative/v2/reference/http/urn-references-POST/) to set up the reference between two files in forge but although I get a message "success" as result when I try it on the forge viewer I still see the files separately even after I translate the models. Has someone been through the same issue?
Without seeing you code it is hard to tell what is happening. Below I copied my bash script code which references/translate an obj with material and texture.
Au.obj
+- Au.mtl
+- Au.jpg
After upload, I got these
idObj="urn:adsk.objects:os.object:cyrillejcrja/Au.obj"
idMtl="urn:adsk.objects:os.object:cyrillejcrja/Au.mtl"
idJpg="urn:adsk.objects:os.object:cyrillejcrja/Au.jpg"
the code to set references, now
urn=$(xbase64encode $idObj)
job='{
"urn": "'${idObj}'",
"filename": "Au.obj",
"references": [{
"urn": "'${idMtl}'",
"relativePath": "./Au.mtl",
"filename": "Au.mtl",
"references": [{
"urn": "'${idJpg}'",
"relativePath": "./Au.jpg"
}]
}]
}'
response=$(curl -H "Content-Type: application/json" \
-H "Authorization: ${bearer}" \
-X POST ${ForgeHost}/modelderivative/v2/designdata/${urn}/references \
-k -s -d "${job}")
Here is got a reply like below which only means that the references are registered.
{
"result": "success"
}
Now, I do this to translate the obj and use the references
urn=$(xbase64encode $idObj)
job='{
"input": {
"urn": "'${urn}'",
"checkReferences": true
},
"output": {
"formats": [
{
"type": "svf",
"views": [
"2d",
"3d"
]
}
]
}
}'
response=$(curl -H "Content-Type: application/json" \
-H "Authorization: ${bearer}" \
-H "x-ads-force: true" \
-X POST ${ForgeHost}/modelderivative/v2/designdata/job \
-k -s -d "${job}")
Note the "checkReferences": true, as documented here.
Now, I can wait the translation to complete and see the result in the Viewer.
For reference the xbase64safeencode function used above
function xbase64safeencode () { local id64=$(echo -ne $1 | base64 $wrap_arg | tr -d '=' | tr '+/' '-_'); echo $id64; }
#Cyrille, this is my request:
curl -X 'POST' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsI' -H 'Content-Type: application/json' -v 'https://developer.api.autodesk.com/modelderivative/v2/designdata/{urn}/references' -d
'{
"urn": "urn:adsk.objects:os.object:bucket/non-existent.rvt",
"filename": "",
"references": [
{
"urn": "urn:adsk.objects:os.object:bucket/non-existent.rvt",
"relativePath": "",
"filename": ""
}
]
}'
I got as result:
'{
"result": "success"
}'
The point is I'm getting success as a result even when I do not have the specified file on the server, so I'd suggest few server-side validations, for example, when a model has been translated once we can't set as reference right so it should at least return an error. Thank you and I hope this helps.

cURL Response JSON RestAPI

I need to delete some wrong data, inserted in a lot of processes, and I need to figure if this is possible with cURL and rest API, with a script in sh, batch or something like this:
curl -u admin:admin -i -H "Accept: application/json" -X GET "http://json_bpm.com/wle/v1/service/6112449?action=getData&fields=context"
First I just need the result map.
Output:
{"status":"200","data":{"result":"{\"context\":{\"name\":\"xxx\" (...)
"resultMap":{"context":{"name\":"xxx\" (...) }}}
Because I need to remove the userDelete array (see below) for thousands of processes, and set this again using curl. If you know how to remove arrays from JSON too, you're the man. :)
{
"context": {
"name": "Change Process",
"startUser": {
"user": "0001"
},
"endUser": {
"user": "0001"
},
"userDelete": {
"user": "0002"
},
"origin": "GUI",
"userAction": "Change Process"
}
}