Watson API - Retrieve and Rank - Error uploading JSON - json

I'm following the tutorial at Retrieve and Rank - Get Started, and I'm at the following step:
Issue the following command to upload the cranfield_data.json data to the example_collection collection. Replace {username}, {password}, {solr_cluster_id}, and {/path_to_file} with your information:
$ curl -X POST -H "Content-Type: application/json" -u "{username}":"{password}" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/{solr_cluster_id}/solr/example_collection/update" --data-binary #{/path_to_file}/cranfield_data.json
I'm lobbing the request with the correct username and password, and correct cluster_id and path to the json, but I get the following error:
$ curl -X POST -H "Content-Type: application/json" -u "username":"password" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/cluster_id/solr/example_collection/update" --data-binary #forum_data/parsed_answers.json
Error: WRRCSH006: Error forwarding request [/solr/example_collection/update] for Solr cluster [sc5b47c5e3_bab3_4aff_a818_f0d786d6dece].

Turns out there were characters in the JSON causing it be malformed.

Just verify the JSON "parsed_answers.json", to check for all the punctuations i.e. ";","," to be correctly placed and as per the defined schema, and try to re-upload

Related

How to GET data of a key in json using curl?

Sorry Pretty noob to json.
Basically I have a simple server where I can upload data in there.
E.g:
curl -vX PUT "http://IP:port/ABC" -H "Content-Type: application/json" -d #"Once Upon a time."
After when I do:
curl -vX GET "http://IP:port/ABC" -H "Content-Type: application/json"
I get:
{"reverse_shell":
{"aliases":{},"mappings":{},"settings":
{"index":{"creation_date":"1561863982371","number_of_shards":"5","number_of_replicas":"1","uuid":"IAWE83rYQqmtKW-9svkBVg","version":{"created":"6040299"},"provided_name":"ABC"}
}
}
}
As you can see there is no where mentioning Once Upon a time, so is there I am missing? or how do I get that data from json using curl?
I am in kali linux env.
It looks like you are trying to post a string as a file.
When you specify a "#" with -d this tells curl to send the data from a file called "Once Upon a time."
If you are trying to put a file then you should do:
my_text_file.txt
Once Upon a time.
curl -vX PUT "http://IP:port/ABC" -H "Content-Type: application/json" -d #./my_text_file.txt https://server/api/path

wso2 5.6 identity server SCIM user creation with email and while find the users email is not returning from JSON

While creating a user using SCIM1.1 endpoint with email.
curl -v -k --user admin:admin --data '{"schemas":[],"name":{"familyName":"gunasinghe","givenName":"hasinitg"},"userName":"hasinitg_sample","password":"hasinitg","emails":[{"primary":true,"value":"hasini_home.com"}]}' --header "Content-Type:application/json" https://localhost:9443/wso2/scim/Users
Created User response:
{"emails":[{"value":"hasini_home.com"}],"meta":{"created":"2018-09-18T15:43:17","location":"https://localhost:9443/wso2/scim/Users/37f051b2-770c-4915-9a3f-623a7404b2a4","lastModified":"2018-09-18T15:43:17"},"schemas":["urn:scim:schemas:core:1.0"],"name":{"familyName":"gunasinghe","givenName":"hasinitg"},"id":"37f051b2-770c-4915-9a3f-623a7404b2a4","userName":"hasinitg_sample"}
Retrieve the USER from SCIM endpoint:
curl -v -k --user admin:admin https://localhost:9443/wso2/scim/Users/37f051b2-770c-4915-9a3f-623a7404b2a4
The response is:
{"meta":{"created":"2018-09-18T15:43:17","location":"https://localhost:9443/wso2/scim/Users/37f051b2-770c-4915-9a3f-623a7404b2a4","lastModified":"2018-09-18T15:43:17"},"schemas":["urn:scim:schemas:core:1.0"],"roles":["Internal/everyone"],"name":{"familyName":"gunasinghe","givenName":"hasinitg"},"id":"37f051b2-770c-4915-9a3f-623a7404b2a4","userName":"hasinitg_sample"}
The email is missing here in the response and My-Sql DB as well. Any idea?
The issue seems to be that if you are sending the email as a JSON object, you have to set the type as well. Following debug log is written when we send an email object without the type attribute.
TID: [-1234] [] [2018-09-19] DEBUG
{org.wso2.carbon.identity.scim.common.utils.AttributeMapper} - Type
attribute cannot be null for sub attribute:
org.wso2.charon.core.attributes.ComplexAttribute#71ebf24c of Complex
MultiValuedAttribute: emails. Skip adding as a claim
Please see the given sample request in the documentation
{"primary":true,"value":"hasini_home.com","type":"home"}
You can use the following request to simply solve your problem.
curl -v -k --user admin:admin --data '{"schemas":[],"name":{"familyName":"gunasinghe","givenName":"hasinitg"},"userName":"hasinitg_sample","password":"hasinitg","emails":"hasini_home.com"}' --header "Content-Type:application/json" https://localhost:9443/wso2/scim/Users

pushbullet api decoding (v2/push)

I want to use the pushbullet api (v2/push) for pushing messages, but if I include '%' character inside title or body the server gives me the following error:
{"error":{"type":"invalid_request","message":"Failed to decode urlencoded POST form body.","cat":"~(=^‥^)ノ"}}
How can I fix this problem?
request: curl https://api.pushbullet.com/v2/pushes -k -u token: -d type=note -d title="%test" -d body="%%test" -X POST
x-www-form-urlencoded is not the most straightforward of formats. You can probably use curl with the --data-urlencode option. You can also try encoding your values with this tool: http://meyerweb.com/eric/tools/dencoder/
That should produce urlencoded output, for instance your request would look more like:
curl -u token: https://api.pushbullet.com/v2/pushes --header "Content-Type: application/x-www-form-urlencoded" --data-binary 'type=note&title=TITLETEXT&body=%25BODYTEXT'

Use cURL to add a JSON web page's data in Solr

I see from the UpdateJSON page how to use a command prompt to index a standalone file stored locally. Using this example I was able to successfully make a .json file accessible through Solr:
curl 'http://localhost:8983/solr/update/json?commit=true' --data-binary #books.json -H 'Content-type:application/json'
What I'm not able to find is the proper syntax to do the same for a webpage containing JSON data. I've tried with the #:
curl 'http://localhost:8983/solr/update/json?commit=true' --data-binary #[URL] -H 'Content-type:application/json'
and without:
curl 'http://localhost:8983/solr/update/json?commit=true' --data-binary [URL] -H 'Content-type:application/json'
Both ways lead to errors. How do I configure a command to prompt Solr to index the contents at [URL]?
According to documentation, (https://wiki.apache.org/solr/ContentStream) you should first ensure remote streaming is enabled (solrconfig, search for enableRemoteStreaming)
Then the command should be of the kind:
curl 'http://localhost:8983/solr/update/json?commit=true&stream.url=YOURURL' -H 'Content-type:application/json'

Github returns "Problems parsing JSON" when trying to create repo using curl

I'm attempting to create a repo on github using curl. No matter how I attempt to format my JSON it always returns "Problems parsing JSON". Is there something I'm missing? Here is my statement:
curl --data '{"name":"newRepoName"}' -XPOST -u username:password http://giturl
Strange since it works for me. Are you using the correct GitHub API endpoint (/user/repos)?
curl --data '{"name":"testrepo"}' -X POST -u username https://api.github.com/user/repos
Try including your token or password like this
curl --data '{"name":"testrepo"}' \
-X POST -u username:$TOKEN \
https://api.github.com/user/repos