I'm unable to understand pagination of Chargebee (https://apidocs.chargebee.com/docs/api) , I need to create a request in where I can add next offset to the request to get further data (without setting limit other then by default, which is 10). But i'm unable to understand how http request will be formed with this given next_offset like attached image.
Screenshot of request and response
I have had success submitting the identical request with the offset tacked onto the end. Here is an example:
First Request
curl -s https://xxyyzz.chargebee.com/api/v2/events -G -K /home/xxyyzz/.cb_curl_key.cfg --data-urlencode limit=2 --data-urlencode occurred_at[between]="[1554076800,1554077099]"
JSON results:
{
"list": [
{"event": {
"id": "ev_xxyyzz1",
"occurred_at": 1554077022,
...
"next_offset": "[\"1556428868000\",\"364450353\"]"
}
The tool I used to display the json is trying to be helpful with the backslashes.
The real next_offset value is
["1556428868000","364450353"]
Second Request
Identical to the first with
offset="["1554077017000","345017569"]"
tacked on to the end:
curl -s https://xxyyzz.chargebee.com/api/v2/events -G -K /home/xxyyzz/.cb_curl_key.cfg --data-urlencode limit=2 --data-urlencode --data-urlencode occurred_at[between]="[1554076800,1554077099]" offset="["1554077017000","345017569"]"
JSON results:
{
"list": [
{"event": {
"id": "ev_xxyyzz3",
"occurred_at": 1556429028,
}
Keep repeating the process until the "next_offset" key does not appear in the JSON result.
Related
This question already has answers here:
Parsing JSON with Unix tools
(45 answers)
Find the value of key from JSON
(5 answers)
Closed 2 years ago.
I'm trying to parse a curl request and parse the output and store it on a file called res.txt
Here is my bash cmd line:
curl --request POST --url 'https://www.virustotal.com/vtapi/v2/url/scan' --data 'apikey=XXXXXXXXXXXXXXX' --data 'url=abcde.xyz' >> grep -Po '"scan_id":.*?[^\\]",' res.txt
The output is something like this:
{"permalink": "https://www.virustotal.com/gui/url/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/detection/u-17f485d68047604e61b4067310ab716ae6fddc774bb46ffab06d081613b28e49-1595992331", "resource": "http://abcde.xyz/", "url": "http://abcde.xyz/", "response_code": 1, "scan_date": "2020-07-29 03:12:11", "scan_id": "000000000000000000000000000000000000000", "verbose_msg": "Scan request successfully queued, come back later for the report"}`
I want to store scan_id code on res.txt, but it is not working, no errors! And I do not know if my regex is correct
Can you help me?
The core of the question is about extracting values from JSON data (created by curl, in this specific case).
While it is possible to parse specific JSON data using regular expressiosns (assuming specific structure of while spaces/line breaks), it is very hard (impossible ?) to write regular expression that will cover all possible formatting. This is similar to parsing XML data - some formats can be parsed with regex, but extremely hard to write generic parser.
Instead of regex, consider using JSON specific tool, e.g., jq
Also, there construction of the pipe (curl to grep) should use '|' and not '>>', and the '>' should be used to specify the name of the file result. See below:
curl --request POST --url 'https://www.virustotal.com/vtapi/v2/url/scan' --data 'apikey=XXXXXXXXXXXXXXX' --data 'url=abcde.xyz' |
jq .scan_id > res.txt
To remove the quotes from the res.txt ,use the 'raw-output format of jq (jq -r .scan_id`)
If not possible to use jq for any reason, consider the following modification. It is using 'sed' (instead of grep) to extract the scan_id value (0000... in this case). It assumes that that the "scan_id" tag and value are on the same line.
curl --request POST --url 'https://www.virustotal.com/vtapi/v2/url/scan' --data 'apikey=XXXXXXXXXXXXXXX' --data 'url=abcde.xyz' |
sed -n -e 's/.*"scan_id": *"\([^"]*\)".*/\1/p' > res.txt
Try
curl --request POST --url 'https://www.virustotal.com/vtapi/v2/url/scan' --data 'apikey=XXXXXXXXXXXXXXX' --data 'url=abcde.xyz'| tr ',' '\n' | grep scan_id
Demo:
$"http://abcde.xyz/", "url": "http://abcde.xyz/", "response_code": 1, "scan_date": "2020-07-29 03:12:11", "scan_id": "000000000000000000000000000000000000000", "verbose_msg": "Scan request successfully queued, come back later for the report"}' <
{"permalink": "https://www.virustotal.com/gui/url/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/detection/u-17f485d68047604e61b4067310ab716ae6fddc774bb46ffab06d081613b28e49-1595992331", "resource": "http://abcde.xyz/", "url": "http://abcde.xyz/", "response_code": 1, "scan_date": "2020-07-29 03:12:11", "scan_id": "000000000000000000000000000000000000000", "verbose_msg": "Scan request successfully queued, come back later for the report"}
$: "000000000000000000000000000000000000000", "verbose_msg": "Scan request successfully queued, come back later for the report"}' | tr ',' '\n' | grep scan_id <
"scan_id": "000000000000000000000000000000000000000"
$
I want to build a request in insomnia to upload person, documents and it's files
How can I put a multipart file inside a JSON object? I don't want to deal with string base64 because it's too big and too slow to travel over the network.
I have a rest api made with spring boot and kotlin that will receive this JSON file.
Here's some code for what I want to achieve:
curl --request POST \
--url http://localhost:8080/ \
--header 'content-type: multipart/form-data; boundary=--
-011000010111000001101001' \
--form 'person={
"first_name": "Foo",
"last_name": "Fighters"
}' \
--form 'document=[
{
"document_name": "test1",
"document_description":"test1",
"document_file": "multipart file1"
},
{
"document_name": "test2",
"document_description":"test2",
"document_file": "multipart file2"
},
{
"document_name": "testN",
"document_description":"testN",
"document_file": "multipart fileN"
}
]'
Where the key document_file value stands for the file itself, not a String.
Some pictures to make it clear:
Here is the overview of the multipart
Person detail:
Document detail:
I need to know what files are from what documents, and I can have 0 or many documents related to the person.
Therefore, that's why adding 1 file for each document I want to create won't work. It needs to be inside the object(just like presented in the images) that way I know that file-X is from document-X and vice-versa.
Thanks in advance!
I'm trying to make a HTTP form POST request to http://example.co.uk.
I used curl http://example.co.uk -d #example.json. However it gives me error saying it should only contain single field called application.
example.json file:
{"application":[
{
"name":"John",
"email":"john#example.com",
"github":"https://github.com/john",
"twitter":"https://twitter.com/john",
}
]
}
What's the correct way to do it? Any help will be appreciated. Thanks
You'll need to indicate to the server that the content type is JSON instead of the standard form-encoded data, so use:
curl -H "Content-Type: application/json" http://example.co.uk -d #example.json
Also, your JSON file should be valid JSON i.e. without any ending/redundant , characters, so:
{"application":[
{
"name":"John",
"email":"john#example.com",
"github":"https://github.com/john",
"twitter":"https://twitter.com/john"
}
]
}
Check the contents at http://jsonlint.com
The online shopify docs for fulfillment show this example:
POST /admin/orders/#{id}/fulfillments.json
{
"fulfillment": {
"tracking_number": null,
"line_items": [
{
"id": 466157049
}
]
}
}
The docs also say that not specifying an item id will cause all items to fulfill.
My postdata to that api endpoint reads:
{"fulfillment":{"tracking_number":null}}
This comes back:
{"errors":{"fulfillment":"can't be blank"}}
I have tried this:
{"fulfillment":{"tracking_number":null,"line_items":[{"id":300668234}]}}
which is a valid item id for my order. The same message comes back.
Ideas?
Are you setting your Content-Type to application/json?
Trying sending the same request using cURL
curl -X POST -H 'Content-Type: application/json' -d #fulfillment.json https://API_KEY:API_TOKEN#SHOP.myshopify.com/admin/orders/ORDER_ID/fulfillments.json
If that works, it means you aren't sending your request correctly.
curl -H "Content-Type: application/json" -X POST -d '{"fulfillment": {"tracking_number": "123456789","notify_customer": true}}' https://API_KEY:TOKEN#SHOP.myshopify.com/admin/orders/ORDER_ID/fulfillments.json
This is the working example of updating tracking number for an order. It took me a minute to realize that the ORDER_ID is the long form shopify order id. It should be about 9 to 10 digits.
I have a very simple 'hello world' Mochiweb server (I just started my introduction into it), which takes a JSON request and sends it back:
'POST' ->
case Path of
"dummy" ->
Data = Req:parse_post(),
Json = proplists:get_value("json", Data),
Struct = mochijson2:decode(Json),
Action_value = struct:get_value(<<"action">>, Struct),
Action = list_to_atom(binary_to_list(A)),
Result = [got_json_request, Action],
DataOut = mochijson2:encode(Result),
Req:ok({"application/json",[],[Result]});
The thing is that when I try to make a request to it with cURL it fails:
curl -i -H "Content-Type: application/json" -H "Accept: application/json" -X POST -d '{"action":"dummy"}' http://localhost:8080/dummy
The Mochiweb log is quite difficult to read, but as I understand the error happens at this line:
Json = proplists:get_value("json", Data)
I have put a couple of io:formats in the code and found out that both Data and Json variables are [] after I make a request with cURL.
On the other hand, when I do a very simple request with cURL:
curl -d '{"action":"dummy"}' http://localhost:8080/dummy
both Data and Json are [{"{\"action\":\"dummy\"}",[]}], but in that case the line Struct = mochijson2:decode(Json) fails.
For some strange reason Mochiweb does not see the JSON data in the POST request in case the header has the "application/json" value.
So, the question is: How do I make a correct POST request with JSON data to a Mochiweb server?
EDIT: Json variable has the undefined value.
Try something along the lines of
Data = Req:recv_body(),
Json = mochijson2:decode(Data),
...
You should at least ensure method post and the content type ahead of this.
This is not about POST nor get.
It's about how you post your data to send to your server
When you send a json data to server, you need to make it as key=value
curl -d "key=value" "http://your.domain.com/path"
Therefore, if you want to post json as '{"action":"dummy"}', for GET request
curl -d "json='{\"action\":\"dummy\"}'" http://localhost:8080/dummy
For POST request as a file,
curl -F "json=#filename.json" http://localhost:8080/dummy
of course, when you send as a file, you need to read the posted file from the server side.