Background
I am making a spreadsheet that refers to information security license registration provided by the Japanese government (https://riss.ipa.go.jp/). The spreadsheet will be used on Microsoft Excel/LibreOffice Calc on Windows/Linux, so I want to avoid using platform-specific functionality like a script with the XMLHTTP60 module.
The site https://riss.ipa.go.jp has a URI that can retrieve registration information with a registration number (https://riss.ipa.go.jp/ajax/findRissRequest). The URI only works with a POST request with the application/x-www-form-urlencoded style request body and doesn't work with a GET request. The response of the URI is JSON format.
Problem #1
Microsoft Excel and LibreOffice Calc have the WEBSERVICE function that can be used to send a request to a URI. This function is supported on all platforms and is suitable for my use case.
Unfortunately, the WEBSERVICE function only supports GET requests, and the URI I want to use only supports POST requests.
Problem #2
Microsoft Excel and LibreOffice Calc have the FILTERXML function that can be used to extract a specific element from XML.
Unfortunately, the URI I want to use returns response in JSON format. There are no functions to parse JSON in Microsoft Excel and LibreOffice Calc.
Question
Is there any way to convert GET request to POST request and extract a JSON property?
For example, is there any Web API like http://api.example.com/convert/get-to-post?uri=https://riss.ipa.go.jp/ajax/findRissRequest®_no=000006&property=result.reg_date that calls https://riss.ipa.go.jp/ajax/findRissRequest with POST request body reg_no=000006 and extract property result.reg_date from its response?
After all, I could not find any existing services. So I made a web API service with AWS Lambda and API Gateway.
First, I made a Lambda function like this:
import json
import urllib.request
import urllib.parse
def lambda_handler(event, context):
queryStringParameters = event.get('params').get('querystring')
data = urllib.parse.urlencode(queryStringParameters)
data = data.encode('UTF-8')
f = urllib.request.urlopen("https://riss.ipa.go.jp/ajax/findRissRequest", data)
j = json.loads(f.read().decode('utf-8'))
return j
Then I made a resource with a GET method in API Gateway and connect it with the Lambda function.
In Integration Request, you have to use non-proxy integration. Also, you have to specify a mapping template for Content-Type application/json with Method Request passthrough template.
In Integration Response, you have to specify a mapping template for Content-Type application/xml like this:
<?xml version="1.0" encoding="UTF-8" ?>
#set($root = $input.path('$.result[0]'))
<result>
#foreach($key in $root.keySet())
<$key>$root.get($key)</$key>
#end
</result>
Then I added the HEAD and OPTIONS method for the resource. It is because the WEBSERVICE function of LibreOffice sends OPTIONS and HEAD requests before a GET request.
You can use a mock in Integration Request with a mapping template for Content-Type application/json like { "statusCode": 200 }.
The result of WEBSERVICE function will be #VALUE! without these methods.
Finally, I can get a property from a web service that only accepts POST requests and returns a JSON with WEBSERVICE and FILTERXML like:
=FILTERXML(WEBSERVICE("https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/prod/passthru?reg_no=000006"),"//result/reg_date")
Related
I am developing a library for working with various types of cloud based queue services.
One of those services is the Azure Queue Storage REST API.
For the Amazon SQS service I can send an Accept: application/json header and the response is in JSON format.
Since JSON is a format that is supported by many APIs and XML is not fun to work with, I would prefer the Azure Storage REST API to also return a response in JSON format.
I have tried to set the Accept: application/json header to no avail. The responses are all in XML format with Content-Type: application/xml, which is obviously not what I was asking for.
Currently all code is in C with dependencies on a couple of libraries, including cURL and jansson, although for this question that doesn't really matter. It's just that I would like the library to be as simple and lightweight as possible.
I have a hard time digging through all kinds of documentation. Most topics I can find are about sending JSON within a message. But that's not what I'm going for.
Is it even possible to receive the actual responses in JSON? I would really like to drop my libxml2 dependency.
As pointed by #Tom Because the documentation is stating that it only return XML, I would personally write an azure function who becomes an adaptor which basically takes your request, sends it to azure queue storage, retrieves the xml response and then converts the xml response to json and return the json to the caller (which will be your C code).
A sample python code to convert xml to json is shown below:
import xmltodict
import json
text = ''
xpars = xmltodict.parse(text)
json = json.dumps(xpars)
print(json)
A sample xml message
text = '<QueueMessagesList>
<QueueMessage>
<MessageId>string-message-id</MessageId>
<InsertionTime>insertion-time</InsertionTime>
<ExpirationTime>expiration-time</ExpirationTime>
<PopReceipt>opaque-string-receipt-data</PopReceipt>
<TimeNextVisible>time-next-visible</TimeNextVisible>
<DequeueCount>integer</DequeueCount>
<MessageText>message-body</MessageText>
</QueueMessage>
</QueueMessagesList>'
And the response will be :
{
"QueueMessagesList": {
"QueueMessage": {
"MessageId": "string-message-id",
"InsertionTime": "insertion-time",
"ExpirationTime": "expiration-time",
"PopReceipt": "opaque-string-receipt-data",
"TimeNextVisible": "time-next-visible",
"DequeueCount": "integer",
"MessageText": "message-body"
}
}
}
Please Note: This whole thing can also be done using a Logic App in azure.
I have only shown the XML to JSON converter part here, but it is really straightforward to write an HTTP Trigger Azure Function to do the same. Or you can even write this converter into your C code as well.
Hope this helps you in moving on with your library development.
I'm exploring Bitso API (Bitso is a mexican crypto exchange).
The docs of the API is well explained at some languages such as Python and Ruby for its use. The problem here is that there are no examples using straight URLs for request.
What I'm planning to do is to create the URL that the code is creating on its requests function.
There is a request for balance account, that is the data I'd like to get.
According documentation, this is a private request that need some headers at the request (Key, nonce and signature), you can take a look here.
The code to make this request in Python is the following one:
import time
import hmac
import hashlib
import requests
bitso_key = "BITSO_KEY"
bitso_secret = "BITSO_SECRET"
nonce = str(int(round(time.time() * 1000)))
http_method = "GET"
request_path = "/v3/balance/"
json_payload = ""
# Create signature
message = nonce+http_method+request_path+json_payload
signature = hmac.new(bitso_secret.encode('utf-8'),
message.encode('utf-8'),
hashlib.sha256).hexdigest()
# Build the auth header
auth_header = 'Bitso %s:%s:%s' % (bitso_key, nonce, signature)
# Send request
response = requests.get("https://api.bitso.com/v3/balance/", headers={"Authorization": auth_header})
print(response.content)
So based in this I could say that the URL is something like this:
https://api.bitso.com/v3/balance/Bitso%20<Key>:<nonce>:<signature>
I'm sure that I'm wrong with that supposition, I understand that headers={"Authorization": auth_header} seems to be a JSON object used as header in the URL, but I'd like to know how that JSON object is translated at the URL to make a request. I'd like to copy-paste that URL at the browser and get the data as response.
I need that URL so I could use it to connect the service to a Business Intelligence tool.
Thanks!
According to the documentation this Authorization is a header in the request. You can try using postman but you still need hash the destination URL with your api key and the nonce to avoid replay attacks.
I need the following URL to return JSON:
https://mysite/_vti_bin/listdata.svc/Pages
I've tried added the following to the URL (per a blog online but didn't work):
https://mysite/_vti_bin/listdata.svc/Pages?format=JSON
It always returns XML. Also, I'm not calling the web service via C# or JS code. I'm using a rapid app dev platform that has built-in API consumption. All I need is JSON to be returned (given a URL structure) and I got what I need. Thanks.
By default SharePoint 2010 REST service returns results in XML format. To get the results in JSON format, include an Accept header set to application/json;odata=verbose.
Example
url: http://site url/_vti_bin/listdata.svc/ListName,
method: GET
Headers:
accept: "application/json;odata=verbose"
Result
above code from the website_mail module controller file email_designer.py file
class WebsiteEmailDesigner(http.Controller):
#http.route('/website_mail/email_designer/<model("email.template"):template>/', type='http', auth="user", website=True, multilang=True)
def index(self, template, **kw):
values = {
'template': template,
}
return request.website.render("website_mail.designer_index", values)
#http.route(['/website_mail/snippets'], type='json', auth="user", website=True)
def snippets(self):
return request.website._render('website_mail.email_designer_snippets')
which situation we are using type="json" and type="http" and why..??
Basically type="json" is used to pass data from controller where as type="html" is for responding over http request.
For example from your above code:
the url "/website_mail/email_designer//" will respond towards any particular http request and route to its web page where as the url "/website_mail/snippets" will just pass json data to its rendered template but there is no physical webpage related to this url.
Methods that received JSON can be defined by passing 'json' to the type argument of http.route(). The OpenERP Javascript client can contact these methods using the JSON-RPC protocol. JSON methods must return JSON. Like the HTTP methods they receive arguments as named parameters (except these arguments are JSON-RPC parameters).
#http.route('/division', type="json")
def division(self, i, j):
return i / j # returns a number
Both of them are about communication between client and server. HttpRequest communicates trough the well known GET and POST methods. That means the following:
The client send a request encoded in the url (GET method) or in the http body (POST method)
The server returns an object corresponding to the request. Could be an html page, PNG image, CSS file, JavaScript, XML encoded data or whatever.
JsonRequest is an implementation of another protocol for client/server communication - JSON-RPC 2.0. You may want lo took here form more information. It's a remote procedure call (RPC) protocol which means that it allows the client to initiate the execution of some method on the server passing some arguments to this method. In response the client gets some data as a result of the method invocation.
EDIT - some more words about the decorators #openerpweb.jsonrequest and #openerpweb.httprequest
Some methods are decorated with the #openerpweb.jsonrequest decorator, other methods - with the #openerpweb.httprequest. This means nothing else but that the first group of methods will be available for execution trough the JSON RPC protocol and the second group will be accessible trough the pure HTTP protocol.
Now, what is the difference? I do we need both jsonrequest and httprequest? Let simplify it like this: JSON is more suitable for executing methods on the server and obtain results. HTTP is simpler and easier to use when all we what is to access some resource on the server.
Let's 'decorate' this with some examples for clarity. Take a look at the following method of the web.controllers.main.Export class:
#openerpweb.jsonrequest
def formats(self, req):
""" Returns all valid export formats
:returns: for each export format, a pair of identifier and printable name
:rtype: [(str, str)]
"""
...
This method accepts some arguments and returns a list (Python list object) containing all known export formats. It will be called in a programmatic way in some python code on the client side.
On the other side are the 'http' methods - like the method css() of the web.controllers.main.Web class:
#openerpweb.httprequest
def css(self, req, mods=None):
....
All this method does is to return a CSS file to the client. It's a simple action like accessing an image, a HTML web page or whatever other resource on the server. The resource we are returning here is nothing complicated as a Python list as in the previous example. We don't need a special format to encode it additionally. So we don't need additional data encoding format as JSON and remote procedure call protocol as JSON RPC.
type="json":
it will call JSONRPC as an argument to http.route() so here , there will be only JSON data be able to pass via JSONRPC, It will only accept json data object as argument.
type="http":
As compred to JSON, http will pass http request arguments to http.route() not json data.
Examples
#http.route('demo_html', type="http") // Work Pefrect when I call this URL
def some_html(self):
return "<h1>This is a test</h1>"
#http.route('demo_json', type="json") // Not working when I call this URL
def some_json(self):
return {"sample_dictionary": "This is a sample JSON dictionary"}
I was checking the files in the controllers of web module in both OpenERP-7.0 and OpenERP-6.1. Then I found that 6.1 uses jsonrequest (#openerpweb.jsonrequest) 7.0 uses httprequest (#openerpweb.httprequest). What is the difference between the two ?
I didn't look at OpenERP v7 but OpenERP v6.1 uses both - HttpRequest and JsonRequest. I suppose it's the same for OpenERP v7...
Both of them are about communication between client and server. HttpRequest communicates trough the well known GET and POST methods. That means the following:
The client send a request encoded in the url (GET method) or in the http body (POST method)
The server returns an object corresponding to the request. Could be an html page, PNG image, CSS file, JavaScript, XML encoded data or whatever.
JsonRequest is an implementation of another protocol for client/server communication - JSON-RPC 2.0. You may want lo look here for more information. It's a remote procedure call (RPC) protocol which means that it allows the client to initiate the execution of some method on the server passing some arguments to this method. In response the client gets some data as a result of the method invocation.
EDIT - some more words about the decorators #openerpweb.jsonrequest and #openerpweb.httprequest
Some methods are decorated with the #openerpweb.jsonrequest decorator, other methods - with the #openerpweb.httprequest. This means nothing else but that the first group of methods will be available for execution trough the JSON RPC protocol and the second group will be accessible trough the pure HTTP protocol.
Now, what is the difference? And do we need both jsonrequest and httprequest? Let simplify it like this: JSON is more suitable for executing methods on the server and obtain results. HTTP is simpler and easier to use when all we what is to access some resource on the server.
Let's 'decorate' this with some examples for clarity. Take a look at the following method of the web.controllers.main.Export class:
#openerpweb.jsonrequest
def formats(self, req):
""" Returns all valid export formats
:returns: for each export format, a pair of identifier and printable name
:rtype: [(str, str)]
"""
...
This method accepts some arguments and returns a list (Python list object) containing all known export formats. It will be called in a programmatic way in some python code on the client side.
On the other side are the 'http' methods - like the method css() of the web.controllers.main.Web class:
#openerpweb.httprequest
def css(self, req, mods=None):
....
All this method does is to return a CSS file to the client. It's a simple action like accessing an image, a HTML web page or whatever other resource on the server. The resource we are returning here is nothing complicated as a Python list as in the previous example. We don't need a special format to encode it additionally. So we don't need additional data encoding format as JSON and remote procedure call protocol as JSON RPC.