I have seen some posts but not getting any example for this use case, here is a link which I have referred to this. I have read some blogs also but all was in vain.
It is mostly the same as using JSON, except that instead of using a serialized JSON string you will send a binary array of bytes (meaning it's important your content-type is binary/octet-stream, not utf-8 or whatever).
This has examples on how to construct and access the byte array (make sure to select JavaScript or whichever language you're using): http://google.github.io/flatbuffers/flatbuffers_guide_tutorial.html
Related
I have a situation where I have to write a api to create a resource and amongst datafields that I need to accept is a string that is basically contents of a html file. As I see it I have a choice between structuring the entire thing as a json object where this field is a string field with urlencoded html string , and having the Content Type as multipart/form-data where each of the fields and the html string (UTF-8 encoded) is a part in the message.
Not using json is something I am not comfortable with as I feel violating the REST standards in not structuring the content of the entity I am about to create thus there is a loss of information for the consumers as they can't tell immediately looking at my api definition about what data to feed to it. But practically multipart/form-data handles stuff like html file content better and more efficient as I will not have to urlencode it and can control the char-encoding also.
What will be a better approach in current context and upholding RESTful principles ? Also are there other trade-offs i should be aware of ? what about parsing a json with a huge string field (~ 200 Kb)embedded?
EDIT :- I was reading some similar questions on SO and one approach that stood out was the 2-step approach of making a first call with metadata to create the entity and then upload the file as an UPDATE process to the created entity wherein we use multipart/form-data. In that context, I guess , what I am asking is how sound is an approach where I send both metadata and the file in a single api call as multipart data , where each metadata field is actually a part in the multipart message as is the file.
The canonical way to upload files to REST API is using the multipart/form-data. As W3 recommendation guide says:
The content type "multipart/form-data" should be used for submitting
forms that contain files, non-ASCII data, and binary data.
Multipart/form-data has advantages over base64 to represent binary data. Is sticked to REST/Http philosophy, and simplify the develop of API clients.
Returning values from Forms: multipart/form-data
W3 Recommendation guide
The good practice is to use multipart/form-data whenever files are uploaded to the server along with database fields. Do not send a base64 JSON string as the request to your Rest API as it might corrupt the file or degrade the performance of your application.
As far as documenting multipart/form-data Rest API for your consumers is concerned you have to force your API consumers to use the same form fields which you have predefined in your web service.
Returning Values from Forms: multipart/form-data
I started using FormData objects everywhere on the client-side, in lieu of regular form input fields, for dynamic REST posts. FormData is presented in a positive light in various tutorials, so I went with it.
However, down the line, this caused me problems when decoding the form data into my Go structs. FormData objects are sent as "multipart/form-data" (regardless of files being sent) and I believe my decoder in Go didn't convert the raw data back to string form. Eventually my SQL queries were throwing panics, as hex data was being sent in where strings should have been.
So with some adjustment, I could use FormData however I've decided to revert to the simple universal recommendation: Use "multipart/form-data" only for special cases like when sending files. Otherwise, just use regular "application/x-www-form-urlencoded".
I'm brand new to Pentaho and I'm trying to do the following workflow:
read a bunch of lines out of a DB
do some transformations
POST them to a REST web service in JSON
I've got the first two figured out using an input step and the Json Output step.
However I have two problems doing the final step:
1) I can't get the JSON formatted how I want. It insists on doing {""=[{...}]} when I just want {...}. This isn't a big deal - I can work around this since I have control over the web service and I could relax the input requirements a bit. (Note: this page http://wiki.pentaho.com/display/EAI/JSON+output gives an example for the output I want by setting no. rows in a block=1 and an empty JSON block name, but it doesn't work as advertised.)
2) This is the critical one. I can't get the data to POST as JSON. It posts as key=value, where the key is the name I specify in the HTTP Post field name (on the 'Fields' tab) and the value is the encoded JSON. I just want to post the JSON as the request body. I've tried googling on this but can't find anyone else doing it, leading me to believe that I'm just approaching this wrong. Any pointers in the right direction?
Edit: I'm comfortable scripting (in Javascript or another language) but when I tried to use XmlHttpRequest in a custom javascript snippet I got an error that XmlHttpRequest is not defined.
Thanks!
This was trivial...just needed to use the REST Client (http://wiki.pentaho.com/display/EAI/Rest+Client) instead of the HTTP Post task. Somehow all my googling didn't discover that, so I'll leave this answer here in case someone else has the same problem as me.
You need to parse the JSON using a Modified JavaScript step. e.g. if the Output Value from the JSON Output is called result and its contents are {"data"=[{...}]}, you should call var plainJSON = JSON.stringify(JSON.parse(result).data[0]) to get the JSON.
In the HTTP Post step, the Request entity field should be plainJSON. Also, don't forget to add a header for Content-Type as application/json (you might have to add that as a constant)
I have a server and I need to upload files along with some fields from the client to the server. I have currently been using standard multipart/form-data.
I have found however that using multipart/form-data is not ideal. Objects on my server may have other objects nested within them, and thus are represented as a JSON object with other JSON objects embedded within.
I would like for the client to start making POST/PUT requests using a JSON representation exactly like it would expect in a GET request to the server, in a REST-ful manner. This way I don't have to flatten the fields which might be nested a couple layers within the JSON object in order to use multipart/form-data.
Problem is, JSON doesn't represent binary data. Multipart/form-data doesn't seem to have a way to represent fields nested within the values of other fields. But it does have much better handling of file-uploads.
I am at a loss for how to design this. Should I just have the client upload JSON with the fields encoded in base64, and take the 25% hit? Or should I have the JSON object being represented as some sort of "json" variable in a Multipart/form-data request, and have the binary files to be uploaded as another variable?
Should I just have the client upload JSON with the fields encoded in
base64, and take the 25% hit?
The hit will be 33% since 4/3=1.33.
Or should I have the JSON object being represented as some sort of
"json" variable in a Multipart/form-data request, and have the binary
files to be uploaded as another variable?
This should work.
You might also consider this approach: send all files using multipart, then get some identificators of files as a response. Put this identificators in your json and send it anyway you like. This approach might be beneficial if you have many scenarios in which you send files: you might always send them to the server with the same request, then get their identificators; after that do with them what you like.
Recently I run into some weird issue with http header usage ( Adding multiple custom http request headers mystery) To avoid the problem at that time, I have put the fields into json string and add that json string into header instead of adding those fields into separate http headers.
For example, instead of
request.addHeader("UserName", mUserName);
request.addHeader("AuthToken", mAuthorizationToken);
request.addHeader("clientId","android_client");
I have created a json string and add it to the single header
String jsonStr="{\"UserName\":\"myname\",\"AuthToken\":\"123456\",\"clientId\":\"android_client\"}";
request.addHeader("JSonStr",jsonStr);
Since I am new to writing Rest and dealing with the Http stuff, I don't know if my usage is proper or not. I would appreciate some insight into this.
Some links
http://lists.w3.org/Archives/Public/ietf-http-wg/2011OctDec/0133.html
Yes, you may use JSON in HTTP headers, given some limitations.
According to the HTTP spec, your header field-body may only contain visible ASCII characters, tab, and space.
Since many JSON encoders (e.g. json_encode in PHP) will encode invisible or non-ASCII characters (e.g. "é" becomes "\u00e9"), you often don't need to worry about this.
Check the docs for your particular encoder or test it, though, because JSON strings technically allow most any Unicode character. For example, in JavaScript JSON.stringify() does not escape multibyte Unicode, by default. However, you can easily modify it to do so, e.g.
var charsToEncode = /[\u007f-\uffff]/g;
function http_header_safe_json(v) {
return JSON.stringify(v).replace(charsToEncode,
function(c) {
return '\\u'+('000'+c.charCodeAt(0).toString(16)).slice(-4);
}
);
}
Source
Alternatively, you can do as #rocketspacer suggested and base64-encode the JSON before inserting it into the header field (e.g. how JWT does it). This makes the JSON unreadable (by humans) in the header, but ensures that it will conform to the spec.
Worth noting, the original ARPA spec (RFC 822) has a special description of this exact use case, and the spirit of this echoes in later specs such as RFC 7230:
Certain field-bodies of headers may be interpreted according to an
internal syntax that some systems may wish to parse.
Also, RFC 822 and RFC 7230 explicitly give no length constraints:
HTTP does not place a predefined limit on the length of each header field or on the length of the header section as a whole, as described in Section 2.5.
Base64encode it before sending. Just like how JSON Web Token do it.
Here's a NodeJs Example:
const myJsonStr = JSON.stringify(myData);
const headerFriendlyStr = Buffer.from(myJsonStr, 'utf8').toString('base64');
res.addHeader('foo', headerFriendlyStr);
Decode it when you need reading:
const myBase64Str = req.headers['foo'];
const myJsonStr = Buffer.from(myBase64Str, 'base64').toString('utf8');
const myData = JSON.parse(myJsonStr);
Generally speaking you do not send data in the header for a REST API. If you need to send a lot of data it best to use an HTTP POST and send the data in the body of the request. But it looks like you are trying to pass credentials in the header, which some REST API's do use. Here is an example for passing the credentials in a REST API for a service called SMSIfied, which allows you to send SMS text message via the Internet. This example is using basic authentication, which is a a common technique for REST API's. But you will need to use SSL with this technique to make it secure. Here is an example on how to implement basic authentication with WCF and REST.
From what I understand using a json string in the header option is not as much of an abuse of usage as using http DELETE for http GET, thus there has even been proposal to use json in http header. Of course more thorough insights are still welcome and the accepted answer is still to be given.
In the design-first age the OpenAPI specification gives another perspective:
an HTTP header parameter may be an object,
e.g. object X-MyHeader is
{"role": "admin", "firstName": "Alex"}
however, within the HTTP request/response the value is serialized, e.g.
X-MyHeader: role=admin,firstName=Alex
want to test my url service in browser but need to pass json format data.
I tried the below but no success:
http://locahost:8042/service/getinfo?body={"name":"H&M"}
or any tool that can be use to pass json formatted test data?
UPDATES1
I mean is to pass the json formatted data manually in browser's url address bar. Like as my example above. This is for quick testing only not for implementation.
Yes, you certainly can pass JSON in a URL Querystring. You just have to URLencode the JSON string first. As #dmn said, the data is probably better passed via POST because of GET's size restrictions.
The { and } characters are considered unsafe for unencoded use in a URL.
http://www.faqs.org/rfcs/rfc1738.html
Probably putting it in the GET won't be a good idea, since url parameters have a limit of 256 characters. It would be better if you use POST and put the JSON in the body and after that you can parse it using Jackson for example or gson ... and get the result as an object.
Yes you can Add Extension to chrome web-browser called 'Advance rest Client' and change contain header to application/json and make json object and post in pay load then hit send bottom it works for json object
enter image description here
This is how I am doing in my web service, I am passing a json and within the service I manage it.
Just do a URL query like this:
http://locahost:8042/service/getinfo?name=h%26m
%26 is the encoding of &