I'm developing an API which, given a request-body with certain contents, generates a document that can be in different formats (e.g. pdf, png, etc). Given that the document can be a binary file, the file is base64-encoded and placed within a json response that looks like this:
{
"document": "document base64-encoded"
}
The response might contain other fields but they are not relevant to the question.
My question is:
What is the best way to specify the content-type (e.g. pdf) and the content-encoding (e.g base64) of the field document?
I thought of some options but I would like to hear your recommendation.
My options
Use header parameters Content-Type and Content-Transfer-Encoding: I assume this option would be wrong as these parameters apply to the message itself which in this case is application/json without any encoding.
Specify custom-parameters (e.g. x-document-content-type and x-document-content-tranfer-encoding.
Include such metadata within the response-body like this:
{
"document": {
"content": "document base64-econded",
"type": "pdf",
"econding": "base64"
}
}
Thanks for your input!
There are 2 layers of communication here, each with their own contract/protocol. Your own JSON API protocol, being sent over the HTTP protocol. You should stick to HTTP standards to comply with the contract of that layer, and it is up to you to define the contract of the JSON data.
The mentioned HTTP headers are used to tell the user-agent/client what the MIME-type of the accompanying HTTP body is. This must be application/JSON to comply with the HTTP standard. If you diver from this it will not be valid HTTP traffic anymore and a regular, well configured HTTP client wouldn't be able to consume your API.
The JSON body inside the HTTP message can be filled with whatever you find is necessary for a consumer of your API to use the encoded contents. You are defining your own contract here, so it is really up to your needs and that of the consumers of your api.
Do you use other content encodings as base64? If so, then you can specify the encoding of the data.
Do you provide multiple different file types? Then it's useful to add the content-type of the encoded data. Be aware that the header of the encoded content might already contain information about the type of the contents of the file.
P.s. it would probably be better to provide the "type" as you name it, as a mime-type. This is more common as a known standard for defining content types.
Related
What HTTP header should I use for specifying stylistic properties of an HTTP response, specifically the case of keys in a JSON object? A note: I am developing a REST API where I would like clients to provide an expected case, for example camelCase or snake_case.
When camelCase is specified a response would look like:
{
"peopleUrl": "…",
"postsUrl": "…"
}
When snake_case is specified a response would look like:
{
"people_url": "…",
"posts_url": "…"
}
The headers I’ve been looking at are Expect and Prefer. Expect is described as mandatory configurations a server must comply with or error, and Prefer is described as optional configurations. The specific syntax I was looking at is:
Expect: case=camel
or:
Prefer: case=camel
Currently, I’m feeling like Expect is the best because it requires the configuration. However, while in RFC 2616 an extension mechanism for Expect is provided, however in RFC 7231 this extension mechanism was removed and only the 100-continue field is allowed (scroll to the next page to see the note providing the warrant). The Prefer header was specified in RFC 7240 and does appear to have an extension mechanism.
Which header is best for this configuration option? Am I looking in the wrong direction with Expect and Prefer?
As some of the comments indicate, there probably isn't an existing HTTP header name, defined in the HTTP RFCs, which would cover/handle your use case. Using Expect or Prefer, with well-defined semantics that don't quite cover your needs, may lead to more pain/frustration than you'd like; those headers might be handled specially by existing clients, servers, and proxies in ways that you do not want/expect.
That said, there is nothing preventing you from using your own custom X- header for requests and responses, e.g. X-JSON-KeyStyle:
GET /path/to/resource HTTP/1.1
Host: ...
X-JSON-KeyStyle: camel
...
And then your response would indicate the style used in the returned data:
HTTP1/1 200 OK
...
X-JSON-KeyStyle: camel
...
By specification, any header names prefaced with X- are deliberately not handled/covered, and are for uses such as this. With this approach, then, you would only need document your custom HTTP header, for use by clients of your REST API.
Hope this helps!
New Relic currently URL encodes the JSON payload of our webhooks. We correctly include the necessary header, however, I'm wondering if this is best practice - or if the JSON payload should be sent un-encoded (or even with some different encoding)?
It's more common to send JSON directly in the request body with a Content-Type header value of application/json. It's not entirely uncommon to do it the way New Relic does however. GitHub post-receive hook payloads are also encoded this way.
In some languages/frameworks it's easier to extract the JSON data from a parameter than from reading the entire request body since form parameters are automatically parsed out and easily referenced and in some cases reading a request body may involve some more elaborate stream parsing.
An example of that is PHP's syntax for extracting the entire raw request body:
$data = file_get_contents('php://input');
var_dump(json_decode($data));
Versus extracting from a form parameter:
$data = $_POST["payload"];
var_dump(json_decode($data));
For frameworks that easily expose the request body, this isn't as much an issue. Here's the example given from the GitHub docs for handling the hook with Sinatra:
post '/' do
push = JSON.parse(params[:payload])
"I got some JSON: #{push.inspect}"
end
If the JSON body were passed directly, this is what it would look like:
post '/' do
push = JSON.parse(request.body.read)
"I got some JSON: #{push.inspect}"
end
It's mostly just an aesthetic preference, there's no technical advantage to either. With the parameter, you do get the flexibility of sending multiple JSON payloads in a single request or the option to send additional meta data outside the JSON. I would suggest sending a list of JSON objects and including all the meta data in a top level wrapper if that's important to you. Something like this:
{
"notification_id": "akjfd8",
"events": [ ... ]
}
I personally prefer raw JSON bodies, but that's mostly because it makes debugging far easier on sites like RequestBin. Your screenshot does a great job of illustrating how unreadable JSON encoded into a form parameter is.
<rant>
Payloads should be the content type you say they are. If they are sending as application/x-www-form-urlencoded then that is what you should expect to find. The culture of using a Russian doll model of nested encodings is beyond stupid in my opinion.
If you want to send JSON send it as application/json. Is that really so hard?
</rant>
Recently I run into some weird issue with http header usage ( Adding multiple custom http request headers mystery) To avoid the problem at that time, I have put the fields into json string and add that json string into header instead of adding those fields into separate http headers.
For example, instead of
request.addHeader("UserName", mUserName);
request.addHeader("AuthToken", mAuthorizationToken);
request.addHeader("clientId","android_client");
I have created a json string and add it to the single header
String jsonStr="{\"UserName\":\"myname\",\"AuthToken\":\"123456\",\"clientId\":\"android_client\"}";
request.addHeader("JSonStr",jsonStr);
Since I am new to writing Rest and dealing with the Http stuff, I don't know if my usage is proper or not. I would appreciate some insight into this.
Some links
http://lists.w3.org/Archives/Public/ietf-http-wg/2011OctDec/0133.html
Yes, you may use JSON in HTTP headers, given some limitations.
According to the HTTP spec, your header field-body may only contain visible ASCII characters, tab, and space.
Since many JSON encoders (e.g. json_encode in PHP) will encode invisible or non-ASCII characters (e.g. "é" becomes "\u00e9"), you often don't need to worry about this.
Check the docs for your particular encoder or test it, though, because JSON strings technically allow most any Unicode character. For example, in JavaScript JSON.stringify() does not escape multibyte Unicode, by default. However, you can easily modify it to do so, e.g.
var charsToEncode = /[\u007f-\uffff]/g;
function http_header_safe_json(v) {
return JSON.stringify(v).replace(charsToEncode,
function(c) {
return '\\u'+('000'+c.charCodeAt(0).toString(16)).slice(-4);
}
);
}
Source
Alternatively, you can do as #rocketspacer suggested and base64-encode the JSON before inserting it into the header field (e.g. how JWT does it). This makes the JSON unreadable (by humans) in the header, but ensures that it will conform to the spec.
Worth noting, the original ARPA spec (RFC 822) has a special description of this exact use case, and the spirit of this echoes in later specs such as RFC 7230:
Certain field-bodies of headers may be interpreted according to an
internal syntax that some systems may wish to parse.
Also, RFC 822 and RFC 7230 explicitly give no length constraints:
HTTP does not place a predefined limit on the length of each header field or on the length of the header section as a whole, as described in Section 2.5.
Base64encode it before sending. Just like how JSON Web Token do it.
Here's a NodeJs Example:
const myJsonStr = JSON.stringify(myData);
const headerFriendlyStr = Buffer.from(myJsonStr, 'utf8').toString('base64');
res.addHeader('foo', headerFriendlyStr);
Decode it when you need reading:
const myBase64Str = req.headers['foo'];
const myJsonStr = Buffer.from(myBase64Str, 'base64').toString('utf8');
const myData = JSON.parse(myJsonStr);
Generally speaking you do not send data in the header for a REST API. If you need to send a lot of data it best to use an HTTP POST and send the data in the body of the request. But it looks like you are trying to pass credentials in the header, which some REST API's do use. Here is an example for passing the credentials in a REST API for a service called SMSIfied, which allows you to send SMS text message via the Internet. This example is using basic authentication, which is a a common technique for REST API's. But you will need to use SSL with this technique to make it secure. Here is an example on how to implement basic authentication with WCF and REST.
From what I understand using a json string in the header option is not as much of an abuse of usage as using http DELETE for http GET, thus there has even been proposal to use json in http header. Of course more thorough insights are still welcome and the accepted answer is still to be given.
In the design-first age the OpenAPI specification gives another perspective:
an HTTP header parameter may be an object,
e.g. object X-MyHeader is
{"role": "admin", "firstName": "Alex"}
however, within the HTTP request/response the value is serialized, e.g.
X-MyHeader: role=admin,firstName=Alex
How to structure an API where the same data may request in different format, in a RESTful format. For example.
GET /person/<id> //get the details of resource <id>
Now depending on the client (browser) requirement, the data may send as html (say normal rendering) or Json (say ajax call). So my doubts are
Can I keep the same url for both requests, or should keep them seperate?
How to detect whether the request is for html/Json at the server. The request type is same (GET). So which parameter should I consider.
How to detect the difference in data type at client (html/Json)\
thanks,
bsr.
Similar question: REST Content-Type: Should it be based on extension or Accept header?
The accepted answers has great points.
Can I keep the same url for both requests, or should keep them seperate?
Yes, keep them the same. Its the same resource, you're just asking for different representations of it.
How to detect whether the request is for html/Json at the server. The request type is same (GET). So which parameter should I consider.
You can use the Accept header to specify the return content-type.
How to detect the difference in data type at client (html/Json)\
You would look at the "Content-Type" header.
What about adding a variable for output type?
The Sun Cloud API at http://kenai.com/projects/suncloudapis/pages/Home is a good example to follow for a RESTful API. True to RESTful principles, when you GET a resource you get no more nor less than a representation of that resource.
The Content-Type header in the response tells you exactly what the type of that resource is, for example application/vnd.com.sun.cloud.Snapshot+json. Sun has registered these mimetypes with the IANA.
How practical is this in general currently? Most API's I have seen have used the Content-Type of "application/json". This tells you that the response is JSON but nothing more about it. You have to have something in the JSON object, like a "type" property, to know what it is.
I'm designing a RESTful API (which will not be made public, therefore I wouldn't be registering mimetypes). I have been using RESTEasy and I find that even if I specify a complete mimetype, the Content-Type in the response header will be exactly what the Accept request header specified. If the request asks for "application/*+json" by default the response header will have "application/*+json". I can probably fix this by changing the header before the response goes out, but should I try to do that? Or should the response have a wildcard just like the request did?
Or should I just serve up "application/json" like most APIs seem to do?
Additional thoughts added later:
Another way of stating the question is: Should I use HTTP as the protocol, or should I use HTTP only as a transport mechanism to wrap my own protocol?
To use HTTP as the protocol, the entity body of the response contains the representation of the object requested (or the representation of an error message object), the "Content-Type" header contains the exact type of the object, and the "Status" header contains a success or error code.
To use HTTP as merely a transport mechanism, the "Status" header is always set to 200 OK, the "Content-Type" is something generic like "application/json", and the entity body contains something that itself has an object, an object type, an error code and whatever else you want. If your own protocol is RESTful, then the whole scheme is RESTful. (HTTP is a RESTful protocol but not the only possible one.)
Your own protocol will be opaque to all the transport layers. If you use HTTP as the protocol, all the transport layers will understand it and may do things you don't want; for instance a browser will intercept a "401 Unauthorized" response and put up a login dialog, even if you want to handle it yourself.
I use my own vnd.mycompany.mymediatype+xml media types for many of my representations. On the client I dispatch to the appropriate controller class based on the media type of the returned representation. This really allows the server to control the behavior of my client application in response to the user following a link.
Personally, I believe that using application/xml and application/json are one of the worst choices you can make if you hoping to support REST clients. The only exception to this is when the client only uses downloaded code (like Javascript) to interpret the data.
Or should I just serve up "application/json" like most APIs seem to do?
I don't think so.
A media type is the only point of coupling between your RESTful web application and the clients that use it. The documentation of your media types is the documentation of your API. Your media types are the contract between your clients and your application. Eliminate the specific media type and you eliminate an important element that makes REST workable.
Sun has registered these mimetypes with the IANA.
Couldn't find any mention of that here. AFAIK, there is no requirement to actually register your custom media type with the IANA. The convention seems to be to use the inverted domain notation of application/vnd.com.example.app.foo+json, which prevents namespace conflicts. If and when your media type becomes stable and public, it might be a good idea, but there's no requirement. Could be wrong on this, though.
Will you get any value by specifying a complete mimetype? Would you do anything with the complete mimetype different than you would if the mimetype was application/json?
My 2 cents- If the API is not going to be made public, then I see no reason for a complete mimetype. A mimetype of application/json should be more than enough. You already know the type of json that the response is returning. If the API eventually becomes public, then worry about a complete mimetype... or just let people figure it out.