I am writing some code to interface with a service that I do not have yet, so I am writing a simulator to attempt to de-risk some of the problems I might run into when I get a chance to integrate against the real system. The interface is basically a REST style interface that returns JSON formatted strings.
The interface specification says the JSON formatted response is returned in lieu of the standard HTTP body. It also says that responses from the server will be zlib compressed and have the "Content-Encoding: gzip" set in the header.
So I created a WCF service that provides a REST interface that returns a JSON formatted string. I now need to deal with the compression portion of the equation. To satisfy the Content-Encoding: gzip requirements, do I simply gzip the JSON string I created and return that rather than the string? Or is it more involved than that? Let me know if there is any other information that is needed here, as I am still a newbie when dealing with REST/HTTP.
Thanks so Much for your time.
You're correct. Just Gzip the JSON string and return it.
Best reference for any REST implementation is the HTTP/1.1 RFC: https://www.rfc-editor.org/rfc/rfc2616
In short: yes, it's as simple as that. The response body just needs to be the gzip-compressed version of the normal response body.
This question may have some useful information for setting up your service.
Related
I have seen some posts but not getting any example for this use case, here is a link which I have referred to this. I have read some blogs also but all was in vain.
It is mostly the same as using JSON, except that instead of using a serialized JSON string you will send a binary array of bytes (meaning it's important your content-type is binary/octet-stream, not utf-8 or whatever).
This has examples on how to construct and access the byte array (make sure to select JavaScript or whichever language you're using): http://google.github.io/flatbuffers/flatbuffers_guide_tutorial.html
I'm a newbie both in SOAP and REST programming.
In SOAP the structure and the fields of the entities being exchanged
are always well and unambiguously documented in the WSDL.
I think the same does not happen in REST.
How is the developer of the REST API supposed to
document the structure and all the fields of the JSON objects, representing the entities being exchanged?
REST and SOAP can't be compared directly, as they are very different things. You can implement REST over SOAP if you want, but not the opposite. I recommend reading this answer.
In RESTful APIs you are supposed to document your media-types properly and let the interaction be driven by the underlying protocol, but don't look for that in most of the HTTP APIs that call themselves REST. They are using the term as a buzzword and often do the opposite of that, documenting the protocol instead of the media-types.
For instance, if you have an User resource, you are supposed to have a proper media-type for it in a format like JSON or XML that looks like application/vnd.mycompany.user.v1+json, and your documentation should describe what this JSON document looks like so the client knows what to expect. This doesn't have to be as strict as a WSDL. In fact, it can be a human readable documentation like you would do for documenting a class API or something like that.
The client can say what media-types he is prepared to accept with the Accept header. For instance, if a client wants the v1 representation in JSON, he can use something like:
GET /users/xyz
Accept: application/vnd.mycompany.user.v1+json
If he wants the v2 representation in XML, he can use something like:
GET /users/xyz
Accept: application/vnd.mycompany.user.v2+xml
And if he simply wants JSON and will let the server decide what to do, he can use a generic media-type and figure out what the server threw at him by checking the Content-Type response header.
GET /users/xyz
Accept: application/json
However, keep in mind that most of the so called REST APIs you'll find don't use custom media-types like this. Instead, they use the same generic media-types like application/json and application/xml everywhere. That means the clients have to identify resources through URI semantics and relations, and that's not REST at all. I'm not saying it's wrong, it's simple and works for many problems, but it's not REST and it doesn't solve the same kind of problems REST really intends to solve.
REST does not establish or recommend a way for you to structure your data. It's only meant to make it easy for anyone to figure out which methods and entities your API supports. You don't even necessarily need to output JSON in your REST API.
New Relic currently URL encodes the JSON payload of our webhooks. We correctly include the necessary header, however, I'm wondering if this is best practice - or if the JSON payload should be sent un-encoded (or even with some different encoding)?
It's more common to send JSON directly in the request body with a Content-Type header value of application/json. It's not entirely uncommon to do it the way New Relic does however. GitHub post-receive hook payloads are also encoded this way.
In some languages/frameworks it's easier to extract the JSON data from a parameter than from reading the entire request body since form parameters are automatically parsed out and easily referenced and in some cases reading a request body may involve some more elaborate stream parsing.
An example of that is PHP's syntax for extracting the entire raw request body:
$data = file_get_contents('php://input');
var_dump(json_decode($data));
Versus extracting from a form parameter:
$data = $_POST["payload"];
var_dump(json_decode($data));
For frameworks that easily expose the request body, this isn't as much an issue. Here's the example given from the GitHub docs for handling the hook with Sinatra:
post '/' do
push = JSON.parse(params[:payload])
"I got some JSON: #{push.inspect}"
end
If the JSON body were passed directly, this is what it would look like:
post '/' do
push = JSON.parse(request.body.read)
"I got some JSON: #{push.inspect}"
end
It's mostly just an aesthetic preference, there's no technical advantage to either. With the parameter, you do get the flexibility of sending multiple JSON payloads in a single request or the option to send additional meta data outside the JSON. I would suggest sending a list of JSON objects and including all the meta data in a top level wrapper if that's important to you. Something like this:
{
"notification_id": "akjfd8",
"events": [ ... ]
}
I personally prefer raw JSON bodies, but that's mostly because it makes debugging far easier on sites like RequestBin. Your screenshot does a great job of illustrating how unreadable JSON encoded into a form parameter is.
<rant>
Payloads should be the content type you say they are. If they are sending as application/x-www-form-urlencoded then that is what you should expect to find. The culture of using a Russian doll model of nested encodings is beyond stupid in my opinion.
If you want to send JSON send it as application/json. Is that really so hard?
</rant>
Recently I run into some weird issue with http header usage ( Adding multiple custom http request headers mystery) To avoid the problem at that time, I have put the fields into json string and add that json string into header instead of adding those fields into separate http headers.
For example, instead of
request.addHeader("UserName", mUserName);
request.addHeader("AuthToken", mAuthorizationToken);
request.addHeader("clientId","android_client");
I have created a json string and add it to the single header
String jsonStr="{\"UserName\":\"myname\",\"AuthToken\":\"123456\",\"clientId\":\"android_client\"}";
request.addHeader("JSonStr",jsonStr);
Since I am new to writing Rest and dealing with the Http stuff, I don't know if my usage is proper or not. I would appreciate some insight into this.
Some links
http://lists.w3.org/Archives/Public/ietf-http-wg/2011OctDec/0133.html
Yes, you may use JSON in HTTP headers, given some limitations.
According to the HTTP spec, your header field-body may only contain visible ASCII characters, tab, and space.
Since many JSON encoders (e.g. json_encode in PHP) will encode invisible or non-ASCII characters (e.g. "é" becomes "\u00e9"), you often don't need to worry about this.
Check the docs for your particular encoder or test it, though, because JSON strings technically allow most any Unicode character. For example, in JavaScript JSON.stringify() does not escape multibyte Unicode, by default. However, you can easily modify it to do so, e.g.
var charsToEncode = /[\u007f-\uffff]/g;
function http_header_safe_json(v) {
return JSON.stringify(v).replace(charsToEncode,
function(c) {
return '\\u'+('000'+c.charCodeAt(0).toString(16)).slice(-4);
}
);
}
Source
Alternatively, you can do as #rocketspacer suggested and base64-encode the JSON before inserting it into the header field (e.g. how JWT does it). This makes the JSON unreadable (by humans) in the header, but ensures that it will conform to the spec.
Worth noting, the original ARPA spec (RFC 822) has a special description of this exact use case, and the spirit of this echoes in later specs such as RFC 7230:
Certain field-bodies of headers may be interpreted according to an
internal syntax that some systems may wish to parse.
Also, RFC 822 and RFC 7230 explicitly give no length constraints:
HTTP does not place a predefined limit on the length of each header field or on the length of the header section as a whole, as described in Section 2.5.
Base64encode it before sending. Just like how JSON Web Token do it.
Here's a NodeJs Example:
const myJsonStr = JSON.stringify(myData);
const headerFriendlyStr = Buffer.from(myJsonStr, 'utf8').toString('base64');
res.addHeader('foo', headerFriendlyStr);
Decode it when you need reading:
const myBase64Str = req.headers['foo'];
const myJsonStr = Buffer.from(myBase64Str, 'base64').toString('utf8');
const myData = JSON.parse(myJsonStr);
Generally speaking you do not send data in the header for a REST API. If you need to send a lot of data it best to use an HTTP POST and send the data in the body of the request. But it looks like you are trying to pass credentials in the header, which some REST API's do use. Here is an example for passing the credentials in a REST API for a service called SMSIfied, which allows you to send SMS text message via the Internet. This example is using basic authentication, which is a a common technique for REST API's. But you will need to use SSL with this technique to make it secure. Here is an example on how to implement basic authentication with WCF and REST.
From what I understand using a json string in the header option is not as much of an abuse of usage as using http DELETE for http GET, thus there has even been proposal to use json in http header. Of course more thorough insights are still welcome and the accepted answer is still to be given.
In the design-first age the OpenAPI specification gives another perspective:
an HTTP header parameter may be an object,
e.g. object X-MyHeader is
{"role": "admin", "firstName": "Alex"}
however, within the HTTP request/response the value is serialized, e.g.
X-MyHeader: role=admin,firstName=Alex
The Sun Cloud API at http://kenai.com/projects/suncloudapis/pages/Home is a good example to follow for a RESTful API. True to RESTful principles, when you GET a resource you get no more nor less than a representation of that resource.
The Content-Type header in the response tells you exactly what the type of that resource is, for example application/vnd.com.sun.cloud.Snapshot+json. Sun has registered these mimetypes with the IANA.
How practical is this in general currently? Most API's I have seen have used the Content-Type of "application/json". This tells you that the response is JSON but nothing more about it. You have to have something in the JSON object, like a "type" property, to know what it is.
I'm designing a RESTful API (which will not be made public, therefore I wouldn't be registering mimetypes). I have been using RESTEasy and I find that even if I specify a complete mimetype, the Content-Type in the response header will be exactly what the Accept request header specified. If the request asks for "application/*+json" by default the response header will have "application/*+json". I can probably fix this by changing the header before the response goes out, but should I try to do that? Or should the response have a wildcard just like the request did?
Or should I just serve up "application/json" like most APIs seem to do?
Additional thoughts added later:
Another way of stating the question is: Should I use HTTP as the protocol, or should I use HTTP only as a transport mechanism to wrap my own protocol?
To use HTTP as the protocol, the entity body of the response contains the representation of the object requested (or the representation of an error message object), the "Content-Type" header contains the exact type of the object, and the "Status" header contains a success or error code.
To use HTTP as merely a transport mechanism, the "Status" header is always set to 200 OK, the "Content-Type" is something generic like "application/json", and the entity body contains something that itself has an object, an object type, an error code and whatever else you want. If your own protocol is RESTful, then the whole scheme is RESTful. (HTTP is a RESTful protocol but not the only possible one.)
Your own protocol will be opaque to all the transport layers. If you use HTTP as the protocol, all the transport layers will understand it and may do things you don't want; for instance a browser will intercept a "401 Unauthorized" response and put up a login dialog, even if you want to handle it yourself.
I use my own vnd.mycompany.mymediatype+xml media types for many of my representations. On the client I dispatch to the appropriate controller class based on the media type of the returned representation. This really allows the server to control the behavior of my client application in response to the user following a link.
Personally, I believe that using application/xml and application/json are one of the worst choices you can make if you hoping to support REST clients. The only exception to this is when the client only uses downloaded code (like Javascript) to interpret the data.
Or should I just serve up "application/json" like most APIs seem to do?
I don't think so.
A media type is the only point of coupling between your RESTful web application and the clients that use it. The documentation of your media types is the documentation of your API. Your media types are the contract between your clients and your application. Eliminate the specific media type and you eliminate an important element that makes REST workable.
Sun has registered these mimetypes with the IANA.
Couldn't find any mention of that here. AFAIK, there is no requirement to actually register your custom media type with the IANA. The convention seems to be to use the inverted domain notation of application/vnd.com.example.app.foo+json, which prevents namespace conflicts. If and when your media type becomes stable and public, it might be a good idea, but there's no requirement. Could be wrong on this, though.
Will you get any value by specifying a complete mimetype? Would you do anything with the complete mimetype different than you would if the mimetype was application/json?
My 2 cents- If the API is not going to be made public, then I see no reason for a complete mimetype. A mimetype of application/json should be more than enough. You already know the type of json that the response is returning. If the API eventually becomes public, then worry about a complete mimetype... or just let people figure it out.