We're having a bit of a discussion on the subject of posting data to a REST endpoint. Since the objects are quite complex, the easiest solution is to simply serialize them as JSON and send this in the request body.
Now the question is this: Is this kosher? Or should the JSON be set as a form parameter like data=[JSON]? Or is sending of JSON in the request body just frowned upon for forcing the clients using the application, to send their data via JavaScript instead of letting the browser package it up as application/x-www-form-urlencoded?
I know all three options work. But which are OK? Or at least recommended?
I'd say that both methods will work well
it's important that you stay consistent across your APIs. The option I would personally choose is simply sending the content as application/json.
POST doesn't force you to use application/x-www-form-urlencoded - it's simply something that's used a lot because it's what webbrowsers use.
There is nothing wrong about sending it directly as serialized JSON, for example google does this by default in it's volley library (which obviously is their recommended REST library for android).
If fact, there are plenty of questions on SO about how not to use JSON, but rather perform "normal" POST requests with volley. Which is a bit counter intuitive for beginners, having to overwrite it's base class' getParams() method.
But google having it's own REST library doing this by default, would be my indicator that it is OK.
You can use JSON as part of the request data as the OP had stated all three options work.
The OP needs to support JSON input as it had to support contain complex structural content. However, think of it this way... are you making a request to do something or are you just sending what is basically document data and you just happen to use the POST operation as the equivalent of create new entry.
That being the case, what you have is basically a resource endpoint with CRUDL semantics. Following up on that you're actually not limited to application/json but any type that the resource endpoint is supposed to handle.
For non-resource endpoints
I find that (specifically for JAX-RS) the application/x-www-urlencoded one is better.
Consistency with OAuth 2.0 and OpenID Connect, they use application/x-www-urlencoded.
Easier to annotate the individual fields using Swagger Annotations
Swagger provides more defaults.
Postman generates a nice form for you to fill out and makes things easier to test.
Examples of non-resource endpoints:
Authentication
Authorization
Simple Search (though I would use GET on this one)
Non-simple search where there are many criteria
Sending a message/document (though I would also consider multipart/form-data so I can pass meta data along with the content, but JAX-RS does not have a standard for this one Jersey and RestEasy have their own implementations)
Related
I am trying to stick to the Restful design pattern for both JSON and HTML. My issue is the design for creating a new resource (amongst others, but this is the gist of the issue). IE:
JSON – POST to /resource creates a new resource.
JSON – GET to /resource returns a list of resources.
JSON – GET to /resource/{id} returns a resource.
HTML – POST to /resource creates a new resource.
HTML – GET to /resource returns a list of resources.
HTML – GET to /resource/{id} returns a resource.
All good so far – but I need a HTML form to actually create the data to send to the HTML POST. Obviously POST and GET already do things. I could use one of the below to return the HTML form:
HTML – GET to /resource?CREATE
HTML - GET to /resource?action=CREATE
HTML – GET to /resources/CREATE
But they seem like a kludge and not that intuitive.
Any thoughts or ideas?
EDIT - See my answer to my question below. At present this is (I consider) the best option.
I would indeed use something like /resources/create. If you want to allow for non-numeric identifiers, then this will not work. In that case you can identify a resource with a prefix, such as /resources/resource-{id} and then you can still use /resources/create.
I found this blog post really helpful to make URI scheme decisions: http://blog.2partsmagic.com/restful-uri-design/
In fact, you should leverage content negotiation (CONNEG) when you want to handle several formats within RESTful services.
I mean:
Set the Content-Type header to specify the type of sent data
Set the Accept header to specify the type of data you want to receive
The server resources should leverage these hints to make the appropriate data conversion.
In the case of JSON, the content type would be obviously application/json. For HTML form, you should leverage the content type application/x-www-form-urlencoded (or multipart/form-data if you want to upload files as well). See the specification for more details.
Otherwise, you shouldn't use action in URL since it's not really RESTful. The HTTP verb should determine the action to do on the resource. I mean, to create a resource, the POST method should be used. The GET method aims to retrieve the state of a resource.
For more details, you could have a look at this blog post:
Designing a Web API (i.e. RESTful service).
I have an answer. I'll use standard RESTful POST from a HTML page, but when I have no form parameters sent and my accept header is text/html, I'll send a HTML form to the requestor. Keeps RESTful URI design and allows a clean HTML form + process (2 step).
HTML - POST - /resources (with no form attributes) generates a HTML form
HTML - POST - /resources (with form attributes) adds a resource
JSON - POST - /resources (with form attributes) adds a resource
OK, it's not "strictly" RESTful as I'm POSTing but not creating a new resource so in theory I should use a GET for that, but it's the best of a mismatched design.
If anyone can provide a better solution, I'm still all ears :-)
I'd rather add and endpoint called /templates/ that returns a template/form/whatever you need for given action. It also seems that the server should be unaware of such form existence. It can accept or reject a request and it's client job to submit it in an appropriate format.
I guess that you mix processing the view with preparing RESTful endpoints. The backend site should be completely unaware of the fact that some sort of view/form is required. It's client job to prepare such form.
We are using the FOS Rest bundle, and, at first, we were unaware that the body listener was active.
In the meanwhile, we created a lot of resources that are receiving data via the body, in JSON format. For now we only support JSON format. Lot of them now in the controllers are retrieving parameters from the Symfony ParameterBag, because, body listener was injecting them there.
So, do you think is a good practice to leave to body listener the responsibility of that, and in the controllers retrieve parameters via the parameter bag? This way, no matter via GET, POST or body, all the parameters arrive to the controllers via parameter bag.
We are looking at this because some of our API clients were making requests without providing the content-type in the header, and because of that, body listener didn't inject the body in the parameter bag. So in the controllers we didn't have the parameters available.
Thanks in advance!
Generally speaking, one of the most important aspects of API design should be consistency across your implementation, so that your consumers understand how to interact with your system. To that end, I would follow this pattern:
Keep the body listener enabled if it assists with your development; this listener is a shortcut to help you write simpler controllers, so if it is working for you, there's no reason to abandon it
For API endpoints that expect a JSON request body, report to the consumer that they are making an illegal request if the proper Content-type: application/json header is missing (which can be done via setting a _format requirement on the route; see advanced routing example)
If you must support these legacy users that are not passing the header, you could create a listener on the kernel.request event with a very high priority which simply checks for the detected mime type and, and if it is missing, does something like this:
if ($request->getRequestFormat() != 'json') {
json_decode($request->getContent());
if (json_last_error() == JSON_ERROR_NONE) {
$request->setRequestFormat('json');
}
}
(The use of json_last_error() is not perfect here; see this question for a more detailed discussion.)
Finally, if you aren't already, make sure that you are restricting your API controllers to the proper request method, again to make sure that they are being consumed correctly and to make your development and use more consistent. If you are using annotations, for example, this is as simple as using the FOSRestBundle-provided #Get, #Post, etc. in place of the usual #Route. If you are using YML or XML configuration, it's the methods property.
Although it is certainly possible for a single controller to accept data via a GET (with request parameters), a POST (via URL-encoded form data), or a POST or PUT with JSON data and a body listener-conversion, this makes for a very convoluted API, as you have to start maintaining these different cases, and I would argue that things become very difficult for your users as well.
As a consumer of these APIs, I would expect my requests to fail if I were passing JSON in a request body but not including the proper header, and I would expect my requests to fail if I were passing x-www-form-urlencode data to a JSON-based API. Overall, both for the maintainer of an API and for the consumers, it is a good idea to be strict on what you accept and to follow the same pattern throughout.
I'm a newbie both in SOAP and REST programming.
In SOAP the structure and the fields of the entities being exchanged
are always well and unambiguously documented in the WSDL.
I think the same does not happen in REST.
How is the developer of the REST API supposed to
document the structure and all the fields of the JSON objects, representing the entities being exchanged?
REST and SOAP can't be compared directly, as they are very different things. You can implement REST over SOAP if you want, but not the opposite. I recommend reading this answer.
In RESTful APIs you are supposed to document your media-types properly and let the interaction be driven by the underlying protocol, but don't look for that in most of the HTTP APIs that call themselves REST. They are using the term as a buzzword and often do the opposite of that, documenting the protocol instead of the media-types.
For instance, if you have an User resource, you are supposed to have a proper media-type for it in a format like JSON or XML that looks like application/vnd.mycompany.user.v1+json, and your documentation should describe what this JSON document looks like so the client knows what to expect. This doesn't have to be as strict as a WSDL. In fact, it can be a human readable documentation like you would do for documenting a class API or something like that.
The client can say what media-types he is prepared to accept with the Accept header. For instance, if a client wants the v1 representation in JSON, he can use something like:
GET /users/xyz
Accept: application/vnd.mycompany.user.v1+json
If he wants the v2 representation in XML, he can use something like:
GET /users/xyz
Accept: application/vnd.mycompany.user.v2+xml
And if he simply wants JSON and will let the server decide what to do, he can use a generic media-type and figure out what the server threw at him by checking the Content-Type response header.
GET /users/xyz
Accept: application/json
However, keep in mind that most of the so called REST APIs you'll find don't use custom media-types like this. Instead, they use the same generic media-types like application/json and application/xml everywhere. That means the clients have to identify resources through URI semantics and relations, and that's not REST at all. I'm not saying it's wrong, it's simple and works for many problems, but it's not REST and it doesn't solve the same kind of problems REST really intends to solve.
REST does not establish or recommend a way for you to structure your data. It's only meant to make it easy for anyone to figure out which methods and entities your API supports. You don't even necessarily need to output JSON in your REST API.
I need to add JSON format parameters to request payload to do a POST request in restful service testing. How can I do that in SOAP UI?
You did not specify which version of SoapUI you are using. In version 4.x of SoapUI, they made the same assumption as what FrAn answered: you generally do not want to include a payload for a GET request. In later versions of SoapUI, they corrected this and you are able to do it.
Once you change the method type to POST, on the individual method call, you will see another panel where you are able to define the body. You can see this in the documentation. You will have to write out the entire body manually.
For REST services, the payload is not part of the WADL - which SoapUI uses internally to store the entire definition. You can create a sample Request in your REST service to make creating test cases easier. You can see this in the documentation.
Lastly, some additional information is available in the API Dojo.
HTTP GET request shouldn't contain payload. While you can do that, insofar as it isn't explicitly precluded by the HTTP specification, I would suggest avoiding it simply because people don't expect things to work that way.
In regards to this Haacked blog, I'm hesitant to implement the proposed anti-JSON GET hijacking solutions since
The recommended solutions to mitigating JSON hijacking involve non-REST-full JSON POSTs to GET data
The alternate solution (object wrapping) causes problems with 3rd party controls I don't have source-code access to.
I can't find a community-vetted implementation that implements the Alternative Solution (listed below) on how to compose the security token, or securely deliver it within the webpage. I also won't claim to be enough of an expert to roll my own implementation.
Referrer headers can't be relied upon
Background
This blog describes a CSRF issue regarding JSON Hijacking and recommends using JSON POSTs to GET data. Since using a HTTP POST to GET data isn't very REST-full, I'd looking for a more RESTfull solution that enables REST actions per session, or per page.
Another mitigation technique is to wrap JSON data in an object as described here. I'm afraid this may just delay the issue, until another technique is found.
Alternative Implementation
To me, it seems natural to extend the use ASP.NET MVC's AntiForgeryToken with jQuery HTTP GETs for my JSON.
For example if I GET some sensitive data, according to the Haacked link above, the following code is vulnerable:
$.getJSON('[url]', { [parameters] }, function(json) {
// callback function code
});
I agree that it isn't RESTfull to GET data using the recommended POST workaround. My thought is to send a validation token in the URL. That way the CSRF-style attacker won't know the complete URL. Cached, or not cached, they won't be able to get the data.
Below are two examples of how a JSON GET query could be done. I'm not sure what implementation is most effective, but may guess that the first one is safer from errant proxies caching this data, thus making it vulnerable to an attacker.
http://localhost:54607/Home/AdminBalances/ENCODEDTOKEN-TOKEN-HERE
or
http://localhost:54607/Home/AdminBalances?ENCODEDTOKEN-TOKEN-HERE
... which might as well be MVC3's AntiForgeryToken, or a variant (see swt) thereof. This token would be set as an inline value on whatever URL format is chosen above.
Sample questions that prevent me from rolling my own solution
What URL format (above) would you use to validate the JSON GET (slash, questionmark, etc) Will a proxy respond to http://localhost:54607/Home/AdminBalances with http://localhost:54607/Home/AdminBalances?ENCODEDTOKEN-TOKEN-HERE data?
How would you deliver that encoded token to the webpage? Inline, or as a page variable?
How would you compose the token? Built in AntiforgeryToken, or by some other means?
The AntiForgeryToken uses a cookie. Would a backing cookie be used/needed in this case? HTTP Only? What about SSL in conjunction with HTTP Only?
How would you set your cache headers? Anything special for the Google Web Accelerator (for example)
What are the implications of just making the JSON request SSL?
Should the returned JSON array still be wrapped in an object just for safety's sake?
How will this solution interop with Microsoft's proposed templating and databinding features
The questions above are the reasons I'm not forging ahead and doing this myself. Not to mention there likely more questions I haven't thought of, and yet are a risk.
The Asp.net MVC AntiForgeryToken won't work through HTTP GET, because it relies on cookies which rely on HTTP POST (it uses the "Double Submit Cookies" technique described in the OWASP XSRF Prevention Cheat Sheet). You can also additionally protect the cookies sent to the client by setting the as httponly, so they cannot be spoofed via a script.
In this document you can find various techniques that can be used to prevent XSRF. It seems the you described would fall into the Approach 1. But we have a problem on how to retrieve the session on the server when using Ajax HTTP GET request since the cookies are not sent with the request. So you would also have to add a session identifier to you action's URL (aka. cookieless sessions, which are easier to hijack). So in order to perform an attack the attacker would only need to know the correct URL to perform the GET request.
Perhaps a good solution would be to store the session data using some key from the users SSL certificate (for example the certs thumb-print). This way only the owner of the SSL certificate could access his session. This way you don't need to use cookies and you don't need to send session identifiers via query string parameters.
Anyway, you will need to roll out your own XSRF protection if you don't want to use HTTP POST in Asp.net MVC.
I came to this problem and the solution was not so trivial however there is a fantastic blog to get you started this can be used with get and post ajax.
http://johan.driessen.se/posts/Updated-Anti-XSRF-Validation-for-ASP.NET-MVC-4-RC
If you place the following in the global name space all your post/gets can take advantage having an anti forgery token and you don't have to modify your ajax calls. Create an input element in a common page.
<form id="__AjaxAntiForgeryForm" action="#" method="post">#Html.AntiForgeryToken()</form>
The following javascript will read the anti forgery tokken and add it to the request header.
// Wire up the global jQuery ajaxSend event handler.
$(document).ajaxSend(namespace.ajax.globalSendHandler);
// <summary>
// Global handler for all ajax send events.
// </summary>
namespace.ajax.globalSendHandler = function (event, xhr, ajaxOptions) {
// Add the anti forgery token
xhr.setRequestHeader('__RequestVerificationToken', $("#__AjaxAntiForgeryForm input[name=__RequestVerificationToken]").val());
};
I think it is legitimate to use AntiforgeryToken (AFT) within an ajax http GET request provided that it is embedded in a form that already provides the AFT and associated cookie. The ajax handler can then do the validate on the server just how it would in a normal form post.