I'm currently working in an environment where several ORION Context Brokers (OCBs) are deployed and I'm guessing how to include headers in the enquiries.
I mean, when one of the OCBs communicates with other OCB and wants to add additional headers, how it should be handled (if there is a way to)?
Thanks!
If the comunication is done using notifications (i.e. OCB1 is subscribed to OCB2, so OCB1 receives notifications from OCB2), you could use the NGSIv2 custom notifications functionality in order to include custom headers.
See "Custom notifications" section in the NGSIv2 specification document. Pay special attention to the headers field within httpCustom.
Related
Let's assume an web application that for each URI presents a nice html view for GET requests and allows to update the underlying resource through POST/PUT/PATCH/WHATEVER.
How do I then expose various forms that actually allow performing such requests from the browser? And broader: assuming I have alternative views (possibly also HTML) for the same resource, where do I put those? Arguably, such forms can be considered alternative views, so having an answer to the broader question would be ideal.
Edit: To clarify, my question is not about pure data APIs serving JSON or whatnot, but about HTML apps such as Stackoverflow. For example you can get the collection of questions under /questions and this particular one at /questions/24696982 which makes sense. To get the form to add a new question, you will have to use /questions/ask, which I'm not sure is alright. And that form POSTs to /questions/ask/submit, which seems just plain wrong. Making a GET request to that URL yields a 404 (if anything it should be a 405). The form should be POSTing to /questions. Still I would like to know whether at least the URI for the form is considered acceptable in a RESTful system.
You have a website like, the one way to build a real RESTFull API is to split the frontend and the API - thats in my opinion the best way (some may disagree) - maybe some other don't think like this but lets say the frontend team got www.domain and your team for the API got api.domain.
GET api.domain/questions - Retrieves a list of tickets
GET api.domain/questions/12 - Retrieves a specific ticket
POST api.domain/questions - Creates a new ticket
PUT api.domain/questions/12 - Updates ticket #12
DELETE api.domain/questions/12 - Deletes ticket #12
PATCH api.domain/questions/12 - Partially updates ticket #12 #I only want to display that this also exists - i don't really use it...
AWESOME EDIT: As you can see also stackoverflow uses this method: api.stackexchange.com
So as you can see you can have these structure - but you also can have a form on www.domain/questions/ask and this form would send the request to api.domain/questions via POST. I want to refer to: https://thenewcircle.com/s/post/1221/designing_a_beautiful_rest_json_api_video its a really nice podcast you should have heard.
EDIT: (another point of view)
Another idea is that you can simply choose which content should come back (Json,XML,HTML) if your client sends you the right Accept-Header.
Example 1:
URL REQUEST ACCEPT HEADER RESPONSE
-----------------------------------------------------------------------------------------
domain/questions GET application/json all questions as json
domain/questions GET text/html the page as html with all questions
domain/questions/ask GET text/html Your html for to add a new question
domain/questions POST application/json Add a new new questions (this would be called from ./ask to add the new questions
domain/questions/ask GET application/json 404 Status-Code because on questions/ask you don't have implemented any resource
Example-2:
URL REQUEST ACCEPT HEADER RESPONSE
-----------------------------------------------------------------------------------------
domain/questions/12 GET application/json Shows the questions with the ID 12 as JSON
domain/questions/12 GET text/html Shows the HTML representation of your page
domain/questions/12/edit GET text/html Your html for to edit a the question
domain/questions/12 PUT application/json Updates the questions with the ID 12 // just to add the PATCH thing.. i don't really use it but if you don't update the whole object you could/should use PATCH instead of PUT :p
domain/questions/12/edit GET application/json 404 Status-Code because on questions/ask you don't have implemented any resource
Yesterday I told you about the first idea (which is - I think for using an api as a team (one for frontend and one team that develops the api - a better way) but as #jackweirdy commented (thanks for that - i then searched a lot and was looking at other podcasts from developer around the world and how they would do that) below it's really all up to you - it's your api and at the end you/your team will decide for one way. Hope this helps you or other that looking for how to build a API on a REST background.
The examples in the EDIT-Section would be (if I got it right) not like here on stackoverflow
This is something I've had trouble with myself, and which I don't think there's a right answer to.
Assuming I have an API exposing /people/:id, I generally reserve an endpoint for /people/new. a GET request to that url with Accept: text/html will return a form for creation, but anything else will throw a 404, since this page only exists for people in a web browser. The form on that page will then post to /people/ as you'd expect.
Similarly, if someone wants to edit an existing person, the form to do that might be served from /people/1/update, again HTML only.
If your API has that structure, then I think reserving keywords such as new or update is perfectly reasonable.
As far as I can understand your question, you want an application that :
displays HTML pages (and eventually other formats ?)
displays form views for creation of new elements or for update of existing ones
accept POST/PUT with url encoded data (sent by submitting above forms) to create of update those elements (and eventually other formats ?)
Ruby on Rails is a framework that is targetted as this kind of requirement. Extract from the guide Rails Routing from the Outside In :
HTTP Verb Path action used for
GET /photos index display a list of all photos
GET /photos/new new return an HTML form for creating a new photo
POST /photos create create a new photo
GET /photos/:id show display a specific photo
GET /photos/:id/edit edit return an HTML form for editing a photo
PUT /photos/:id update update a specific photo
DELETE /photos/:id destroy delete a specific photo
You can have HTML views for the actions index, new, show and edit.
Personally, I would recommend to add the following :
POST /photos/:id update update a specific photo
POST /photos/:id/delete destroy delete a specific photo
so that it would be simpler to update or delete elements via html forms.
All those paths are only Rails convention and are not imposed by REST but it gives a clean example of what can be done.
But it is quite easy to make an application following the same or slightly different conventions using other frameworks. Java + Spring MVC can do that very easily, with HTML views using JSP, Velocity, Thymeleaf or others, and the possibility of using JSON in input or output simply using HTTP headers or suffixes in URL (GET /photos/:id.json) with a little less magic but more control than RoR. And I'm not an expert in other framework like Struts2 (still Java), or Django (Python) but I am pretty sure that it is possible too.
What is important :
choose a language (Ruby, Python, Java, PHP, ASP.NET, ...)
choose a framework compatible with RESTfull urls
ensure you can have views in HTML, or JSON, or enter the format you want by adding a suffix or a HTTP header and eventually the appropriate adapter/converter
You could do it by hand but frameworks limits boiler plate code.
The essence of REst was never about how URLs looks like,but how http verbs and headers are used to transfer datas.
This whole "restfull urls" thing is made up by people who dont understand what Rest is. All the Rest spec says is that URLs must be unique.
Now if you really want "restfull" forms,then form should be a resource with an id, like /form/2929929 .Of course it doesnt make sense to do so,since forms are strictly for web users and REst doesnt care about how data is acquiered, only about how it is transfered.
In short,choose whatever URL you want. Some frameworks use new and update for forms. By the way the /questions/ask/submit is totally valid in a Rest context, because what you submit and a question can be 2 totally difference resources.
You need to understand that there is a difference between a RESTfull application and a REST client.
A RESTfull application has pure restfull urls as you described, such as
GET /persons : gets a list of all the persons in database
POST /persons : adds a new person
GET /person/1 : gets a person with id 1
PUT /person/1 : updates person with id 1
DELETE /person/1 : deletes person with id 1
and so on...
Such an application does not have any forms or UI for submitting data. It only accepts data via HTTP requests. To use such an application you can send and receive data using tools like curl or even your browser, which allow you to make HTTP requests.
Now, clearly such an application is not usable from the user point of view. Hence we need to create client applications which consume these restfull applications. These clients are not restfull at all and have urls like:
GET /person/showall : displays a list of all persons
GET /person/create : shows new person form
POST /person/create : submits the data to the restfull application via ajax or simillar technology.
and so on...
These clients can be another HTML application, an android application, an iOS application, etc.
What you are trying to do here is create a single application which has both restful urls for objects as well as forms/pages for data display and input. This is absolutely fine.
Just make sure that you design proper restfull urls for your objects while you can have any url you find suitable for your forms.
In 100% RESTful Web services resources are identified using descriptive URLs, that is URLs composed only of noun phrases.
Generaly speaking, for creating a new resource, you would use PUT, although some frameworks (such as Zend Framework 2, if I remember well), use POST for this purpose. So, for creating a question you could PUT questions, then providing the question identifier in the body of the request, or PUT questions/{identifier}, thus providing the id in the URL.
Contemporary web/cloud applications have moved to what is known as a single page application architecture.
This architecture has a back end REST API (typically JSON based) which is then consumed by either single page applications or native client apps on mobile phones and tablet. The server is then much easier to implement and scale and provides the needed access regardless if its a web client or a native phone/tablet platform.
The client architecture is known as MV* for Model, View and * is anything else the framework provides such as controller logic and persistence.
In my applications I have used a number of MV* frameworks and libraries in anger and investigated many many more. I've had some success with backbone, and my favorite Ember.js, although there are many frameworks and everyone has their favorite for different reasons and that is a whole topic on its own. I will say that depending on the needs of your application different frameworks will be more or less appropriate. I know what matters to my productivity so I have settled on Ember after doing the rounds.
On the backend you have a similar myriad of choices but choose a platform that is known to be mature and stable ans same goes for your data persistence. There are a number of cloud services that give you a REST/JSON api with no coding or deployment concerns now so you can focus more on the client development and less on the server.
It is important to understand that in single page applications the browser url does not need to have a 1 to 1 correspondance with the backend rest api. In fact it would be detrimental to usability taking such a simple minded approach. Of all the client frameworks Ember gets this right as it has a built-in router, and as a result client state is captured in the URL so the page can survive a refresh and can also be bookmarked. You really can keep your client view independent to the backend api endpoints. I design my client URLs around the menu/structure of my forms. In complex apps the URLs nest as far as I need the app to partition and drill down into the details, yet the api endpoints are flat and may span multiple service providers. A view in my client app often assembles data from multiple endpoints and similarly on Accept/Save it pushes to multiple endpoints. It is also possible to implement local persistence so the web client can be used offline and so that temporary or half filled out forms can survive a page refresh.
Another consideration with such an architecture is SEO. With single page applications one needs to be able to provide prerendered pages to web crawlers. Fortunately there are a number of tools which can auto generate the pages for single page applications so that web crawlers can still index your sites content, tools such as pretender.io and many others can solve this for you.
At the end of all this you have a server with a number of REST endpoints and typically a single index.html, app.js app.css and any other assets such as images and fonts.
Typically you need a toolchain for generating these files from your source code which are then either hosted on your domain or on a CDN. I also configure my app and server for CORS so the web client can be hosted on a different domain to the REST back end which also works well in development.
I recommend the broccoli or ember-cli tool chain for assembling all your web client assets and I have also had good experience with Brunch. I've tried most of the tools out there and those are the only ones that get my vote.
For API design I've been actively providing feedback on the latest drafts of JSON API. There is a lot of good work being done there and you can use that as a good starting point.
Usually in production Web Applications I recommend separating how static content is delivered vs how dynamic content is delivered.
Let us hope you are not constrained by SEO and can actually use the wonder of DOM manipulation (ie Client-Side templating)...
I would highly recommend going down the path of learning how to create a SPA (Single Page Application)
However, back to the topic at hand.
Static content (HTML, CSS, Javascript, images) should be delivered thru a different server than your dynamic content (the REST data in json/xml format).
Your HTML should use JQuery/AngularJS/Backbone -- some type of JavaScript framework to actually "render" your HTML on the client-side using JavaScript.
The JavaScript frameworks will also make the proper RESTful calls to POST or PUT a form (which should be a UI representation of some REST path)
Lets say you have a form for a Profile,
GET /profile/{id} would be called to pre-populate a profile FORM
PUT /profile/{id} would be called to update the profile
** JavaScript will pre-populate the FORM by calling one or more RESTful GET methods.
** JavaScript will take entered data from FORM and POST/PUT it to the RESTful server.
The point you should take away from this is:
Let an advanced JavaScript library handle the sending of RESTful requests and "rendering" of the HTML.
HTML is only a template (static content) and can be hosted on a completely different server that is optimized for the job of delivering "static content" :)
Hope that makes sense.
Cheers!
P.S.
Learn about Cross-origin resource sharing (CORS) if you have not already. You will likely need that knowledge to properly host your static content on a different server/domain than your dynamic content.
I'm new to REST Web based services and trying to understanding how the contract is created for JSON returned REST services.
From my understanding, any XML based SOAP/REST services will have a WSDL document.
What document is created for JSON based REST Services?
a REST web service doesn't have any auto explanation document like wsdl, you need to know how the webservice works, reading the documentation provided with it. Generally it works with common requests. Assuming that you have a products REST webservice, you could have:
GET /products -> read all products
GET /products/1 -> read the product 1
POST /products -> create a new product
PUT /products/1 -> update product 1
DELETE /products/1 -> delete product 1
but you have to know which parameters you need to send to any request. I hope I was clear...
Every HTTP response has metadata in HTTP headers. One of those HTTP headers is ContentType. The content type identifies a media type which is the contract that the response payload must conform to. The specifications for media types can be found here http://www.iana.org/assignments/media-types/media-types.xhtml
One of the major differences between SOAP and HTTP (as an application protocol) is that SOAP defines the contract at design time, whereas with HTTP the contract is specified in the response message so it can change over time. Therefore it is important for the client to read the content type on each response to know how to process the response.
There is WADL (http://en.wikipedia.org/wiki/Web_Application_Description_Language) although it is not so much used as WSDL for SOAP. REST services written in Java EE automatically generate it as .../application.wadl, PHP suppor is pretty poor as far as I know.
As the others mentioned a web service definition is not necessary for RESTful services, however if you want to create something similar for your API the industry standard is Swagger/OpenAPI, though GraphQL schemas are also becoming a defacto standard too.
There are also a few other options you can also explore (see wikipedia).
Here is a list of the most common options:
swagger.json or openapi.json files in the OpenAPI Spec
GraphQL Schemas with full spec here
Postman Collections which can be published online.
RAML
RSDL
I understand that deleting data with an http GET is a security hole. Is it also the case that calling delete from a link in general is a bad idea? I'm referring specifically to an asp.net mvc actionlink that specifies the http method as a POST.
#Ajax.ActionLink("Delete",
"DeleteNote", new { noteid = Model.noteid },
new AjaxOptions
{
Confirm = "Delete?",
HttpMethod = "POST",
OnSuccess = "postmessage('note_Deleted_" + Model.noteid + "')"
})
Using HTTP GET is not a security hole, but could be considered bad design for a RESTful API. The convention is to use HTTP DELETE for deletion of resources. The security hole is if your API does not support any authentication/authorization, whether it is a GET, DELETE, POST or PUT. Then anybody with a web browser or tool like Fiddler could manipulate data they should not have access to. Take a look at this article that describes how to use a customized Authorize attribute to secure a Web API.
You can still use a link to invoke your API but do not use a direct link that defines an href to the URL for delete. Instead define your anchor like this with an id.
Delete
Then define a click event that calls a Javascript function that makes a secure ajax call to your API as described in the previously mentioned article.
$('aDeleteNote').click(DeleteNoteFunc);
If you want to use Razor to specify the noteid used to identify what to delete then you can put it in a hidden field our a Javascript variable that the DeleteNoteFunc has access to.
What you're touching on here is a vulnerability called Cross Site Request Forgery.
A quick example of this would be as follows.
You have a GET link on your website, e.g. "www.example.com/notes/delete/232"
A user is logged into your website with permissions to delete note 232.
The user receives an email from the attacker that entices them to visit the attacker's website.
The user visits the website in the email, but one of the image tags is defined as follows: "<img src="http://www.example.com/notes/delete/232" width="0" height="0" />"
This causes the browser to make a HTTP request to your website, sending the correct authentication cookies which will result in the note being deleted without the user's knowledge.
Now this is not a vulnerability in the GET itself, but it is one of the reasons why methods such as GET should not be used to make changes on the server.
Yes you are correct you should use POST for this. POST would still be vulnerable to an attack similar to the above but the attacker would either need to create a standard HTML form to POST the data, or would have to create the POST via AJAX. If you pass the HTTP header "X-Requested-With: XMLHttpRequest" and check this in your server method this will help prevent this attack as this header cannot be added to a cross domain request.
If you want to further secure this then you should implement the Synchroniser Token Pattern which will validate a token as part of the POST request to ensure the request came from your own site.
I don't know if I'd describe deleting data from a GET request as necessarily a security hole (provided your application implements proper security), but it's often not a great idea as you know.
If you have delete links, you should definitely put nofollow on them, but I don't think that links which delete data are inherently bad. Even if you have no link in your markup, you still end up making an http request to a url that will delete something, either through a form submission or ajax.
I am trying to build my application's admin UI using sling's userManager REST interface, but I would like to customize the json rendering. For example, I would like the response of "Get group" to include the members only if the requestor is a member.
I started by adding libs/sling/group/json.esp but I don't understand how I can get hold of the default response and customize it. Even if I had to query and form the json from scratch, where can I find information about APIs available to get this data from JCR/Sling?
I found that I could use ResourceTraversor to dump the resource object in json form but using new Packages.org.apache.sling.servlets.get.impl.helpers.ResourceTraversor(-1, 10000, resource, true) in the esp throws up an error
There are a few things to note here.
First, you should avoid putting your code under the libs directory. Your app code should live under the apps directory. When attempting to resolve a servlet for a URI, Sling will check apps before it checks libs so if you need to completely override functionality delivered with Sling, you would place your code in apps.
Second, what is (probably, depending on how you have things setup) happening when you request http://localhost:8080/system/userManager/group/administrators.tidy.1.json is the request is being handled by Sling's default GET servlet, because it finds no other script or servlet which is applicable. For research purposes it might be worth looking at the code for the default get servlet, org.apache.sling.servlets.get.impl.DefaultGetServlet, to see what it's using to render JSON. If you need to handle the rendering of a user group in a manner different than what the default GET servlet is doing, then you would need to create a servlet which is listening for requests for resources of type sling/group. It would probably be ideal to create a servlet for this purpose and register it with OSGI. http://sling.apache.org/site/servlets.html provides the various properties you would need to set to ensure the servlet resolver finds your servlet. Your servlet then would handle the request and as such would have direct and easy access to the requested resource.
Third, the particular need you specified is that you do not want the group members to render unless the requesting user is a member of the group requested. This is more of an access control issue than a rendering issue. Sling and Jackrabbit, out of the box, make as few assumptions as possible concerning how you might want your application to be setup. That being the case, you need to establish the access controls that are applicable for your particular use case. The wiki post on Access Control in the Jackrabbit wiki ( http://wiki.apache.org/jackrabbit/AccessControl ) goes into this to an extent.
Using directions from Paul Michelotti's answer, I researched further and found a suitable solution to my problem.
Sling accepts request filters (javax.servlet.Filter) through SCR annotations like the one below
#SlingFilter(scope = SlingFilterScope.REQUEST, order = Integer.MIN_VALUE)
Every request is passed down to the filter before it is processed by the servlet. Using the resourceType, I was able to distinguish requests to group.1.json and group/mygroup.1.json. Since the filter also has access to the current user, I was able to decide to deny the request if it did not abide by my security model and return a 404 status code.
Please refer to this page for details on filters. You can also check out the sample project urlfilter for directions on usage.
In regards to this Haacked blog, I'm hesitant to implement the proposed anti-JSON GET hijacking solutions since
The recommended solutions to mitigating JSON hijacking involve non-REST-full JSON POSTs to GET data
The alternate solution (object wrapping) causes problems with 3rd party controls I don't have source-code access to.
I can't find a community-vetted implementation that implements the Alternative Solution (listed below) on how to compose the security token, or securely deliver it within the webpage. I also won't claim to be enough of an expert to roll my own implementation.
Referrer headers can't be relied upon
Background
This blog describes a CSRF issue regarding JSON Hijacking and recommends using JSON POSTs to GET data. Since using a HTTP POST to GET data isn't very REST-full, I'd looking for a more RESTfull solution that enables REST actions per session, or per page.
Another mitigation technique is to wrap JSON data in an object as described here. I'm afraid this may just delay the issue, until another technique is found.
Alternative Implementation
To me, it seems natural to extend the use ASP.NET MVC's AntiForgeryToken with jQuery HTTP GETs for my JSON.
For example if I GET some sensitive data, according to the Haacked link above, the following code is vulnerable:
$.getJSON('[url]', { [parameters] }, function(json) {
// callback function code
});
I agree that it isn't RESTfull to GET data using the recommended POST workaround. My thought is to send a validation token in the URL. That way the CSRF-style attacker won't know the complete URL. Cached, or not cached, they won't be able to get the data.
Below are two examples of how a JSON GET query could be done. I'm not sure what implementation is most effective, but may guess that the first one is safer from errant proxies caching this data, thus making it vulnerable to an attacker.
http://localhost:54607/Home/AdminBalances/ENCODEDTOKEN-TOKEN-HERE
or
http://localhost:54607/Home/AdminBalances?ENCODEDTOKEN-TOKEN-HERE
... which might as well be MVC3's AntiForgeryToken, or a variant (see swt) thereof. This token would be set as an inline value on whatever URL format is chosen above.
Sample questions that prevent me from rolling my own solution
What URL format (above) would you use to validate the JSON GET (slash, questionmark, etc) Will a proxy respond to http://localhost:54607/Home/AdminBalances with http://localhost:54607/Home/AdminBalances?ENCODEDTOKEN-TOKEN-HERE data?
How would you deliver that encoded token to the webpage? Inline, or as a page variable?
How would you compose the token? Built in AntiforgeryToken, or by some other means?
The AntiForgeryToken uses a cookie. Would a backing cookie be used/needed in this case? HTTP Only? What about SSL in conjunction with HTTP Only?
How would you set your cache headers? Anything special for the Google Web Accelerator (for example)
What are the implications of just making the JSON request SSL?
Should the returned JSON array still be wrapped in an object just for safety's sake?
How will this solution interop with Microsoft's proposed templating and databinding features
The questions above are the reasons I'm not forging ahead and doing this myself. Not to mention there likely more questions I haven't thought of, and yet are a risk.
The Asp.net MVC AntiForgeryToken won't work through HTTP GET, because it relies on cookies which rely on HTTP POST (it uses the "Double Submit Cookies" technique described in the OWASP XSRF Prevention Cheat Sheet). You can also additionally protect the cookies sent to the client by setting the as httponly, so they cannot be spoofed via a script.
In this document you can find various techniques that can be used to prevent XSRF. It seems the you described would fall into the Approach 1. But we have a problem on how to retrieve the session on the server when using Ajax HTTP GET request since the cookies are not sent with the request. So you would also have to add a session identifier to you action's URL (aka. cookieless sessions, which are easier to hijack). So in order to perform an attack the attacker would only need to know the correct URL to perform the GET request.
Perhaps a good solution would be to store the session data using some key from the users SSL certificate (for example the certs thumb-print). This way only the owner of the SSL certificate could access his session. This way you don't need to use cookies and you don't need to send session identifiers via query string parameters.
Anyway, you will need to roll out your own XSRF protection if you don't want to use HTTP POST in Asp.net MVC.
I came to this problem and the solution was not so trivial however there is a fantastic blog to get you started this can be used with get and post ajax.
http://johan.driessen.se/posts/Updated-Anti-XSRF-Validation-for-ASP.NET-MVC-4-RC
If you place the following in the global name space all your post/gets can take advantage having an anti forgery token and you don't have to modify your ajax calls. Create an input element in a common page.
<form id="__AjaxAntiForgeryForm" action="#" method="post">#Html.AntiForgeryToken()</form>
The following javascript will read the anti forgery tokken and add it to the request header.
// Wire up the global jQuery ajaxSend event handler.
$(document).ajaxSend(namespace.ajax.globalSendHandler);
// <summary>
// Global handler for all ajax send events.
// </summary>
namespace.ajax.globalSendHandler = function (event, xhr, ajaxOptions) {
// Add the anti forgery token
xhr.setRequestHeader('__RequestVerificationToken', $("#__AjaxAntiForgeryForm input[name=__RequestVerificationToken]").val());
};
I think it is legitimate to use AntiforgeryToken (AFT) within an ajax http GET request provided that it is embedded in a form that already provides the AFT and associated cookie. The ajax handler can then do the validate on the server just how it would in a normal form post.