I have a view set up to work with datatables; it accepts the data posted by datatables in JSON format, so I use:
json.loads(request.body.decode('utf-8'))
to grab the data I need to search, sort, and paginate my tables via ajax. All of this works, but I'm worried that loading the body of the request in this way could leave me open to attacks, especially since I could be loading arbitrary data in that line.
What can/should I do to make sure this doesn't leave the code open to exploitation?
Note: I am using csrf tokens in each request. I'm worried more about direct attacks from a malicious user of the system.
Related
I have minimal knowledge over preventing XSS vulnerabilities of how it can be done especially with form inputs by not allowing the special characters like <,> etc., But the question that i have is with Ajax :
Get Request : How efficiently it can be prevented for the query string params.
POST Request : Is the data sent via POST request could still be a cause for XSS attack OR POST requests prevents the XSS attacks.
Please do help me in understanding this.
The first thing you have to understand about XSS is that it targets the browser, not the server. If tags are injected in your GET or POST requests, this is irrelevant to the server. It only matters how you present that data back to the user.
Say you have a field in the database where the content is user-generated input. When presenting this data to any viewer, filter it for HTML tags, or filter it before storing it, the order doesn't matter.
There's however a different form of XSS attack that you don't seem to be picturing here. An external site generating URLs and POST requests towards yours. This is legal and have to be treated in order to prevent things like changepassword.php?newpass=hacked (crude example). The POST equivalent to this is also possible.
What many websites do is generate what is commonly referred to as "postkey", a random value or phrase that is generated and bound to the user's current session and is attached to all POST and GET calls that modify information, either in the form of a Query String variable (GET) or a hidden field in a form (POST). All important POSTs and GETs should include it for validation.
An external attacker won't be able to know what the postkey is, since they can't do "cross-site reading" prior to a successful XSS attack, so any attempt to submit a POST or GET request directed towards your website will be missing this key and should be harmless.
I got a (Flask) backend powering an API that serves JSON to an Angular app.
I love the fact that my backend (algorithms, database) is totally disconnected from my frontend (design, UI) as it could literally run from two distinct servers. However since the view is entirely generated client side everyone can access the JSON data obviously. Say the application is a simple list of things (the things are stored in a JSON file).
In order to prevent direct access to my database through JSON in the browser console I found these options :
Encrypting the data (weak since the decrypting function will be freely visible in the javascript, but not so easy when dealing with minified files)
Instead of $http.get the whole database then filtering with angular, $http.get many times (as the user is scrolling a list for example) so that it is programmatically harder to crawl
I believe my options are still weak. How could I make it harder for a hacker to crawl the whole database ? Any ideas ?
As I understand this question - the user should be permitted to access all of the data via your UI, but you do not want them to access the API directly. As you have figured out, any data accessed by the client cannot be secured but we can make accessing it a little more of PITA.
One common way of doing this is to check the HTTP referer. When you make a call from the UI the server will be given the page the request is coming from. This is typically used to prevent people creating mashups that use your data without permission. As with all the HTTP request headers, you are relying on the caller to be truthful. This will not protect you from console hacking or someone writing a scraper in some other language. #see CSRF
Another idea is to embed a variable token in the html source that bootstraps your app. You can specify this as an angular constant or a global variable and include it in all of your $http requests. The token itself could be unique for each session or be a encrypted expiration date that only the server can process. However, this method is flawed as well as someone could parse the html source, get the code, and then make a request.
So really, you can make it harder for someone, but it is hardly foolproof.
If users should only be able to access some of the data, you can try something like firebase. It allows you to define rules for who can access what.
Security Considerations When designing web applications, consider
security threats from:
JSON vulnerability XSRF Both server and the client must cooperate in
order to eliminate these threats. Angular comes pre-configured with
strategies that address these issues, but for this to work backend
server cooperation is required.
JSON Vulnerability Protection A JSON vulnerability allows third party
website to turn your JSON resource URL into JSONP request under some
conditions. To counter this your server can prefix all JSON requests
with following string ")]}',\n". Angular will automatically strip the
prefix before processing it as JSON.
For example if your server needs to return:
['one','two'] which is vulnerable to attack, your server can return:
)]}', ['one','two'] Angular will strip the prefix, before processing
the JSON.
Cross Site Request Forgery (XSRF) Protection XSRF is a technique by
which an unauthorized site can gain your user's private data. Angular
provides a mechanism to counter XSRF. When performing XHR requests,
the $http service reads a token from a cookie (by default, XSRF-TOKEN)
and sets it as an HTTP header (X-XSRF-TOKEN). Since only JavaScript
that runs on your domain could read the cookie, your server can be
assured that the XHR came from JavaScript running on your domain. The
header will not be set for cross-domain requests.
To take advantage of this, your server needs to set a token in a
JavaScript readable session cookie called XSRF-TOKEN on the first HTTP
GET request. On subsequent XHR requests the server can verify that the
cookie matches X-XSRF-TOKEN HTTP header, and therefore be sure that
only JavaScript running on your domain could have sent the request.
The token must be unique for each user and must be verifiable by the
server (to prevent the JavaScript from making up its own tokens). We
recommend that the token is a digest of your site's authentication
cookie with a salt for added security.
The name of the headers can be specified using the xsrfHeaderName and
xsrfCookieName properties of either $httpProvider.defaults at
config-time, $http.defaults at run-time, or the per-request config
object.
Please Kindly refer the below link,
https://docs.angularjs.org/api/ng/service/$http
From AngularJS DOCs
JSON Vulnerability Protection
A JSON vulnerability allows third party website to turn your JSON resource URL into JSONP request under some conditions. To counter this your server can prefix all JSON requests with following string ")]}',\n". Angular will automatically strip the prefix before processing it as JSON.
There are other techniques like XSRF protection and Transformations which will further add security to your JSON communications. more on this can be found in AngularJS Docs https://docs.angularjs.org/api/ng/service/$http
You might want to consider using JSON Web Tokens for this. I'm not sure how to implement this in Flask but here is a decent example of how it can be done with a Nodejs backend. This example at least shows how you can implement it in Angularjs.
http://www.kdelemme.com/2014/03/09/authentication-with-angularjs-and-a-node-js-rest-api/
Update: JWT for Flask:
https://github.com/mattupstate/flask-jwt
Title question asks it all, what's the process going on under there? Why do I have to use AJAX if I wanted to submit that form asynchrously?
It's due to the way HTTP was designed. Back then, JavaScript was not as ubiquitous and not as powerful as it is today.
As it is, when you POST data to a page (a path), you are issuing a request to a server. The server can then respond in a variety of manners. There is the simple "return some content", whether it be HTML, text, JSON, XML, etc. There is also the possibility for the server to return a redirect, sending you to a different location.
What AJAX does is simply to run this request in the background and hide the fact that data was submitted to the server and a response was returned from the user's perspective.
In regards to this Haacked blog, I'm hesitant to implement the proposed anti-JSON GET hijacking solutions since
The recommended solutions to mitigating JSON hijacking involve non-REST-full JSON POSTs to GET data
The alternate solution (object wrapping) causes problems with 3rd party controls I don't have source-code access to.
I can't find a community-vetted implementation that implements the Alternative Solution (listed below) on how to compose the security token, or securely deliver it within the webpage. I also won't claim to be enough of an expert to roll my own implementation.
Referrer headers can't be relied upon
Background
This blog describes a CSRF issue regarding JSON Hijacking and recommends using JSON POSTs to GET data. Since using a HTTP POST to GET data isn't very REST-full, I'd looking for a more RESTfull solution that enables REST actions per session, or per page.
Another mitigation technique is to wrap JSON data in an object as described here. I'm afraid this may just delay the issue, until another technique is found.
Alternative Implementation
To me, it seems natural to extend the use ASP.NET MVC's AntiForgeryToken with jQuery HTTP GETs for my JSON.
For example if I GET some sensitive data, according to the Haacked link above, the following code is vulnerable:
$.getJSON('[url]', { [parameters] }, function(json) {
// callback function code
});
I agree that it isn't RESTfull to GET data using the recommended POST workaround. My thought is to send a validation token in the URL. That way the CSRF-style attacker won't know the complete URL. Cached, or not cached, they won't be able to get the data.
Below are two examples of how a JSON GET query could be done. I'm not sure what implementation is most effective, but may guess that the first one is safer from errant proxies caching this data, thus making it vulnerable to an attacker.
http://localhost:54607/Home/AdminBalances/ENCODEDTOKEN-TOKEN-HERE
or
http://localhost:54607/Home/AdminBalances?ENCODEDTOKEN-TOKEN-HERE
... which might as well be MVC3's AntiForgeryToken, or a variant (see swt) thereof. This token would be set as an inline value on whatever URL format is chosen above.
Sample questions that prevent me from rolling my own solution
What URL format (above) would you use to validate the JSON GET (slash, questionmark, etc) Will a proxy respond to http://localhost:54607/Home/AdminBalances with http://localhost:54607/Home/AdminBalances?ENCODEDTOKEN-TOKEN-HERE data?
How would you deliver that encoded token to the webpage? Inline, or as a page variable?
How would you compose the token? Built in AntiforgeryToken, or by some other means?
The AntiForgeryToken uses a cookie. Would a backing cookie be used/needed in this case? HTTP Only? What about SSL in conjunction with HTTP Only?
How would you set your cache headers? Anything special for the Google Web Accelerator (for example)
What are the implications of just making the JSON request SSL?
Should the returned JSON array still be wrapped in an object just for safety's sake?
How will this solution interop with Microsoft's proposed templating and databinding features
The questions above are the reasons I'm not forging ahead and doing this myself. Not to mention there likely more questions I haven't thought of, and yet are a risk.
The Asp.net MVC AntiForgeryToken won't work through HTTP GET, because it relies on cookies which rely on HTTP POST (it uses the "Double Submit Cookies" technique described in the OWASP XSRF Prevention Cheat Sheet). You can also additionally protect the cookies sent to the client by setting the as httponly, so they cannot be spoofed via a script.
In this document you can find various techniques that can be used to prevent XSRF. It seems the you described would fall into the Approach 1. But we have a problem on how to retrieve the session on the server when using Ajax HTTP GET request since the cookies are not sent with the request. So you would also have to add a session identifier to you action's URL (aka. cookieless sessions, which are easier to hijack). So in order to perform an attack the attacker would only need to know the correct URL to perform the GET request.
Perhaps a good solution would be to store the session data using some key from the users SSL certificate (for example the certs thumb-print). This way only the owner of the SSL certificate could access his session. This way you don't need to use cookies and you don't need to send session identifiers via query string parameters.
Anyway, you will need to roll out your own XSRF protection if you don't want to use HTTP POST in Asp.net MVC.
I came to this problem and the solution was not so trivial however there is a fantastic blog to get you started this can be used with get and post ajax.
http://johan.driessen.se/posts/Updated-Anti-XSRF-Validation-for-ASP.NET-MVC-4-RC
If you place the following in the global name space all your post/gets can take advantage having an anti forgery token and you don't have to modify your ajax calls. Create an input element in a common page.
<form id="__AjaxAntiForgeryForm" action="#" method="post">#Html.AntiForgeryToken()</form>
The following javascript will read the anti forgery tokken and add it to the request header.
// Wire up the global jQuery ajaxSend event handler.
$(document).ajaxSend(namespace.ajax.globalSendHandler);
// <summary>
// Global handler for all ajax send events.
// </summary>
namespace.ajax.globalSendHandler = function (event, xhr, ajaxOptions) {
// Add the anti forgery token
xhr.setRequestHeader('__RequestVerificationToken', $("#__AjaxAntiForgeryForm input[name=__RequestVerificationToken]").val());
};
I think it is legitimate to use AntiforgeryToken (AFT) within an ajax http GET request provided that it is embedded in a form that already provides the AFT and associated cookie. The ajax handler can then do the validate on the server just how it would in a normal form post.
Even if I offer alternatives to PUT and DELETE (c.f. "Low REST"), how can I provide user-friendly form validation for users who access my web service from the browser, while still exposing RESTful URIs? The form validation problem (described below) is my current quandry, but the broader question I want to ask is: if I go down the path of trying to provide both a RESTful public interface and a non-javascript HTML interface, is it going to make life easier or harder? Do they play together at all?
In theory, it should be merely a matter of varying the output format. A machine can query the URL "/people", and get a list of people in XML. A human user can point their browser at the same URL, and get a pretty HTML response instead. (I'm using the URL examples from the microformats wiki, which seem fairly reasonable).
Creating a new person resource is done with a POST request to the "/people" URL. To achieve this, the human user can first visit "/people/new", which returns a static HTML form for creating the resource. The form has method=POST and action="/people". That will work fine if the user's input is valid, but what if we do validation on the server side and discover an error? The friendly thing would be to return the form, populated with the data the user just entered, plus an error message so that they can fix the problem and resubmit. But we can't return that output directly from a POST to "/people" or it breaks our URL system, and if we redirect the user back to the "/people/new" form then there is no way to report the error and repopulate the form (unless we store the data to session state, which would be even less RESTful).
With javascript, things would be much easier. Just do the POST in the background, and if it fails then display the error at the top of the form. But I want the app to degrade gracefully when javascript support isn't available. At the moment, I'm led to conclude that a non-trivial web app cannot implement an HTML interface without javascript, and use a conventional RESTful URL scheme (such as that described on the microformats wiki). If I'm wrong, please tell me so!
Related questions on Stack Overflow (neither of which deal with form validation):
How to send HTML form RESTfully?
How do you implement resource "edit" forms in a RESTful way?
you could have the html form post directly to /people/new. If the validation fails, rerender the edit form with the appropriate information. If it succeeds, forward the user to the new URL. This would be consistent with the REST architecture as I understand it.
I saw you comment to Monis Iqbal, and I have to admit I don't know what you mean by "non-RESTful URLS". The only thing the REST architecture asks from a URL is that it be opaque, and that it be uniquely paired to a resource. REST doesn't care what it looks like, what's in it, how slashes or used, how many are used, or anything like that. The visible design of the URL is up to you and REST has no bearing.
Thanks for the responses. They have freed my mind a bit, and so in response to my own question I would like to propose an alternative set of RESTful URL conventions which actually embrace the two methods (GET and POST) of the non-AJAX world, instead of trying to work around them.
Edit: As commenters have pointed out, these "conventions" should not be part of the RESTful API itself. On the other hand, internal conventions are useful because they make the server-side implementation more consistent and hence easier for developers to understand and maintain. RESTful clients, however, should treat the URLs as opaque, and always obtain them as hyperlinks, never by constructing URLs themselves.
GET /people
return a list of all records
GET /people/new
return a form for adding a new record
POST /people/new
create a new record
(for an HTML client, return the form again if the input is invalid, otherwise redirect to the new resource)
GET /people/1
return the first record
GET /people/1/edit
return a form for editing the first record
POST /people/1/edit
update the first record
GET /people/1/delete
return a form for deleting the record
(may be simply a confirmation - are you sure you want to delete?)
POST /people/1/delete
delete the record
There is a pattern here: GET on a resource, e.g. "/people/1", returns the record itself. GET on resource+operation returns an HTML form, e.g. "/people/1/edit". POST on resource+operation actually executes the operation.
Perhaps this is not quite so elegant as using additional HTTP verbs (PUT and DELETE), but these URLs should work well with vanilla HTML forms. They should also be pretty self-explanatory to a human user...I'm a believer in the idea that "the URL is part of the UI" for users accessing the web server via a browser.
P.S. Let me explain how I would do the deletes. The "/people/1" view will have a link to "/people/1/delete", with an onclick javascript handler. With javascript enabled, the click is intercepted and a confirmation box presented to the user. If they confirm the delete, a POST is sent, deleting the record immediately. But if javascript is disabled, clicking the link will instead send a GET request, which returns a delete confirmation form from the server, and that form sends the POST to perform the delete. Thus, javascript improves the user experience (faster response), but without it the website degrades gracefully.
Why do you want to create a second "API" using XML?
Your HTML contains the data your user needs to see. HTML is relatively easy to parse. The class attribute can be used to add semantics as microformats do. Your HTML contains forms and links to be able to access all of the functionality of your application.
Why would you create another interface that delivers completely semantic free application/xml that will likely contain no hypermedia links so that you now have to hard code urls into your client, creating nasty coupling?
If you can get your application working using HTML in a web browser without needing to store session state, then you already have a RESTful API. Don't kill yourself trying to design a bunch of URLs that corresponds to someone's idea of a standard.
Here is a quote from Roy Fielding,
A REST API must not define fixed
resource names or hierarchies
I know this flies in the face of probably almost every example of REST that you have seen but that is because they are all wrong. I know I am starting to sound like a religious zealot, but it kills me to see people struggling to design RESTful API's when they are starting off on completely the wrong foot.
Listen to Breton when he says "REST doesn't care what [the url] looks like" and #Wahnfrieden will be along soon to tell you the same thing. That microformats page is horrible advice for someone trying to do REST. I'm not saying it is horrible advice for someone creating some other kind of HTTP API, just not a RESTful one.
Why not use AJAX to do the work on the client side and if javascript is disabled then design the html so that the conventional POST would work.