I've been working with ServiceStack for a while now and recently a new need came up that requires receiving of some html templates inside a JSON request body. I'm obviously thinking about escaping this HTML and it seems to work but I would prefer consumers not to bother escaping a potentially large template. I know ASP.NET is capable of handling this but I wonder if it can be done with ServiceStack.
The following JSON body is received incomplete at the REST endpoint because of the second double quote ...="test...
{
"template" : "<html><body><div name="test"></div></body></html>"
}
"I would prefer consumers not to bother escaping a potentially large template"
I'm not sure I follow. Why would the consumers know anything about escaping a template? Shouldn't that be transparent to the consumer? Any call to JSON.stringify(sourceString) or sourceString.toJson() will automatically escape embedded double-quotes.
"I know ASP.NET is capable of handling this"
Embedded double-quotes must be escaped in valid json syntax. I don't see how ASP.NET wouldn't have the same problem. Am I missing something?
Related
I'm using Django REST Framework and all the API calls come from Android and iOS apps. The system works perfectly most of the time, however, when an internal server error happens and I get an email from Django, the POST of the WSGIRequest contains <could not parse> instead of the actual posted JSON data (even though 'CONTENT_TYPE': 'application/json' is also in the header, and the data is sent as JSON).
This is really annoying, as it would be great to see the request body that actually causes the error, not just the stacktrace.
The <could not parse> part is very similar to this question (in the ModPythonRequest part): django request.POST contains <could not parse>, except the actual problem is slightly different. Also the reference link in that question (https://stackoverflow.com/questions/12471661/mod-python-could-not-parse-the-django-post-request-for-blackberry-and-some-andro) seems to have gone down, even though the name looked very promising.
I'm on Django 1.6.2 and DRF 2.3.13.
The POST dictionary of the WSGIRequest is always going to be invalid, because it is intended to hold the parsed form data when the Content-Type is application/x-www-form-urlencoded or multipart/form-data.
The data you want is in the body attribute of the WSGIRequest object, which isn't output when that object is converted to a string to be written to the log.
When using Django REST framework, you will typically want to access request.DATA (which will handle whatever formats you have parsers configured for - defaulting to form content and JSON) instead of Django's standard request.POST, which will only handle form encoded data.
I am designing a Web API which returns JSON as the content-type, the response body could contain characters like ', ", < and >, they are valid characters in JSON. So, my question is should I do HTML encode for my Web API response body or should I leave this task to HTML client who is consuming my Web API?
No; you must not.
You must only escape data if and when you concatenate it into a structured format.
If you return JSON like { "text": "Content by X & Y" }, anyone who reads that JSON will see the literal text &.
It will only work correctly for extremely broken clients who concatenate it directly into their HTML without escaping.
In short:
Never escape text except when you're about to display it
What platform are you using? For example, Node.js, you can use restify to handle that very well. You don't need to explicitly encode the data. Therefore, please find a restful framework or component to help you out.
I'm trying to return an html snippet from a service that can only return valid JSON.
I've tried some things like:
This gets me a bunch of character like \n\n\n\n\t\t\t\t
return JSON.stringify({html: $('body').html()});
or
return JSON.stringify($('body').html());
On the receiving end, I'd like to be able to parse that HTML via Cheerio, or jQuery or JSDom so I can then run queries like $(".some_selector") on that data.
What is the proper way of doing this? Any special libraries / methods that can handle the escaping for me? I've googled it, but haven't had any clear results...
Thanks.
On the receiving end, you need to simply undo the JSON serialization. That's it!
Your HTML will be in its regular format at obj.html, which you can then parse with whatever DOM parser you want.
Well, you are probably going to need to worry about quotes in the HTML (like with attributes) because the could interfere with the quotes that delimit your JSON values.
Here is similar question as well as this web page that explains some of what you need to consider.
Briefly looking at npmsj.org, I didn't see any reputable modules that might help you make HTML JSON compatible, but I think you can probably figure it out fairly easily on your own, given a large enough sample set of HTML. You can always run your JSON through this validator to check it. I suppose you could also simply do a JSON.parse(jsonContainingHtml) on it as well. You'll get an exception if the string is not valid JSON.
RoR 3 automagically sanitizes ERB templates (when done correctly). However, I've got a little project where I'm using RoR for the application tier only and javascript for the presentation. So, typical request is ajax call to rails route and render returned json. Issue is it is currently possible for me to inject js, create a new product with title <script>alert('hello')</script> and this is returned as is on the next request and the browser happily interprets the script.
Is it best to
sanitize the inputs on post?
sanitize the json response on the server? (override to_json?)
sanitize the json response on the client?
I appreciate any input.
You should encode HTML entities on the content client-side as you're appending data to the page.
The question is, do your other product fields intentionally contain markup like links or paragraph tags that will also be encoded? If this is the case, and you intend to render some parts of the json response as HTML on the page, then you should be sanitizing your input at the point when new products are created, and limited the HTML tags you allow to a specific subset, and then scrubbing away their attributes. There are libraries to automate this process, like the sanitize gem.
G'day gurus,
I'm calling the REST APIs of an enterprise application that shall remain nameless, and they return JSON such as the following:
throw 'allowIllegalResourceCall is false.';
{
"data": ... loads of valid JSON stuff here ...
}
Is this actually valid JSON? If (as I suspect) it isn't, is there any compelling reason for these kinds of shenanigans?
The response I received from the application vendor is that this is done for security purposes, but I'm struggling to understand how this improves security much, if at all.
Thanks in advance!
Peter
According to
http://jsonlint.com/
It is not.
Something like the below is.
{
"data": "test"
}
Are they expecting you to pull the JSon load out of the message above?
Its not a JSON format at all. From your question it seems you are working with enterprise systems like JIVE :). I am also facing same issue with JIVE api. This is the problem with their V3 API. Not standard , but following thing worked for me. (I am not sure if you are talking about JIVE or not)
//invalid jason response... https://developers.jivesoftware.com/community/thread/2153
jiveResponse = jiveResponse.Replace
("throw 'allowIllegalResourceCall is false.';",String.Empty);
There is a valid reason for this: it protects against CSRF attacks. If you include a JSON url as the target of a <script> tag, then the same-origin policy doesn't apply. This means that a malicious site can include the URL of a JSON API, and any authenticated users will successfully request that data.
By appropriately overriding Object.prototype and/or Array.prototype, the malicious site can get any data parsed as an object literal or array literal (and all valid JSON is also valid javascript). The throw statement protects against this by making it impossible to parse javascript included on a page via <script> tags.
Definitely NOT valid JSON. Maybe there's an error in the implementation that is mixing some kind of debug output with the correct output?
And, by no means this is for security reasons. Seems to me this is a plain bug.
throw 'allowIllegalResourceCall is false.'; is certainly not valid JSON.
What MIME type is reported?
It seems they have added that line to prevent JSON Hijacking. Something like that line is required to prevent JSON Hijacking only if you return a JSON array. But they may have added that line above all of their JSON responses for easier implementation.
Before using it, you have to strip out the first line, and then parse the remaining as JSON.