I have to pass a json array consisting of json objects from the servlet to the jsp page. This data transfer slows down page responsiveness. Is there any way to optimize performance while passing large json from servlet to jsp.
code looks like:
request.setAttribute("jsonStringForDataTable", jsonArrayForDataTable);
response.setContentType("text/plain");
response.setContentLength(jsonArrayForDataTable.toString().getBytes().length);
ServletOutputStream out=response.getOutputStream();
out.print(jsonArrayForDataTable.toString().replace('_',' '));
out.close();
...
Any help is highly appreciated
You're using ajax, right? It's hard to tell since you're setting that request attribute, as if about to forward to a jsp, but then you're writing directly to the response. If this is ajax, you can compress the response with gzip if the client accepts it. You'll have to try it to know if it will speed things up in your situation. You'll be reducing the size of the response by a ratio that depends on the original content, but you'll be increasing the amount of processor work on the server (and on the client, which has to decompress the response).
ServletOutputStream out = response.getOutputStream();
response.setContentType("application/json");
String strVal = jsonArrayForDataTable.toString().replace('_',''));
if (request.getHeader("Accept-Encoding") != null && request.getHeader("Accept-Encoding").contains("gzip"))
{
ByteArrayOutputStream baos = new ByteArrayOutputStream(strVal.length());
GZipOutputStream gzip = new GZIPOutputStream(baos);
gzip.write(strVal.getBytes());
gzip.close();
response.setHeader("Content-Encoding", "gzip");
out.write(baos.toByteArray());
baos.close();
}
else
{
out.print(strVal);
}
On the client side, you have to be prepared for a gzipped response by setting the Accept-Encoding XMLHttpRequest header to gzip. The browser will decompress it into the json string.
Related
I am automating Adobe InDesign to create documents using JSON data gathered from a web API with a SQL Server backend. I am using the Sockets object to make an HTTP 1.0 call to our server. Sometimes the response received is missing about 1700 characters from various points within the JSON string, yet when I call the same API endpoint using curl or Postman I get a complete and valid response.
The response should be about 150k characters long, and I'm using conn.read(99999999) to read it. In addition, the appearance of the end of the string looks correct, so I don't believe it's any kind of truncation problem.
The problem only seems to occur when I request a UTF-8 encoding. If I request ASCII I get a complete and valid response, but missing various Unicode characters. If I request BINARY I get a complete and valid response but the JavaScript/ExtendScript seems to be handling any multi-byte Unicode characters received as individual bytes, rather than as the Unicode characters we want to display.
Here is an illustration of the behavior I'm seeing, using bogus data...
"Expected" response...
[{"Id":1, "name":"Random Name", "Text":"A bunch of text", "AnotherId": 1}]
"Actual" response...
[{"Id":1, "name":"Random Name", "Text":"A bunc": 1}]
The problem first manifested itself as a JSON2 parsing error, for obvious reasons, but the root of it seems to be the fact that parts of the data are going missing in-transit.
So far we've only seen this problem when making the call using the InDesign Sockets object, and not every response exhibits this behavior.
Any help or insights you could offer would be appreciated.
Here is the function I'm using to call for data...
function httpRequest(url, encoding) {
try {
var response = "";
var hostName = getHostFromUrl(url);
var pathAndQuery = getPathAndQueryFromUrl(url);
var httpGet = "GET ";
httpGet += pathAndQuery;
httpGet += " HTTP/1.0\r\nHost: ";
httpGet += hostName;
httpGet += "\r\n";
var conn = new Socket;
conn.timeout = 30;
//conn.encoding = encoding || "UTF-8";
//conn.charset = "UTF-16";
if (conn.open(hostName + ":80", encoding || "UTF-8")) {
// send a HTTP GET request
conn.writeln(httpGet);
// and read the server's response
response = conn.read(99999999);
conn.close();
}
return parseHttpResponse(response);
}
catch (e) {
$.writeln(e);
$.global.alert("There was a problem making an HTTP Request: " + e);
return null;
}
}
It turns out my handling of the HTTP response was too simplistic and needed extra logic to handle Unicode characters properly.
The solution, in my case, was to use the GetURL method made available by Kris Coppieter here.
Im trying to figure out which part of my app (Xamarin Forms and proxy written in PHP) is buggy. Firstly I thought that my proxy (written in PHP) is working incorrectly with long set of data (ie. json containing 1.300.000 characters) and returns malformed response, but every single request with Postman gives me correct JSON, which is successfully decoded with third-party tools. So I assume, proxy is working well.
The problem is (I guess) with decoding response in my Xamarin Forms (2.0.0-beta.22) app. I'm using HttpClient to read response with this code:
response.EnsureSuccessStatusCode();
var entries = new List<HistoryEntry>();
var content = await response.Content.ReadAsStringAsync();
_loggerService.Error(content);
response is just GetAsync response from HttpClient. The problem is: content is randomly incomplete/malformed. Saying this I mean last character is missing (}) or JSON keys/values have additional " character, which breaks everything. Unfortunately, I can make exactly the same requests many times and once it works, once not. I found out that this behavior happens only with large set of data (as I mentioned before, long JSON string).
Is there any possibility that ReadAsStringAsync does not wait for full response or in any way alters my response string? How can I find the reason of wrongly downloaded data?
EDIT 21.05.2019:
Just copied valid JSON (available here: https://github.com/jabools/xamarin/blob/master/json.txt) and returned it from Lumen app by response()->json(json_decode(..., true)) and still the same result. Hope someone will be able to reproduce this and help me with this issue :( More informations in comments.
Used this code in C#:
Device.BeginInvokeOnMainThread(async () =>
{
for (int i = 0; i < 10; i++)
{
var response = await client.GetAsync("<URL_TO_PHP>");
//var response = await client.GetAsync("https://jsonplaceholder.typicode.com/photos");
var content = await response.Content.ReadAsAsync<object>();
Debug.WriteLine("Deserialized: " + i);
}
});
I am trying to pass parameters to a server and extract the report in csv format. So the code i have has PUT/GET/POST in the order. I could get GET and POST work, but when i add PUT there is no error just blank screen.
String output1 = null;
URL url = new URL("http://<servername>/biprws/raylight/v1/documents/12345/parameters");
HttpURLConnection conn1 = (HttpURLConnection) url.openConnection();
conn1.setRequestMethod("PUT");
conn1.setRequestProperty("Accept", "application/json");
conn1.setRequestProperty("Content-Type", "application/json; charset=utf-8");
conn1.setDoInput(true);
conn1.setDoOutput(true);
String body = "<parameters><parameter><id>0</id><answer><values><value>EN</value></values></answer></parameter></parameters>";
int len1 = body.length();
conn1.setRequestProperty("Content-Length", Integer.toString(len1));
conn1.connect();
OutputStreamWriter out1 = new OutputStreamWriter(conn1.getOutputStream());
out1.write(body, 0, len1);
out1.flush();
What i am trying to do is pass parameter EN to the report and refresh it, take the output in csv using GET. POST is used for login to the server. I could make GET and POST work and get the output in CSV but not refreshed one.
Appreciate very much any help here.
Thanks,
Ak
What is the response code from the server when using PUT?
A PUT may not actually return a body to display on the screen; often times a PUT will only return a 200 or 204 response code. 204 would clearly mean that the server took the data and applied it, but is not sending you anything back, 200/201 may include a response, but maybe not. It depends on the folks who implemented the API.
https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html (section 9.6)
Should a RESTful 'PUT' operation return something
I am uploading a file to the server using BackgroundTransferRequest object. I need to access the response header sent from the server. Based on the research, I understand that there is no direct access to the Response of the request and one possible work around is to provide a download location to which the response content will be written.
However, when I try to do that, there is no data written to that file and its 0 bytes. My request server is actually sending a response having ONLY the headers (without any content).
So my question is, does the response have to have body in order to get the response written to the DownloadLocation?
My code sample:
var request = new BackgroundTransferRequest(targetUri)
{
DownloadLocation = new Uri(downloadTo, UriKind.Relative),
UploadLocation = new Uri(uploadFrom, UriKind.Relative),
Method = "POST"
};
I've got compression propperly configured for my Azure web role. Both .aspx pages and static pages like *.css are being compressed correctly.
<urlCompression doStaticCompression="true" doDynamicCompression="true" dynamicCompressionBeforeCache="true" />
I've got several different [System.Web.Services.WebMethod]'s though, that are not returning GZIP'd data. The size of each request is around 350KB, so I'm thinking it should be quite a bit faster if I can get this to work.
Within my webMethod, I create a list of objects, return the objects, and I assume some type of built in serializer turns this into JSON?
Is there anyway to force this content to be compressed?
Thanks so much!
I've seen people have issues with built in Compression for numerous reasons
The simplest way is to use a third party component such as Telerik's RadCompression to enforce compression on the response to AJAX calls.
Alternatively, you can override the application's BeginRequest method or write your own handler
to pack up the responses on the fly. A basic VB version of how to do this is here:
Sub Application_BeginRequest(...)
If Request.RawUrl.Contains(".aspx") And _
Not Request.Headers("Accept-Encoding") Is Nothing Then
If Request.Headers("Accept-
encoding").ToLower().Contains("gzip") Then
Response.Filter = New GZipStream(Response.Filter,CompressionMode.Compress, True)
Response.AppendHeader("Content-encoding", "gzip")
' Else...attempt deflate if GZip is not allowed
End If
End If
End Sub
I've done a method with the handler as well (and that's what I believe Telerik's RadCompression uses), but it is a good bit more complicated as you have to modify the response size, etc.
Here's what I ended up with, a variation Yak's answer.
HttpApplication app = (HttpApplication)sender;
HttpRequest request = app.Request;
HttpResponse response = app.Response;
System.Web.HttpApplication Appl = (System.Web.HttpApplication)sender;
HttpContext context = Appl.Context;
string origpath = context.Request.Url.AbsolutePath;
//Ajax Web Service request is always starts with application/json
if (request.ContentType.ToLower(CultureInfo.InvariantCulture).StartsWith("application/json"))
{
//User may be using an older version of IE which does not support compression, so skip those
if (!((request.Browser.IsBrowser("IE")) && (request.Browser.MajorVersion <= 6)))
{
string acceptEncoding = request.Headers["Accept-Encoding"];
if (!string.IsNullOrEmpty(acceptEncoding))
{
acceptEncoding = acceptEncoding.ToLower(CultureInfo.InvariantCulture);
if (acceptEncoding.Contains("gzip"))
{
response.Filter = new GZipStream(response.Filter, CompressionMode.Compress);
response.AddHeader("Content-encoding", "gzip");
}
else if (acceptEncoding.Contains("deflate"))
{
response.Filter = new DeflateStream(response.Filter, CompressionMode.Compress);
response.AddHeader("Content-encoding", "deflate");
}
}
}
}