I've been trying to construct a simple HTTP request to post all process data in JSON format to a server with PowerShell 2.0, which I need to use out of necessity. Unfortunately, every time I execute my script on a Windows 7 OS, I notice that the the JSON payload is cut off (very frustrating). The actual payload is very large at around 74,100 bytes in the request body. It is also probably useful to note that this request works successfully on a Windows Server 2012 machine, though I suspect that the payload request size for that particular OS is smaller.
Here is the code:
$request = [System.Net.HttpWebRequest]::Create($endpoint);
$request.ContentType = 'application/json';
$request.Method = 'POST';
$request.Headers.Add('Authorization', "Token " + $authToken);
$seconds = 1000;
$request.Timeout=3*$seconds;
$stringContentBytes = [System.Text.Encoding]::UTF8.GetBytes($stringContent);
$request.ContentLength = $stringContentBytes;
$requestStream = $request.GetRequestStream();
$requestStream.Write($requestContentBytes, 0, $requestContentBytes.length);
$requestStream.Close();
$response = $request.GetResponse();
In this example $endpoint represents the target server endpoint, $stringContent represents the JSON-serialized payload that needs to be sent over the wire, and authToken is a string validation token.
I'm currently pointing $endpoint to a service called Request Bin, which very simply details the HTTP request information in a pretty, readable format. So, I know my server can't be causing the payload split.
Also, I've read through various posts about C# and PowerShell and they all mentioned this should be related to not closing the stream. However, as you can see in the code, I'm very clearly executing $requestStream.Close(), which from my research should be able to successfully flush the request stream.
Does anyone know why the output stream would not be processing all the JSON serialized information?
Whoops, I just realized that the Request Bin service that I was using (in fact) was limiting the request body!
Related
I have a http request I am trying to make on an afterSave method in my Cloud Code. I have been able to create my request, and when I console.log(response) it outputs a block that contains the information that I am after. I am aware that response.text is a string so I am trying to run JSON.parse(response.text) so I can access my API response.
I can print out what appears to be an object after running JSON.parse, but much of the data within my response is stripped out. I know it is not the fault of the API because I have another function that runs on the client with the same query and it works correctly.
What is the correct way to parse the response.text from a Parse.Cloud.httpRequest to maintain my data.
Try var result = JSON.parse(response['text']).
I'm trying to send JSON content to the remote endpoint as a POST request over HTTPS using Netty 5.0.0.Alpha2. My problem is that the new API doesn't seem to offer a way of simply setting the JSON content as the request body without using an HttpPostRequestEncoder object and specifying a name for the JSON attribute. Is it possible to set the content "anonymously", and if so, how?
I was able to make the request successfully by using the concrete DefaultFullHttpRequest handle instead of the HttpRequest interface. The code now looks a bit like this (edited for brevity):
DefaultFullHttpRequest request = new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.POST, "/path/to/x");
request.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE);
request.headers().set(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.GZIP + "," + HttpHeaderValues.DEFLATE);
request.headers().set(HttpHeaderNames.CONTENT_TYPE, "application/json");
request.headers().set(HttpHeaderNames.ACCEPT, "application/json");
request.headers().set(HttpHeaderNames.HOST, "somehost.com:443");
ByteBuf buffer = request.content().clear();
int p0 = buffer.writerIndex();
buffer.writeBytes(json.getBytes());
int p1 = buffer.writerIndex();
request.headers().set(HttpHeaderNames.CONTENT_LENGTH, Integer.toString(p1 - p0));
I'm sure this could be prettier (in particular the content-length calculation), but it seems to work and saves anyone wasting any brain cycles on this question.
Thanks if you happened to consider it.
Could you please explain Square's Retrofit response parsing logic.
I'm interested in case when we should receive & parse a big json (>100Kb) - will Retrofit wait while all content will be received from server and only than parse it, or it will start to parse it immediately while getting stream data?
My goal is to speedup response processing.
Are there any options about it available to configure?
As soon as the HTTP client parses the headers, the InputStream will be handed back to Retrofit which will then hand it directly to the Converter. This means that as the underlying converter mechanism (say, Gson) is pulling bytes they are being read (and potentially blocking) directly from the network.
Note: this is only true if logging is off (as it should be in production / release builds). When logging is turned on beyond HEADERS level, the response body has to be read in its entirety into a byte[] in order to both log and hand the data to the converter.
I have successfully created my app and now want to connect it to a localhost to check the working of my app. I have been suggested to use restsharp for connecting to the server using php and json to receive data from server.
I have looked at codes for both but do not completely understand how the process works. I have looked into all forums but found could snippets with no explanation as how it works. I have even tried restsharp.org and google search results. Please explain me as to how this works.
RestSharp is a library that helps you invoking REST web services.
You use RestSharp on your client to invoke Rest style Web Services (send and receive data)
Here is an example on the usage of your service:
var client = new RestClient(baseUrl);
var request = new RestRequest("/*rest_resource*/", Method.POST);
// see Rest services
// set the request format - HTTP Content-Type text/xml
request.RequestFormat = DataFormat.Xml;
// add data to the request
request.AddBody("<books><book>RestSharp Book</book></books>");
/* send the request and if your service returns text put the as expected return type; otherwise you will get raw byte array*/
IRestResponse response = client.Execute(request);
//HTTP status code 200-success
Assert.IsTrue(response.StatusCode == HttpStatusCode.OK);
Assert.IsTrue(!string.IsNullOrEmpty(response.Data)); // the response is not empty
So, in playframework, I can stream any response back so when i get a json request, I can do http chunking and stream the response back for some really really large responses. I was wondering if the same can be done on POST calls. If a client has a very very large POST call, can they stream me the request? Is this possible with html?
That said, if I can't do that, i need an api that curl or some other non-browser client will use to upload a file(the json request, or a csv, etc). How to create such an api?
I should note that I canNOT receive the whole request at once or will get out of memory. I need to receive pieces and as I receive pieces put that to the backend datastore a piece at a time.
Also, what would be the curl syntax to make sure it is streaming the file rather than sending it in one huge huge request that would break the server? How to force the client to stream the file in?
thanks,
Dean
You can get full control over HTTP request processing by using an EssentialAction. An EssentialAction processes the request body and returns a Result.
Normal Play Actions are a special case of EssentialAction. Actions process request bodies and return Results too, but they always perform their processing in two steps. Actions first parse the request body. Then the Actions parsethe parsed value to a function to get a Result. For you, having a separate parsing step is a problem because it means that the parsed value needs to be stored in memory.
If you use an EssentialAction then you can avoid storing a parsed value in memory, because you can just process the request body as it arrives.
What you need to do is add a method to your controller that returns an EssentialAction.
The signature of EssentialAction is:
trait EssentialAction extends (RequestHeader) ⇒ Iteratee[Array[Byte], SimpleResult]
The EssentialAction needs to accept the request header and then return an iteratee to process the request body. The iteratee will incrementally process the request body as it arrives. You can use your iteratee to put each piece into your data store as each piece arrives. When you're done processing all the pieces you can return a Result.
More info here: http://www.playframework.com/documentation/2.2.x/HttpApi.