Extracting gzip data from Apache-httpclient without decompression - apache-httpclient-4.x

I'm using apache httpclient 4.3.5 to send a request to an upstream server which returns a gzipped response. I need to pass this response AS-IS to a downstream server without any form of decompression. However, httpclient is far too helpful and insists on decompressing the response and I can't find any way of persuading it to stop.
CloseableHttpClient client = HttpClients.createDefault();
CloseableHttpResponse serverResponse = client.execute(serverRequest);
try {
HttpEntity entity = serverResponse.getEntity();
downstreamResponse.setStatus(serverResponse.getStatusLine().getStatusCode());
for (Header header : serverResponse.getAllHeaders()) {
downstreamResponse.setHeader(header.getName(), header.getValue());
}
entity.writeTo(downstreamResponse.getOutputStream());
downstreamResponse.flushBuffer();
} finally {
serverResponse.close();
}
I'm sure that there is some way of configuring the client using some form of the construct
return HttpClients.custom()
....
.build();
but I can't find it. Can the experts please advise?

CloseableHttpClient client = HttpClients.custom()
.disableContentCompression()
.build();

Related

How can I prevent API returning response as gibberish data instead of JSON?

I am using RapidAPI for integration of https://priceline-com-provider.p.rapidapi.com/v1/hotels/search external API with my asp.net micro-service. I need to get a JSON response from the API however it is returning gibberish data. I have tried many things to no success.
To make sure it is good the request, I used Postman to test and it worked out fine.
What can I do to solve this?
In my case, the response was Accept-Encoding: deflate/gzip and therefore I had to insert a HttpClientHandler into the HttpClient with the correct settings:
using (var HttpClientHandler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
})
{
using (var client = new HttpClient(HttpClientHandler))
{
}
}
Once I did, the await response.Content.ReadAsStringAsync() returned deserializable JSON data.
This answer led me to the solution: https://stackoverflow.com/a/69222690/2504659

Make a NestJS route send in response a pretty formatted JSON

I have a NestJS route which sends back in response, a JSON not well formatted (like minified),
I want to make this JSON easier to read, like a JSON prettier or JSON formatted,
Do someone knows how to do it in NestJS ? I accept answers for other NodeJS frameworks like Express, maybe it will work in NestJS too...
Prettifying the JSON response should be the responsibility of the client, not the server. Otherwise you could be sending a lot of white space which will bloat the response size and could lead to a crash due to having a response too large. If you are using something like Postman, it should be easy to prettify it, I think Postman might do it by default. If you are looking at the response in some web browser, you could use JSON.parse() on the response and it should make the response an actual JSON which console.log() would then print in a pretty way.
You should try https://www.postman.com or https://insomnia.rest/. It can save you a lot of time when it comes to testing an API.
While you shouldn't do it in prod as mentioned above, there's number of cases where it makes a lot of sense (e.g. in dev env). You can achieve this in a bit hacky way:
Access express instance inside nest through breaking abstraction. It's not exposed on INest interface, so you'll need to cast it to any type to bypass Typescript check
Set undocumented express property "json spaces", which will set formatting for all JSON responses over the app
const app = await NestFactory.create(AppModule);
if (process.env.NODE_ENV === 'development') {
(app as any).httpAdapter.instance.set('json spaces', 2);
}
await app.listen(3000);
It works for me:
// src/main.ts
async function bootstrap() {
const app = await NestFactory.create(AppModule);
if (process.env.NODE_ENV !== 'production') {
app.getHttpAdapter().getInstance().set('json spaces', 2);
}
await app.listen(process.env.PORT);
}
Define return type string on your controller
Set the Content-Type: application/json response header
Use JSON.stringify to format your object with whitespace
Example controller code:
import { Controller, Get, Header } from '#nestjs/common';
#Controller('diagnostics')
export class DiagnosticsController {
#Get()
#Header('Content-Type', 'application/json')
findAll(): string {
const statusCode = 200;
const statusText = 'OK';
const info = {
self: 'NestJS Diagnostics Report',
status: {
statusCode,
statusText,
},
};
return JSON.stringify(info, null, 2);
}
}

Why is my HTTP response both 502 on my side but 401 on the server?

I am making an HTTP POST of a JSON from an Android app to my server (server code was not set by myself).
Most of the times this works correctly and I get a response of 200. However sometimes I get a 502 error code on the phone, but it logs as a 401 error code on the server.
The JSON is generated using unknown data from another source, however I have reviewed the JSON and it appears to be correct. The issue has only been seen when the JSON is particularly big.
Below is the code to set credentials and attach JSON to the HTTP post:
//set httpClient
httpClient = new DefaultHttpClient(httpParameters);
httpClient.getCredentialsProvider().setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("username", "password"));
//create http post
HttpPost httpPost = new HttpPost(ServerURL);
//TODO SET THE JSON
try {
JSONObject thisObject = new JSONObject(query);
StringEntity se = new StringEntity(thisObject.toString());
httpPost.setEntity(se);
}
catch (JSONException ex)
{
longInfo("JSONException - JSON error" + ex.getMessage(), 0);
}
Why are the error codes different? Is it because the credentials are not verified by the server when it gets a 401 error?
Is there a common cause for this pairing of error codes which is known?
(NOTE the username and password are definitely correct as they are successful when sending small JSON file)

WinRT use HttpClient to call Web API based URL with token

We are building a WinRT app which gets data from server which is Web API based & so it gives data in json and/or XML format.
When app user logs in for the first time using his credentials(username,password), the response that comes from server is a success bit & a TOKEN, which should be used in successive URL requests.
I am using httpclient for sending requests
using (HttpClient httpClient1 = new HttpClient())
{
string url = "http://example.com/abc/api/process1/GetLatestdata/10962f61-4865-4e7a-a121-3fdd968824b5?employeeid=6";
//The string 10962f61-4865-4e7a-a121-3fdd968824b5 is the token sent by the server
var response = await httpClient1.GetAsync(new Uri(url));
string content = await response.Content.ReadAsStringAsync();
}
Now the response that i get is with status code 401 "unauthorised".
And the xml i get in response is "Unauthorised User".
Is there anything i need to change in appManifest??
I've checked this, but cant we use httpclient without credentials??
Your Capabilities are enough. You don't even need Internet (Client) because it's included in Internet (Client & Server).
You do not have credentials for WinRT HttpClient, in your linked post they referr to System.Net.Http.HttpClientHandler.
Maybe you can use the HttpBaseProtocolFilter to add the credentials?
using (var httpFilter = new HttpBaseProtocolFilter())
{
using (var httpClient = new HttpClient(httpFilter))
{
httpFilter.ServerCredential...
}
}
I don't know your security mechanism, I'm using a HttpClient and my session-key is in a cookie. But I think your client code looks fine.

Json Webservice Call in Apex Salesforce

Can anyone share an end to end example for making a JSON webservice Call Through Apex ( Visual Force Pages and Controllers ) in Salesforce .
Pretty Much like we do in HTML5 ,Jquery by Ajax !
There are examples right in the documentation of calling REST web services.
From HTTP Classes:
public class HttpCalloutSample {
// Pass in the endpoint to be used using the string url
public String getContent(String url) {
// Instantiate a new http object
Http h = new Http();
// Instantiate a new HTTP request, specify the method (GET) as well as the endpoint
HttpRequest req = new HttpRequest();
req.setEndpoint(url);
req.setMethod('GET');
// Send the request, and return a response
HttpResponse res = h.send(req);
return res.getBody();
}
}
You can change the method to one of:
GET, POST, PUT, DELETE, TRACE, CONNECT, HEAD, and OPTIONS
A more complete example is available at HTTP (RESTful) Services
There is also support for JSON deserialization.
Don't forget to use the Remote Site Settings to open up access to the target domain.
For a SOAP web service you can define Apex classes from a WSDL.