How to disable compression handling in Apache HttpAsyncClient - apache-httpclient-4.x

When creating a normal CloseableHttpClient I can disable compression (and redirections) by doing
httpclient = HttpClients.custom()
.disableRedirectHandling()
.disableContentCompression()
.build();
This way I receive gzip data unaltered.
Now I'm trying the same with CloseableHttpAsyncClient, but the relevant methods are missing from HttpAsyncClients.custom():
asyncHttpClient = HttpAsyncClients.custom()
.disableContentCompression() // COMPILE ERROR
.build();
I'm just at the start of evaluating the feasability or rewriting my code to the async API so I might be missing something. How is the async client going to handle compressed content? Can its behavior be customized?

HttpAsyncClient 4.x currently does not support automatic content decompression.

Related

Why do the weather samples in FetchData seem to get cached for the sample Blazor app?

The Blazor app in Visual Studio uses a Http.GetFromJsonAsync call to get the data for Weather Forecasts from a json file in wwwroot.
When I change the data in the file, I still see the same data in the table?
When I copy the file, and change the code to use the new filename, I get the changed results.
Is there some caching happening with wwwroot files? I've tried hard refresh, that doesn't make a difference, but changing browser does. I know that Blazor caches the framework files...but is this happening to all wwwroot, how do I change this behaviour?
Thanks in advance.
The fetchdata sample page (from new blazorwasm) retrieves data on initialize component:
protected override async Task OnInitializedAsync()
{
forecasts = await Http.GetFromJsonAsync<WeatherForecast[]>("sample-data/weather.json");
}
When you go out of this page and come back, initialize is running again and a request is done.
But, because this is a GET request, the browser can deliver answer from cache:
They are some ways to avoid cache on Blazor GET requests, learn about it here: Bypass HTTP browser cache when using HttpClient in Blazor WebAssembly
Also, you can use the simple trick to add a random string to query string:
protected override async Task OnInitializedAsync()
{
var randomid = Guid.NewGuid().ToString();
var url_get = $"sample-data/weather.json?{randomid}";
forecasts = await Http.GetFromJsonAsync<WeatherForecast[]>(url_get);
}
In short, it seems to get cached because a get request can be cached by browser and is the browser who retrieve the data.

Apache HttpClient 4.5: Why does the second package wait for the ack of the first package?

I am using apache http client in version 4.5.13 within my java application to send a post request. I used following line of code to set up the http client.
SocketConfig socketConfig = SocketConfig.custom()
.setSoKeepAlive(true)
.setTcpNoDelay(true)
.build();
ConnectionConfig connectionConfig = ConnectionConfig.custom()
.setMalformedInputAction(CodingErrorAction.IGNORE)
.setUnmappableInputAction(CodingErrorAction.IGNORE)
.setCharset(Consts.UTF_8)
.setMessageConstraints(messageConstraints)
.build();
RequestConfig defaultRequestConfig = RequestConfig.custom()
.setCookieSpec(CookieSpecs.DEFAULT)
.setExpectContinueEnabled(true)
.setTargetPreferredAuthSchemes(Arrays.asList(AuthSchemes.NTLM, AuthSchemes.DIGEST))
.setContentCompressionEnabled(true)
.build();
BasicHttpClientConnectionManager connectionManager = new BasicHttpClientConnectionManager();
connectionManager.setSocketConfig(socketConfig);
connectionManager.setConnectionConfig(connectionConfig);
CloseableHttpClient httpClient = HttpClients.custom()
.setConnectionManager(connectionManager)
.setDefaultRequestConfig(defaultRequestConfig)
.build();
And I am sending the data via
CloseableHttpResponse response = httpClient.execute(postRequest);
The issue I am experiencing is that when I look how the messages are send (using tshark) I can see that the application data is split in two messages. The first one leaves my system around 0.5ms after the httpClient.execute(postRequest), but the second part is send around 10ms-20ms after the first one. It looks like the second part is waiting to receive the ack for the first part of the message. I tried to change a a lot of configurations (buffer sizes, TcpNoDelay, different TLS ...) but cannot figure out what is causing this behavior.
I also tried http.net client to send post requests. With this client the message was also split in two messages but they where both send right after each other (with around 0.3ms delay).
I am pretty new to network so I would appreciate a helpful answer and apologize upfront if I did not explain it very well (I do not know all the specific wordings).
Thanks
Try disabling expect-continue handshake.

Exception: '<' is an invalid start of a value

I have a Blazor Webassembly project with a controller method as follows:
[HttpGet]
public async Task<List<string>> GetStatesForProfile()
{
IConfigurationSection statesSection = configuration.GetSection("SiteSettings:States");
var sections = statesSection.GetChildren();
var states = statesSection.GetChildren().Select(s => s.Key).ToList<string>();
return states;
}
The razor page calls this method:
private async Task<bool> GetStatesModel()
{
try
{
States = await http.GetJsonAsync<List<string>>("api/account/getstatesforprofile");
...
}
catch (Exception ex)
{
Console.WriteLine($"Exception: {ex.Message}, Inner: {ex.InnerException.Message}");
}
I get this Exception:
Exception: '<' is an invalid start of a value.
I read these values from appsettings.json file, And there is no '<' in values.
{
"SiteSettings": {
"States": {
"New York": ["NYC"],
"California": ["Los Angeles", "San Francisco"]
}
}
Also I put a breakpoint in the controller method and it doesn't hit.
What is this error? Is it from parsing json? and how to resolve this?
I had a very similar problem.
In the end it turned out that my browser had cached the HTML error page (I guess I had some problems with the code when I first tried it). And no matter how I tried fixing the code I still only got the error from cache. Clearing my cache also cleared the problem.
It happens when you're trying to access an API that doesn't exist. You have to check your API project connectionstring under AppSettings and make sure it's correct and running. If it's a Blazor project, you can set it as your default project, execute and see if you get a json response.
Most probably the response you are receiving is html instead of actual JSON format for the endpoint you are requesting. Please check that.
An as HTML usually starts with <html> tag, the JSON validator fails on the very first character.
You should also clear any cache, that might be interfering with the returned data. (this has helped people resolve this same issue)
I know this is an old question, but it's one of the top results when Googling the error.
I've just spent more time than I care to admit to tracking down this error. I had a straightforward Blazor hosted app, basically unchanged from the template. It worked just fine when run locally, but when published to my web host API calls failed. I finally figured out that the problem was that I was running the publish from the Client project. When I changed to the Server project it worked properly.
Hopefully my long frustration and slight stupidity will save someone else making a similar mistake.
Seems like your api is not not accessible and its returning error HTML page by default.
You can try below solution:-
I think you are using httpclient to get data to blazor application.
If you have separate projects in solution for blazor and web api,
currently your startup application may set to run blazor project only.
Change startup projects to multiple (blazor and web api app) and give httpClient url in startup of blazor application, as webApi application url, that may solve your issue.
This error indicates a mismatch of the project targeting framework version and installed runtime on the machine. So make sure that the target framework for your project matches an installed runtime - this could be verified by multiple means; one of them is to check out the Individual Components tab of the Visual Studio Installer and lookup the target version.
E.g., there is the TargetFramework attribute in the proj file:
<PropertyGroup>
<TargetFramework>net5.0</TargetFramework>
</PropertyGroup>
Then launch the Visual Studio Installer, click Modify, and visit the Individual Components tab:
Install the missing runtime (.NET 5 Runtime in this case) and you're good to go.
I got the same error. Red herring. use your browser or postman to check your api endpoint is returning the json data and not some HTML. In my case my "api/companytypes" had a typo.
private CompanyType[] companytypesarray;
private List<CompanyType> CompanyTypeList;
private List<CompanyType> CompanyTypeList2;
public async Task<bool> LoadCompanyTypes()
{
//this works
CompanyTypeList = await Http.GetFromJsonAsync<List<CompanyType>>("api/companytype");
//this also works reading the json into an array first
companytypesarray = await Http.GetFromJsonAsync<CompanyType[]>("api/companytype");
CompanyTypeList2 = companytypesarray.ToList();
return true;
}
I know this is an old question, but I had the same problem. It took some searching, but I realized that the return data was in XML instead of JSON.
I'm assuming your "http" variable is of type HttpClient, so here's what I found worked for me.
By setting the "Accept" header to allow only JSON, you avoid a miscommunication between your app and the remote server.
http.DefaultRequestHeaders.Add("Accept", "application/json");
States = await http.GetJsonAsync<List<string>>("api/account/getstatesforprofile");
I had the same issue when passing in an empty string to a controller method. Creating a second controller method that doesn't accept any input variables, and just passing an empty string to the first method helped to fix my problem.
[HttpGet]
[ActionName("GetStuff")]
public async Task<IEnumerable<MyModel>> GetStuff()
{
return await GetStuff("");
}
[HttpGet("{search}")]
[ActionName("GetStuff")]
public async Task<IEnumerable<MyModel>> GetStuff(string search)
{
...
}
Versions of package
Try to update your packages to old or new version. In my case, system.net.http.json is updated from 6.0 to 5.0
Likely you are using an Asp.NetCore hosted WASM application. By default the client's App.razor has something similar to:
<CascadingAuthenticationState>
<Router AppAssembly="#typeof(Program).Assembly">
<Found Context="routeData">
<AuthorizeRouteView DefaultLayout="#typeof(MainLayout)"
RouteData="#routeData">
<NotAuthorized>
<RedirectToLogin />
</NotAuthorized>
<Authorizing>
<Loading Caption="Authorizing..."></Loading>
</Authorizing>
</AuthorizeRouteView>
</Found>
<NotFound>
<LayoutView Layout="#typeof(MainLayout)">
<p>Sorry, there's nothing at this address.</p>
</LayoutView>
</NotFound>
</Router>
</CascadingAuthenticationState>
Herein lies the problem. Since the Client and Server share the same base address, when the application cannot find "api/account/getstatesforprofile" it gives you the client's "Sorry, there's nothing at the address" page. Which is of course HTML.
I have not found the solution to this issue, but I am working on it and will reply once I find an issue.
I was having the same problem,
"JsonReaderException: '<' is an invalid start of a value."
In my case the url for the REST service was wrong.
I was using the URL from the client project. Then I looked at the Swagger screen,
https://localhost:44322/swagger/index.html
and noticed the right URL should start with "44322"...
Corrected, worked.
In my case, I had a comma (,) written mistakenly at the beginning of the appsettings.json file ...
Just check your file and verify
///////
my error details
//////
System.FormatException HResult=0x80131537 Message=Could not parse the JSON file.
Source=Microsoft.Extensions.Configuration.Json StackTrace: at line 16 This exception was originally thrown at this call stack: [External Code] Inner Exception 1: JsonReaderException: ',' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
////
For me, most of the time it is the #lauri-peltonen answer above. However, now and again, depending on who wrote the controller I have found that this will work in Swagger but not when you call it via the client (at least in this Blazor project we are on.)
[HttpGet]
[Route("prog-map-formulations")]
public async Task<List<GetProgramMapFormulationsResult>> GetProgramMapFormulations(int formulationId)
{
...
}
It sends the request as:
api/formulation-performance-program-map/analytical-assoc-values?formulationId=1
And I get results in Swagger but failes with the '<' OP error.
When I change ONLY the route to:
[HttpGet]
[Route("prog-map-formulations/{formulationId:int}")]
public async Task<List<GetProgramMapFormulationsResult>> GetProgramMapFormulations(int formulationId)
{
...
}
It sends the request as:
api/formulation-performance-program-map/analytical-assoc-values/1
And this works in both Swagger as well as from the Client side in Blazor.
Of course, once updated, I did have to clear the cache!
If you delete "obj" folder in your directory then clean the solution and rebbuild it the exception will be resolved
In all these, there is two things that was my issue and realized, first off was that Route[("api/controller")] instead of Route[("api/[controller]")], that is missing square brackets. In the second exercise I was doing, with the first experience in mind, was from the name of the database. The database had a dot in the name (Stock.Inventory). When I change the database name to StockInventory it worked. The second one I am not so sure but it worked for me.

How to check NON-html responses with Geb?

Some of requests to my Web Application return data not in HTML format (JSON).
How to handle this correctly?
I wrote the following page definition:
import com.fasterxml.jackson.databind.ObjectMapper
import geb.Page
class JsonResponse extends Page {
static url = null;
static at = {
true;
}
static ObjectMapper mapper = new ObjectMapper();
static content = {
readTree {
def jsonString = $("pre").text();
mapper.readTree(jsonString)
}
}
}
and it apparently works. But the question is, how correct it is?
It takes data from inside pre tag. This is because I saw this content inside driver.pageSource. Is this correct? May be it is driver-dependent?
I put null into url, since the page has different url depending on query. Is this correct?
Geb is not intended to be used to interact with HTTP API endpoints because it is built on top of WebDriver and hence expects to be used via a browser and on HTML pages.
If you want to test HTTP API endpoints then I would suggest using an http client to back your tests. There are many of them out in the wild, just to name a few in no particular order:
Apache HttpClient
RestAssured - this one is specifically for testing
Ratpack's TestHttpClient - again intended to be used for testing
OkHttp
Unirest
I was able to download the content of a PDF in a geb unit test using the Direct Download API. It is handy because it takes all the cookies from the session but does the download separately from the browser.
Example from that documentation:
Browser.drive {
    go "http://myapp.com/login"
 
    // login
    username = "me"
    password = "secret"
    login().click()
 
    // now find the pdf download link
    def downloadLink = $("a.pdf-download-link")
 
    // now get the pdf bytes
    def bytes = downloadBytes(downloadLink.#href)
}
There are different methods for downloading different kinds of data. See the DownloadSupport API docs.
Because geb uses HttpsURLConnection to connect to an https endpoint instead of using the browser you can have problems with self-signed certificates. I solved that using this Stack Overflow answer.
I agree that Geb is not intended to be used to interact with HTTP API endpoints, but there are some contexts where this can be helpful, so add this snippet here for posterity:
when: 'I retrieve the JSON transaction list'
go '/transaction/transactionList'
then: 'Valid data is retrieved'
JsonSlurper jsonSlurper = new JsonSlurper()
Map<String, List> transactionList = jsonSlurper.parseText(driver.pageSource)
assert transactionList.categories.class == ArrayList

upload images with html5's drag 'n' drop. server side?

I am implementing a drag'n'drop code for uploading images. I'm a newbie on this technology/API.
I'm using the drag'n'drop API of the HTML5. I also use Apache as http server and node.js as websocket server. I found lots of tutorials for the client side implementation.
This is maybe a silly question, but what about the server side? I guess I have to implement a code on server side to handle the incoming image's upload and storage...
Any ideas?Or links?
Thanks
EDIT
I will use these APIs : drag'n'drop, FormData, XHR progress event and FileReader. I will create code based on this tutorial. I have not implemented anything specific yet, I'm still experimenting.
The file saving on the server side is used as normally!
After using drag&drop you will get the file-objects in javascript and you can do what you want with them, either XHR-post them, use them in a normal form-POST or convert to a data-URI and post it to the server as base64 text.
i.e
element.ondrop = function(ev) {
var files = ev.dataTransfer.files
// post the files via XHR POST
var formData = new FormData()
// im lazy, use a supported loop
for (file of files) {
formData.append("file", file)
}
var req = new XMLHttpRequest()
req.open('POST', '/saveimage/')
req.send(formData)
}
Will post the files just as you had used a normal non-drag and drop action.
For a better answer on how to save files we need some info about what language and framework your website uses.