Need only the stream for response while using Apache Async http client - apache-httpclient-4.x

Question

I am using Apache Async Http Client to download huge files from Azure Storage.
This is the sample code I am using ->
CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
httpclient.execute(request, new FutureCallback<org.apache.http.HttpResponse>() {
#Override
public void completed(final org.apache.http.HttpResponse httpResponse) {
}
#Override
public void failed(final Exception e) {
future.completeExceptionally(e);
}
#Override
public void cancelled() {
future.completeExceptionally(new Exception("Request cancelled"));
}
});
But this is storing the file in a local buffer before invoking completed callback .
I tried using AsyncByteConsumer ->
AsyncByteConsumer<org.apache.http.HttpResponse>
consumer = new AsyncByteConsumer<org.apache.http.HttpResponse>() {
#Override
protected void onByteReceived(ByteBuffer buf, IOControl ioctrl) throws IOException {
}
#Override
protected void onResponseReceived(org.apache.http.HttpResponse response) throws HttpException, IOException {
}
#Override
protected org.apache.http.HttpResponse buildResult(HttpContext context) throws Exception {
return null;
}
};
This also did not work for me.
I am getting the following error ->
java.lang.IllegalStateException: Content has not been provided
All I want is to get the stream for the response which I will pass on to my clients so that they can download the file directly using the stream.
EDIT 1 ->
So I extended AbstractAsyncResponseConsumer to write my own consumer ->
public abstract class MyConsumer extends AbstractAsyncResponseConsumer<HttpResponse> {
private volatile HttpResponse response;
private volatile SimpleInputBuffer buf;
public MyConsumer() {
super();
}
#Override
protected void onResponseReceived(HttpResponse response) {
this.response = response;
}
#Override
protected void onContentReceived(ContentDecoder decoder, IOControl ioctrl) throws IOException {
Asserts.notNull(this.buf, "Content buffer");
System.out.println("onContentReceived");
buf.consumeContent(decoder);
}
protected abstract void onEntitySet(HttpResponse httpResponse);
#Override
protected void onEntityEnclosed(HttpEntity entity, ContentType contentType) throws IOException {
System.out.println("onEntityEnclosed");
long len = entity.getContentLength();
if (len > Integer.MAX_VALUE) {
throw new ContentTooLongException("Entity content is too long: " + len);
}
if (len < 0) {
len = 4096;
}
this.buf = new SimpleInputBuffer((int) len, new HeapByteBufferAllocator());
this.response.setEntity(new ContentBufferEntity(entity, this.buf));
onEntitySet(this.response);
}
#Override
protected HttpResponse buildResult(HttpContext context) throws Exception {
System.out.println("buildResult");
return response;
}
#Override
protected void releaseResources() {
}
}
Here is the code I am using to execute a http request
CompletableFuture<HttpResponse> makeRequest() {
HttpAsyncRequestProducer producer3 = HttpAsyncMethods.create(request);
CompletableFuture<HttpResponse> future = new CompletableFuture<>();
httpclient.execute(producer3, new MyConsumer() {
#Override
protected void onEntitySet(HttpResponse httpResponse) {
future.complete(httpResponse);
}
},
new FutureCallback<HttpResponse>() {
#Override
public void completed(HttpResponse result) {
System.out.println("Completed" + result);
}
#Override
public void failed(Exception ex) {
}
#Override
public void cancelled() {
}
});
}
makeRequest().thenAccept((HttpResponse httpResponse) -> {
try {
System.out.println(IOUtils.toString(httpResponse.getEntity().getContent()));
} catch (IOException e) {
e.printStackTrace();
}
});
I am getting this output ->
onEntityEnclosed .
java.io.IOException: Underlying input stream returned zero bytes .
I am completing the response future as soon as I am getting the onResponseReceived callback which returns the status and headers of the response.
What I believe should happen is that onContentReceived callback will be invoked in a separate thread which will write the buffer data to the stream and my caller thread can read it in a separate thread.

Answer 1

I am not sure why you are having those problems but the following code snippet works for me (using HttpAsyncClient 4.1.3)
CloseableHttpAsyncClient httpClient = HttpAsyncClients.createDefault();
httpClient.start();
final Future<Void> future = httpClient.execute(
HttpAsyncMethods.createGet("http://httpbin.org/"),
new AsyncByteConsumer<Void>() {
#Override
protected void onByteReceived(final ByteBuffer buf, final IOControl ioctrl) throws IOException {
System.out.println(buf.remaining());
}
#Override
protected void onResponseReceived(final HttpResponse response) throws HttpException, IOException {
System.out.println(response.getStatusLine());
}
#Override
protected Void buildResult(final HttpContext context) throws Exception {
return null;
}
},
null);
future.get();
httpClient.close();
console >
HTTP/1.1 200 OK
1060
8192
3877