binding network interface to Apache httpclient - apache-httpclient-4.x

I am using Apache HttpClient on a machine which has two network cards. I can not find how I can bind HttpClient to use one of the NICs only. I have found some solutions but they are all depreciated now. I am using Apache HttpClient 4.5.2
Are there any examples that use GET/POST requests while using NIC binding?

Arya , you will have to get the list of network interfaces and use the RequestBuilder interfaces to get this accomplished. The following will give you a rough cut idea.
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.util.Enumeration;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
public static void main(String[] args) throws Exception {
//Enumeration<NetworkInterface> nifs = NetworkInterface.getNetworkInterfaces();
/*if (nifs == null) {
System.err.println("Error getting the Network Interface");
return;
}*/
//A specific network interface can be obtained using getByName
NetworkInterface nif = NetworkInterface.getByName("sup0");
System.out.println("Starting to using the interface: " + nif.getName());
Enumeration<InetAddress> nifAddresses = nif.getInetAddresses();
RequestConfig config = RequestConfig.custom()
.setLocalAddress(nifAddresses.nextElement()).build();
HttpGet httpGet = new HttpGet("http://localhost:8080/admin");
httpGet.setConfig(config);
CloseableHttpClient httpClient = HttpClients.createDefault();
try {
CloseableHttpResponse response = httpClient.execute(httpGet);
try {
//logic goes here
} finally {
response.close();
}
} finally {
httpClient.close();
}
}

Related

How do i return Json array using vertx routingcontext

I'm writing api using vertx router, and i need my api to return list of json object, is it possible? Thanks in advance
Yes its possible, here is a full example with vert.x json abstraction:
import io.vertx.core.Vertx;
import io.vertx.core.json.JsonArray;
import io.vertx.core.json.JsonObject;
import io.vertx.ext.web.Router;
public class Main {
public static void main(String[] args) {
Vertx vertx = Vertx.vertx();
Router router = Router.router(vertx);
router.get("/foo")
.handler(ctx -> {
JsonArray jsonArray = new JsonArray();
jsonArray.add(new JsonObject());
jsonArray.add(new JsonObject());
ctx.response()
.setChunked(true)
.setStatusCode(200)
.end(jsonArray.toBuffer());
});
vertx
.createHttpServer()
.requestHandler(router)
.listen(8080);
}
}
In case you want to use your own library for json manipulation + pojos, you can work with strings instead when ending the request. e.g. with jackson:
router.get("/foo")
.handler(ctx -> {
List<MyObject> myObjectList = new ArrayList<>();
ObjectMapper objectMapper = new ObjectMapper();
String myObjectListJsonStr = objectMapper.writeValueAsString(myObjectList);
ctx.response()
.setChunked(true)
.setStatusCode(200)
.end(myObjectListJsonStr);
});

Spring webflux Netty: How to expose proto as json endpoints without duplication of code?

Use-case:
Developers/I, want to only implement a Protobuf implementation (binary protocol). However, I need a way to add config, so, the same implementation is exposed as rest/json api as well -- without code duplication.
I have proto endpoints exposed. I also want consumers to post json equivalent of those proto objects and return/receive json equivalent of the results with type info (Pojo?). The type info helps with OpenAPI / Swagger documentation too!
What are the most elegant/simple ways to achieve that without code duplication?
Any example github code that achieves that would be helpful.
Note: This is for webflux & netty - no tomcat.
ProtobufJsonFormatHttpMessageConverter - works for tomcat, does not work for netty. A working example code would be great.
I was messing around with this and ended up with this. Nothing else worked for me.
Using protov3 and setting a protobuf like this
syntax = "proto3";
option java_package = "com.company";
option java_multiple_files = true;
message CreateThingRequest {
...
message CreateThingResponse {
....
I can scan for the protobuf files by setting app.protoPath in my application.properties
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.google.common.reflect.ClassPath;
import com.google.protobuf.Message;
import com.google.protobuf.util.JsonFormat;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.codec.ServerCodecConfigurer;
import org.springframework.http.codec.json.Jackson2JsonDecoder;
import org.springframework.http.codec.json.Jackson2JsonEncoder;
import org.springframework.http.converter.json.Jackson2ObjectMapperBuilder;
import org.springframework.web.reactive.config.WebFluxConfigurer;
#Configuration
public class WebConfig implements WebFluxConfigurer {
#Value("${app.protoPath:com.}")
private String protoPath;
#Override
public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
configurer.defaultCodecs().jackson2JsonEncoder(
new Jackson2JsonEncoder(Jackson2ObjectMapperBuilder.json().serializerByType(
Message.class, new JsonSerializer<Message>() {
#Override
public void serialize(Message value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
String str = JsonFormat.printer().omittingInsignificantWhitespace().print(value);
gen.writeRawValue(str);
}
}
).build())
);
final ClassLoader loader = Thread.currentThread().getContextClassLoader();
Map<Class<?>, JsonDeserializer<?>> deserializers = new HashMap<>();
try {
for (final ClassPath.ClassInfo info : ClassPath.from(loader).getTopLevelClasses()) {
if (info.getName().startsWith(protoPath)) {
final Class<?> clazz = info.load();
if (!Message.class.isAssignableFrom(clazz)) {
continue;
}
#SuppressWarnings("unchecked") final Class<Message> proto = (Class<Message>) clazz;
final JsonDeserializer<Message> deserializer = new CustomJsonDeserializer() {
#Override
public Class<Message> getDeserializeClass() {
return proto;
}
};
deserializers.put(proto, deserializer);
}
}
} catch (IOException e) {
throw new RuntimeException(e);
}
configurer.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(Jackson2ObjectMapperBuilder.json().deserializersByType(deserializers).build()));
}
private abstract static class CustomJsonDeserializer extends JsonDeserializer<Message> {
abstract Class<? extends Message> getDeserializeClass();
#Override
public Message deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException {
Message.Builder builder = null;
try {
builder = (Message.Builder) getDeserializeClass()
.getDeclaredMethod("newBuilder")
.invoke(null);
} catch (Exception e) {
throw new RuntimeException(e);
}
JsonFormat.parser().merge(jp.getCodec().readTree(jp).toString(), builder);
return builder.build();
}
}
}
Then I just use the object types in the returns;
#PostMapping(
path = "/things",
consumes = {MediaType.APPLICATION_JSON_VALUE, "application/x-protobuf"},
produces = {MediaType.APPLICATION_JSON_VALUE, "application/x-protobuf"})
Mono<CreateThingResponse> createThing(#RequestBody CreateThingRequest request);
With https://github.com/innogames/springfox-protobuf you can get the responses to show in swagger but the requests still aren't showing for me.
You'll have to excuse the messy Java I'm a little rusty.
I needed to support json and the following code helped
#Bean
public WebFluxConfigurer webFluxConfigurer() {
return new WebFluxConfigurer() {
#Override
public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
ObjectMapper mapper = new ObjectMapper()
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false)
.registerModule(new ProtobufModule());
configurer.customCodecs().register(new Jackson2JsonEncoder(mapper));
configurer.customCodecs().register(new Jackson2JsonDecoder(mapper));
}
};
}
Try adding ProtoEncoder in your WebFlux config:
#EnableWebFlux
public class MyConfig implements WebFluxConfigurer {
#Override
public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
configurer.customCodecs().register(new ProtobufEncoder());
}
}
Then in your request mapping return the proto object:
#GetMapping (produces = "application/x-protobuf")
public MyProtoObject lookup() {
return new MyProtoObject();
}
Furthermore, if you want to serialize the proto object into JSON and return String, then have a look at com.googlecode.protobuf-java-format:protobuf-java-format library and JsonFormat::printToString capability (https://code.google.com/archive/p/protobuf-java-format/):
#GetMapping
public String lookup() {
return new JsonFormat().printToString(new MyProtoObj());
}
Since version 4.1 spring provides org.springframework.http.converter.protobuf.ProtobufHttpMessageConverter for reading and writing protos as Json.
However, If you are using Spring 5.x and Protobuf 3.x there is org.springframework.http.converter.protobuf.ProtobufJsonFormatHttpMessageConverter for more explicit conversion of Json.
This documentation should help you:
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/http/converter/protobuf/ProtobufHttpMessageConverter.html
https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/http/converter/protobuf/ProtobufJsonFormatHttpMessageConverter.html

Apache HTTPClient DigestAuth doesn't forward "opaque" value from Challenge

I'm trying to use Digest authentication with HTTP Client against a 3rd-party web service that I don't control.
I started out with the sample code from here:
http://hc.apache.org/httpcomponents-client-4.5.x/httpclient/examples/org/apache/http/examples/client/ClientPreemptiveDigestAuthentication.java
I got it working against httpbin.org, before attempting the next step described below.
It appears that the target 3rd-party service that I'm using requires the opaque value to be copied from the WWW-Authentication header on the initial response to the Authorization header on the next request, as described here:
https://security.stackexchange.com/questions/24425/what-is-the-opaque-field-in-http-digest-access-authentication-used-for
However, I have turned on wire-logging and stepped through the code (again this is really just the sample code linked above, no need to copy/paste it here) and I see that the opaque is NOT copied.
Any ideas what prevents it from being copied?
I even tried overriding the processChallenge method:
DigestScheme digestAuth = new DigestScheme() {
#Override
public void processChallenge(
Header header) throws MalformedChallengeException {
but it appears that any value introduced into the Parameters at this point is ignored in the next request.
Finally fixed by overriding the Authorize header explicitly, instead of relying on the internals of HttpClient to do it automatically:
package [...];
import java.util.Arrays;
import java.util.Map;
import java.util.stream.Collectors;
import org.apache.commons.io.IOUtils;
import org.apache.http.*;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.*;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.impl.auth.DigestScheme;
import org.apache.http.impl.client.*;
import org.testng.Assert;
public class DigestTest {
private static final String URL
= "https://...";
private static final String PASSWORD = ...;
private static final String USER = ...;
public static void main(String[] args) throws Exception {
new DigestTest().run();
}
public void run() throws Exception {
HttpGet httpget = new HttpGet(URL);
HttpHost target
= new HttpHost(httpget.getURI().getHost(), 443, "https");
CredentialsProvider credsProvider = new BasicCredentialsProvider();
UsernamePasswordCredentials credentials
= new UsernamePasswordCredentials(USER, PASSWORD);
credsProvider.setCredentials(
new AuthScope(target.getHostName(), target.getPort()),
credentials);
CookieStore cookieStore = new BasicCookieStore();
CloseableHttpClient httpclient
= HttpClients.custom().setDefaultCookieStore(cookieStore)
.setDefaultCredentialsProvider(credsProvider).build();
try {
DigestScheme digestAuth = new DigestScheme();
digestAuth.overrideParamter("qop", "auth");
digestAuth.overrideParamter("nc", "0");
digestAuth.overrideParamter("cnonce", DigestScheme.createCnonce());
AuthCache authCache = new BasicAuthCache();
authCache.put(target, digestAuth);
HttpClientContext localContext = HttpClientContext.create();
localContext.setAuthCache(authCache);
CloseableHttpResponse response;
response = httpclient.execute(target, httpget, localContext);
Map<String, String> wwwAuth = Arrays
.stream(response.getHeaders("WWW-Authenticate")[0]
.getElements())
.collect(Collectors.toMap(HeaderElement::getName,
HeaderElement::getValue));
// the first call ALWAYS fails with a 401
Assert.assertEquals(response.getStatusLine().getStatusCode(), 401);
digestAuth.overrideParamter("opaque", wwwAuth.get("opaque"));
digestAuth.overrideParamter("nonce", wwwAuth.get("nonce"));
digestAuth.overrideParamter("realm", wwwAuth.get("Digest realm"));
Header authenticate = digestAuth.authenticate(credentials, httpget,
localContext);
httpget.addHeader(authenticate);
response = httpclient.execute(target, httpget, localContext);
// the 2nd call is the real deal
Assert.assertEquals(response.getStatusLine().getStatusCode(), 200);
System.out.println(IOUtils
.toString(response.getEntity().getContent(), "utf-8"));
} finally {
httpclient.close();
}
}
}

NetworkOnMainThreadException when trying to read a json file

Good morning together from Germany!
I am trying to read a little json file from a webserver. When I start the app, I got the error message: android.os.NetworkOnMainThreadException.
The Target should be, that logcat is showing my json content:
{"first":"one","second":"two"}
Web server is working. When I access: 127.0.0.1/index.php, the browser shows me the upper line.
This is my Main:
package com.example.u0017007.jsonclient;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import android.app.Activity;
import android.os.Bundle;
import android.util.Log;
public class MainActivity extends Activity {
/** Called when the activity is first created. */
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet("http://127.0.0.1:80/index.php");
StringBuilder stringBuilder = new StringBuilder();
try {
HttpResponse response = httpClient.execute(httpGet);
int statusCode = response.getStatusLine().getStatusCode();
if (statusCode == 200) {
BufferedReader reader = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
String line;
while ((line = reader.readLine()) != null) {
stringBuilder.append(line);
}
Log.i(this.getClass().getSimpleName(), stringBuilder.toString());
} else {
Log.e(this.getClass().getSimpleName(), "Fehler");
}
} catch (ClientProtocolException e) {
Log.e(this.getClass().getSimpleName(), e.getMessage());
} catch (IOException e) {
Log.e(this.getClass().getSimpleName(), e.getMessage());
}
}
}
That is the error log
FATAL EXCEPTION: main
Process: com.example.u0017007.jsonclient, PID: 16115
java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.u0017007.jsonclient/com.example.u0017007.jsonclient.MainActivity}: android.os.NetworkOnMainThreadException
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2646)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2707)
at android.app.ActivityThread.-wrap12(ActivityThread.java)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1460)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:154)
at android.app.ActivityThread.main(ActivityThread.java:6077)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:866)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:756)
Caused by: android.os.NetworkOnMainThreadException
at android.os.StrictMode$AndroidBlockGuardPolicy.onNetwork(StrictMode.java:1303)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:333)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:196)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:356)
at java.net.Socket.connect(Socket.java:586)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:124)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:149)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:169)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:366)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:560)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:492)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:470)
at com.example.u0017007.jsonclient.MainActivity.onCreate(MainActivity.java:29)
at android.app.Activity.performCreate(Activity.java:6662)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1118)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2599)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2707) 
at android.app.ActivityThread.-wrap12(ActivityThread.java) 
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1460) 
at android.os.Handler.dispatchMessage(Handler.java:102) 
at android.os.Looper.loop(Looper.java:154) 
at android.app.ActivityThread.main(ActivityThread.java:6077) 
at java.lang.reflect.Method.invoke(Native Method) 
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:866) 
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:756
There are related questions that you can find and make use of them, but to give you the gist of the solutions available - you can solve them using any of the approaches given below:
You can refrain writing any Network related calls in your Main UI thread but use an AsyncTask to achieve what you want to, in it.
You can add the following code into your Main file by adding the following import as well.
The import:
import android.os.StrictMode;
The code:
if (android.os.Build.VERSION.SDK_INT > 9) {
StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
StrictMode.setThreadPolicy(policy);
}

Creating a JSON file from a url

Hi guys I have a problem creating a JSON file from a google url that i have. This is my code that im using.
import android.util.Log;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class DownloadUrl {
public String readUrl(String strUrl) throws IOException, InterruptedException {
Log.d("URLS = ",strUrl);
Thread.sleep(2000);
String data = "";
InputStream iStream = null;
HttpURLConnection urlConnection = null;
try {
URL url = new URL(strUrl);
// Creating an http connection to communicate with url
urlConnection = (HttpURLConnection) url.openConnection();
// Connecting to url
urlConnection.connect();
// Reading data from url
iStream = urlConnection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(iStream));
StringBuffer sb = new StringBuffer();
String line = "";
while ((line = br.readLine()) != null) {
sb.append(line);
}
data = sb.toString();
Log.d("downloadUrl", data.toString());
br.close();
} catch (Exception e) {
Log.d("Exception", e.toString());
} finally {
iStream.close();
urlConnection.disconnect();
}
return data;
}
}
It works fine when i throw a url that looks like this into it.
https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=40.7207523,-73.383851&radius=4828&type=bar&key=MYKEY
But when i try and throw a url that looks like this into it.
https://maps.googleapis.com/maps/api/place/details/json?placeid=ChIJe3AmoGsr6IkRuWcK1LAh-DE&key=MYKEY
I get an error: D/GooglePlacesReadTask: java.lang.NullPointerException: Attempt to invoke virtual method 'void java.io.InputStream.close()' on a null object reference
I dont know how i fix this. Any help?
Aah
you did not mention this is in android,
I presume this because you said ,
android.os.NetworkOnMainThreadException
in your comment
Android does not allow time consuming tasks on main thread,
use AsyncTask to call your function or use plain old java thread
Network on main thread exception comes when you run a networking operation on main thread .
Generally AsyncTask is used for these works but if you want to use the same code you written Just add..
StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
StrictMode.setThreadPolicy(policy);