Apache HTTP client doesn't respect HTTP status code 307 - apache-httpclient-4.x

When our Android client doest a request to our server with Apache HTTP client I want it to redirect to a new url (or more specific, an other context path) with the same HTTP method.
In my httpd.conf I sat up this rule with status code 307:
Redirect 307 /mybipper/reg /mybipperapi/old/reg
According to the status code description in Wikipedia a 307 should:
http://en.wikipedia.org/wiki/HTTP_307#3xx_Redirection
307 Temporary Redirect (since HTTP/1.1)
In this case, the request should be repeated with another URI; however, future requests can still use the original URI.[2] In contrast to how 302 was historically implemented, the request method should not be changed when reissuing the original request. For instance, a POST request must be repeated using another POST request.
But in my access log we see that HTTP client doesnt seem to respect it and executes a GET instead just as if I returned a status code 302
172.29.9.120 - - [21/Sep/2012:14:02:11 +0300] "POST /mybipper/reg HTTP/1.1" 307 248
172.29.9.120 - - [21/Sep/2012:14:02:11 +0300] "GET /mybipperapi/old/reg HTTP/1.1" 400 1016
According to Apache HTTP Client web site its a bit unclear how it should handle status code 307, but they list it at least there.
http://hc.apache.org/httpclient-3.x/redirects.html
I have a strong feeling its Apache HTTP client which doesn't implement the HTTP 1.1 protocol correctly, am I correct or have I misunderstood something?
The Apache HTTP client we use is bundled with the Android SDK. The phone I was testing on had Android SDK 15, ergo this one:
http://developer.android.com/about/versions/android-4.0.3.html

The DefaultRedirectStrategy only allows GET and HEAD to be redirected automatically. If you want to also allow POST (but not PUT or DELETE), you can switch to the LaxRedirectStrategy by doing something like:
HttpClientBuilder hcb = HttpClients.custom();
hcb.setRedirectStrategy(new LaxRedirectStrategy());
HttpClient client = hcb.build();
If you want to also follow PUT and DELETE (like we do here), you'll have to implement a custom strategy (note: we ran into a bug in the HttpClient where it appears it was trying to add a second Content-Length header when we do this, so we manually remove it. YMMV). By using this strategy, HttpClient will also support 308 redirects, something the Apache team couldn't even be bothered to include.
You can do something like this:
hcb.setRedirectStrategy(new DefaultRedirectStrategy() {
public boolean isRedirected(HttpRequest request, HttpResponse response, HttpContext context) throws ProtocolException {
Args.notNull(request, "HTTP request");
Args.notNull(response, "HTTP response");
int statusCode = response.getStatusLine().getStatusCode();
switch(statusCode) {
case 301:
case 307:
case 302:
case 308:
case 303:
return true;
case 304:
case 305:
case 306:
default:
return false;
}
}
public HttpUriRequest getRedirect(HttpRequest request, HttpResponse response, HttpContext context) throws ProtocolException {
URI uri = this.getLocationURI(request, response, context);
String method = request.getRequestLine().getMethod();
if(method.equalsIgnoreCase("HEAD")) {
return new HttpHead(uri);
} else if(method.equalsIgnoreCase("GET")) {
return new HttpGet(uri);
} else {
int status = response.getStatusLine().getStatusCode();
HttpUriRequest toReturn = null;
if(status == 307 || status == 308) {
toReturn = RequestBuilder.copy(request).setUri(uri).build();
toReturn.removeHeaders("Content-Length"); //Workaround for an apparent bug in HttpClient
} else {
toReturn = new HttpGet(uri);
}
return toReturn;
}
}
});

To expand on Cody's correct answer - if you need to follow PUT (or any other method) 307 redirects then you can alternatively extend LaxRedirectStrategy which is even easier:
hcb.setRedirectStrategy(new LaxRedirectStrategy()
{
protected boolean isRedirectable(String method)
{
return "PUT".equalsIgnoreCase(method)||super.isRedirectable(method);
}
});
However that will not fix the issue of following 308s as well. I know this is an old question but I hit the same issue just today (thanks Cody).

if you want to add 308 as well to LaxRedirectStrategy - see code below
.setRedirectStrategy(new LaxRedirectStrategy() {
#Override
public boolean isRedirected(
final HttpRequest request,
final HttpResponse response,
final HttpContext context) throws ProtocolException {
Args.notNull(request, "HTTP request");
Args.notNull(response, "HTTP response");
final int statusCode = response.getStatusLine().getStatusCode();
final String method = request.getRequestLine().getMethod();
final Header locationHeader = response.getFirstHeader("location");
switch (statusCode) {
case HttpStatus.SC_MOVED_TEMPORARILY:
return isRedirectable(method) && locationHeader != null;
case HttpStatus.SC_MOVED_PERMANENTLY:
case HttpStatus.SC_TEMPORARY_REDIRECT:
return isRedirectable(method);
case HttpStatus.SC_SEE_OTHER:
return true;
case 308:
return true;
default:
return false;
} //end of switch
}
})
.build();```

Related

ServiceStack Can not get real exception message

Server Side
public class MyServices : Service
{
public object Get(Hello request)
{
throw new InvalidOperationException("test error message");
//return new HelloResponse { Result = "Hello, {0}!".Fmt(request.Name) };
}
}
Client Side
try
{
var client = new JsonServiceClient("http://localhost:28586/");
var response = client.Get<HelloResponse>(new Hello { Name = "DHJ" });
}
catch (WebServiceException ex)
{
// ex.ErrorCode = "InvalidOperationException" // No Problem.
// ex.ErrorMessage = null // always null. Why?
}
And i saw the docs of ServiceStack like below:
Throwing C# Exceptions
In most cases you won’t need to be concerned with ServiceStack’s error handling since it provides native support for the normal use-case of throwing C# Exceptions, e.g.:
public object Post(User request)
{
if (string.IsNullOrEmpty(request.Name))
throw new ArgumentNullException("Name");
}
Default Mapping of C# Exceptions to HTTP Errors
By Default C# Exceptions:
Inheriting from ArgumentException are returned with a HTTP StatusCode of 400 BadRequest
NotImplementedException or NotSupportedException is returned as a 405 MethodNotAllowed
AuthenticationException is returned as 401 Unauthorized
UnauthorizedAccessException is returned as 403 Forbidden
OptimisticConcurrencyException is returned as 409 Conflict
Other normal C# Exceptions are returned as 500 InternalServerError
This list can be extended with user-defined mappings on Config.MapExceptionToStatusCode.
Your HelloResponse class needs a ResponseStatus property, from the Error Handling docs:
Error Response Types
The Error Response that gets returned when an Exception is thrown varies on whether a conventionally-named {RequestDto}Response DTO exists or not.
If it exists:
The {RequestDto}Response is returned, regardless of the service method's response type. If the {RequestDto}Response DTO has a ResponseStatus property, it is populated otherwise no ResponseStatus will be returned. (If you have decorated the {ResponseDto}Response class and properties with [DataContract]/[DataMember] attributes, then ResponseStatus also needs to be decorated, to get populated).
Otherwise, if it doesn't:
A generic ErrorResponse gets returned with a populated ResponseStatus property.

Should a server always send a JSON object as an http response?

I'm working on a node.js server using express and a android native app, using Retrofit 1.9.
For a login API that returns only a true/false answer to the client, should JSON still be used?
As I see it, the server has only to send a status code response:
if(isLegal) {
res.sendStatus(200);
dbConnector.updateUser(token);
}
else{
console.log('Token is not legal');
res.sendStatus(403);
}
But the Retrofit framework tries to convert the response to JSON, which makes me think I must send a JSON object with the answer, though it seems weird.
My retrofit restClient:
public class RestClient {
private static final String URL = SessionDetails.getInstance().serverAddress;
private retrofit.RestAdapter restAdapter;
private ServerAPI serverAPI;
public RestClient() {
restAdapter = new retrofit.RestAdapter.Builder()
.setEndpoint(URL)
.setLogLevel(retrofit.RestAdapter.LogLevel.FULL)
.build();
serverAPI = restAdapter.create(ServerAPI.class);
}
public ServerAPI getService() {
return serverAPI;
}
}
And usage:
restClient.getService().login(token.getToken(), token.getUserId(), new Callback<Void>() {
#Override
public void success(Void aVoid, Response response) {
Log.d("Chooser", "Successful login on server.");
}
#Override
public void failure(RetrofitError error) {
error.printStackTrace();
Log.d("Chooser", "Login failed on server.");
}
});
Using it as it is results with the following error:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING
There are many topics on this issue but no certain answer about the correct (or better) method to use.
Any ideas about the best implementation in these cases?
Sending an empty body with your HTTP response is perfectly legal and some clients may care only about the response status but some clients may expect to get a response so sending a body never hurts and sometimes may be useful.
You can include a JSON response in addition to the HTTP response status:
// Express 4.x:
res.status(403).json({error: 'Token is not legal'});
// Express 3.x:
res.json(403, {error: 'Token is not legal'});
Such an error message can be very useful for the client development. You can get 403 for many reasons, illegal token, expired token, a legal not expired token but for the wrong user that doesn't have some privilege - adding a specific error message in addition to the HTTP response code can tell the client what exactly went wrong and allows the client-side code to show a better error message to the user.
Also, note that true and false are also valid JSON.

Restlet is returning HTTP 415 for every POST request I make

I am getting this error every time I try to post data to my server:
Server logs:
Starting the internal [HTTP/1.1] server on port 9192
Starting facilitymanager.api.rest.FacilityManagerAPIRestWrapper application
2015-06-22 13:18:11 127.0.0.1 - - 9192 POST /devices/rename - 415 554 45 64 http://localhost:9192 Java/1.7.0_79 -
Stopping the internal server
However In the service handler I am stating that I will handle JSON messages as you can see here:
public static final class RenameDevice extends ServerResource {
#Post("application/json")
public String doPost() throws InterruptedException, ConstraintViolationException, InvalidChoiceException, JSONException {
configureRestForm(this);
final String deviceId = getRequest().getAttributes().get("device_id").toString();
final String newName = getRequest().getAttributes().get("new_name").toString();
return renameDevice(deviceId, newName).toString(4);
}
}
/**
* Enables incoming connections from different servers.
*
* #param serverResource
* #return
*/
#SuppressWarnings({ "unchecked", "rawtypes" })
private static Series<Header> configureRestForm(ServerResource serverResource) {
Series<Header> responseHeaders = (Series<Header>) serverResource.getResponse().getAttributes()
.get("org.restlet.http.headers");
if (responseHeaders == null) {
responseHeaders = new Series(Header.class);
serverResource.getResponse().getAttributes().put("org.restlet.http.headers", responseHeaders);
}
responseHeaders.add("Access-Control-Allow-Origin", "*");
responseHeaders.add("Access-Control-Allow-Methods", "GET, POST, PUT, OPTIONS");
responseHeaders.add("Access-Control-Allow-Headers", "Content-Type");
responseHeaders.add("Access-Control-Allow-Credentials", "false");
responseHeaders.add("Access-Control-Max-Age", "60");
return responseHeaders;
}
What am I missing here?
Thanks!
Edit: This is the full log concerning the request:
Processing request to: "http://localhost:9192/devices/rename"
Call score for the "org.restlet.routing.VirtualHost#54594d1d" host: 1.0
Default virtual host selected
Base URI: "http://localhost:9192". Remaining part: "/devices/rename"
Call score for the "" URI pattern: 0.5
Selected route: "" -> facilitymanager.api.rest.FacilityManagerAPIRestWrapper#d75d3d7
Starting facilitymanager.api.rest.FacilityManagerAPIRestWrapper application
No characters were matched
Call score for the "/devices/list" URI pattern: 0.0
Call score for the "/groups/rename" URI pattern: 0.0
Call score for the "/devices/rename" URI pattern: 1.0
Selected route: "/devices/rename" -> Finder for RenameDevice
15 characters were matched
New base URI: "http://localhost:9192/devices/rename". No remaining part to match
Delegating the call to the target Restlet
Total score of variant "[text/html]"= 0.25
Total score of variant "[application/xhtml+xml]"= 5.0E-4
Converter selected for StatusInfo: StatusInfoHtmlConverter
2015-06-22 13:28:31 127.0.0.1 - - 9192 POST /devices/rename - 415 554 45 67 http://localhost:9192 Java/1.7.0_79 -
POST /devices/rename HTTP/1.1 [415 Unsupported Media Type] ()
KeepAlive stream used: http://localhost:9192/devices/rename
sun.net.www.MessageHeader#2bf4dee76 pairs: {null: HTTP/1.1 415 Unsupported Media Type}{Content-type: text/html; charset=UTF-8}{Content-length: 554}{Server: Restlet-Framework/3.0m1}{Accept-ranges: bytes}{Date: Mon, 22 Jun 2015 12:28:31 GMT}
To obtain a full log one must invoke this line of code anywhere before opening the restlet/component server:
// Create a new Component.
component = new Component();
// Add a new HTTP server listening on default port.
component.getServers().add(Protocol.HTTP, SERVER_PORT);
Engine.setLogLevel(Level.ALL); /// <----- HERE
component.start();
I've found the problem! The thing is that a tagged #Post method must receive an argument.
So the method should be like this:
#Post("application/json")
public String doPost(Representation entity) throws InterruptedException, ConstraintViolationException,
InvalidChoiceException, JSONException, IOException {
configureRestForm(this);
final Reader r = entity.getReader();
StringBuffer sb = new StringBuffer();
int c;
// Reads the JSON from the input stream
while ((c = r.read()) != -1) {
sb.append((char) c);
}
System.out.println(sb.toString()); // Shows the JSON received
}
}
The Representation entity argument brings you the means to detect the media type you are receiving. But since I have my tag like #Post("application/json") I do not need to verify this again.
Imagine that I use just "#Post" instead of "#Post("application/json")", I would have to validate the media type (or types) this way:
#Post
public Representation doPost(Representation entity)
throws ResourceException {
if (entity.getMediaType().isCompatible(MediaType.APPLICATION_JSON)) {
// ...
}
// ...
}
A method with an #Post annotation is not required to receive an argument, unless you intend to receive a payload from your request.
If you want to filter on the media type of the incoming representation, use the "json" shortcut, as follow
#Post("json")
This will prevent you to test the media type of the representation.
The list of all available shortcut is available here. Most of them are quite simple to remember. The main reason to use shortcuts (or "extension" such as file extension) is that "xml" is related to several media types (application/xml, text/xml).
If you want to get the full content of the representation, simply call the "getText()" method, instead of using the getReader() and consume it.
If you want to support CORS, I suggest you to use the CorsService (available in the 2.3 version of the Restlet Framework.
Notice there exists a shortcut for getting the headers from a Request or a Response, just call the "getHeaders()" method.
Notice there exists a shortcut for getting the attributes taken from the URL, just call the "getAttribute(String) method.
Here is an updated version of your source code:
public class TestApplication extends Application {
public final static class TestPostResource extends ServerResource {
#Post
public String doPost(Representation entity) throws Exception {
final String deviceId = getAttribute("device_id");
final String newName = getAttribute("new_name");
System.out.println(entity.getText());
System.out.println(getRequest().getHeaders());
System.out.println(getResponse().getHeaders());
return deviceId + "/" + newName;
}
}
public static void main(String[] args) throws Exception {
Component c = new Component();
c.getServers().add(Protocol.HTTP, 8183);
c.getDefaultHost().attach(new TestApplication());
CorsService corsService = new CorsService();
corsService.setAllowedOrigins(new HashSet<String>(Arrays.asList("*")));
corsService.setAllowedCredentials(true);
corsService.setSkippingResourceForCorsOptions(true);
c.getServices().add(corsService);
c.start();
}
#Override
public Restlet createInboundRoot() {
Router router = new Router(getContext());
router.attach("/testpost/{device_id}/{new_name}", TestPostResource.class);
return router;
}
}

How to count all HTTP requests sent, retries in?

Some use cases require being able to count the requests sent by the Apache API. For example, when massively requesting a web API, which API requires an authentication through an API key, and which TOS limits the requests count in time for each key.
Being more specific on the case, I'm requesting https://domain1/fooNeedNoKey, and depending on its response analyzed data, I request https://domain2/fooNeedKeyWithRequestsCountRestrictions. All sends of those 1-to-2-requests sequences, are performed through a single org.apache.http.impl.client.FutureRequestExecutionService.
As of now, depending on org.apache.httpcomponents:httpclient:4.3.3, I'm using those API elements:
org.apache.http.impl.client.FutureRequestExecutionService, to perform multi-threaded HTTP requests. It offers time metrics (how much time did an HTTP thread took until terminated), but no requests counter metrics
final CloseableHttpClient httpClient = HttpClients.custom()
// the auto-retry feature of the Apache API will retry up to 5
// times on failure, being also allowed to send again requests
// that were already sent if necessary (I don't really understand
// the purpose of the second parameter below)
.setRetryHandler(new StandardHttpRequestRetryHandler(5, true))
// for HTTP 503 'Service unavailable' errors, also retrying up to
// 5 times, waiting 500ms between each retry. Guessed is that those
// 5 retries are part of the previous "global" 5 retries setting.
// The below setting, when used alone, would allow to only enable
// retries for HTTP 503, or to get a greater count of retries for
// this specific error
.setServiceUnavailableRetryStrategy(new DefaultServiceUnavailableRetryStrategy(5, 500))
.build();, which customizes the Apache API retry behavior
Getting back to the topic :
A request counter could be created by extending the Apache API retry-related classes quoted before
Alternatively, an Apache API support unrelated ticket tends to indicate this requests-counter metrics could be available and forwarded out of the API, into Java NIO
Edit 1:
Looks like the Apache API won't permit this to be done.
Quote from the inside of the API, RetryExec not beeing extendable in the API code I/Os:
package org.apache.http.impl.execchain;
public class RetryExec implements ClientExecChain {
..
public CloseableHttpResponse execute(
final HttpRoute route,
final HttpRequestWrapper request,
final HttpClientContext context,
final HttpExecutionAware execAware) throws IOException, HttpException {
..
for (int execCount = 1;; execCount++) {
try {
return this.requestExecutor.execute(route, request, context, execAware);
} catch (final IOException ex) {
..
if (retryHandler.retryRequest(ex, execCount, context)) {
..
}
..
}
}
The 'execCount' variable is the needed info, and it can't be accessed since it's only locally used.
As well, one can extend 'retryHandler', and manually count requests in it, but 'retryHandler.retryRequest(ex, execCount, context)' is not provided with the 'request' variable, making it impossible to know on what we're incrementing a counter (one may only want to increment the counter for requests sent to a specific domain).
I'm out of Java ideas for it. A 3rd party alternative: having the Java process polling a file on disk, managed by a shell script counting the desired requests. Sure it will make a lot of disk read-accesses and will be a hardware killer option.
Ok, the work around was easy, the HttpContext class of the API is intended for this:
// optionnally, in case your HttpCLient is configured for retry
class URIAwareHttpRequestRetryHandler extends StandardHttpRequestRetryHandler {
public URIAwareHttpRequestRetryHandler(final int retryCount, final boolean requestSentRetryEnabled)
{
super(retryCount, requestSentRetryEnabled);
}
#Override
public boolean retryRequest(final IOException exception, final int executionCount, final HttpContext context)
{
final boolean ret = super.retryRequest(exception, executionCount, context);
if (ret) {
doForEachRequestSentOnURI((String) context.getAttribute("requestURI"));
}
return ret;
}
}
// optionnally, in addition to the previous one, in case your HttpClient has specific settings for the 'Service unavailable' errors retries
class URIAwareServiceUnavailableRetryStrategy extends DefaultServiceUnavailableRetryStrategy {
public URIAwareServiceUnavailableRetryStrategy(final int maxRetries, final int retryInterval)
{
super(maxRetries, retryInterval);
}
#Override
public boolean retryRequest(final HttpResponse response, final int executionCount, final HttpContext context)
{
final boolean ret = super.retryRequest(response, executionCount, context);
if (ret) {
doForEachRequestSentOnURI((String) context.getAttribute("requestURI"));
}
return ret;
}
}
// main HTTP querying code: retain the URI in the HttpContext to make it available in the custom retry-handlers code
httpContext.setAttribute("requestURI", httpGET.getURI().toString());
try {
httpContext.setAttribute("requestURI", httpGET.getURI().toString());
httpClient.execute(httpGET, getHTTPResponseHandlerLazy(), httpContext);
// if request got successful with no need of retries, of if it succeeded on the last send: in any cases, this is the last query sent to server and it got successful
doForEachRequestSentOnURI(httpGET.getURI().toString());
} catch (final ClientProtocolException e) {
// if request definitively failed after retries: it's the last query sent to server, and it failed
doForEachRequestSentOnURI(httpGET.getURI().toString());
} catch (final IOException e) {
// if request definitively failed after retries: it's the last query sent to server, and it failed
doForEachRequestSentOnURI(httpGET.getURI().toString());
} finally {
// restoring the context as it was initially
httpContext.removeAttribute("requestURI");
}
Solved.

How to override response header in jersey client

I have a jersey client that I am trying to unmarshall a response entity with. The problem is the remote web service sends back application/octet-stream as the content type so Jersey does not know how to unmarshall it (I have similar errors with text/html coming back for XML and such). I cannot change the web service.
What I want to do is override the content-type and change it to application/json so jersey will know which marshaller to use.
I cannot register application/octet-stream with the json marshaller as for a given content type I actually might be getting back all kinds of oddities.
As laz pointed out, ClientFilter is the way to go:
client.addFilter(new ClientFilter() {
#Override
public ClientResponse handle(ClientRequest request) throws ClientHandlerException {
request.getHeaders().putSingle(HttpHeaders.CONTENT_TYPE, "application/json");
return getNext().handle(request);
}
});
I'm not well-versed in the Jersey client API, but can you use a ClientFilter to do this? Perhaps you could add a property to the request via ClientRequest.getProperties().put(String, Object) that tells the ClientFilter what Content-Type to override the response with. If the ClientFilter finds the override property, it uses it, otherwise it does not alter the response. I'm not sure if the ClientFilter is invoked prior to any unmarshalling though. Hopefully it is!
Edit (Have you tried something like this):
public class ContentTypeClientFilter implements ClientFilter {
#Override
public ClientResponse handle(ClientRequest request) throws ClientHandlerException {
final ClientResponse response = getNext().handle(request);
// check for overridden ContentType set by other code
final String overriddenContentType = request.getProperties().get("overridden.content.type");
if (overriddenContentType != null) {
response.getHeaders().putSingle(HttpHeaders.CONTENT_TYPE, overriddenContentType);
}
return response;
}
}
Under Java 8 and Jersey 2 you can do it with a lambda:
client.register((ClientResponseFilter) (requestContext, responseContext) ->
responseContext.getHeaders().putSingle("Content-Type", "application/json"));