JSON CSRF Interceptor for RestTemplate - json

I have a small Rest-Service App (Java 8, Spring 4.1.6, Spring Security 5.0.1, Jetty 9.3) and i'am accessing some services by JSON using Spring RestTemplate. Until now csfr was disabled, now i want to enable it.
As is understood csfr there is a common token (the client sends it with each request, the server stores it in the session) which is compared on server side. Access is denied if there is no token available or the token is different.
So i thought it would be a good idea to do this token-adding by using an interceptor. I also read, that in json i have to send the token as a header-parameter... but i did something wrong, already the login fails.
Here is the login-source:
MultiValueMap<String, String> form = new LinkedMultiValueMap<>();
form.add("username", username);
form.add("password", password);
return restTemplate.postForLocation(serverUri + "login", form);
Here the source of the interceptor:
import java.io.IOException;
import java.util.UUID;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpRequest;
import org.springframework.http.client.ClientHttpRequestExecution;
import org.springframework.http.client.ClientHttpRequestInterceptor;
import org.springframework.http.client.ClientHttpResponse;
public class MyCsfrInterceptor implements ClientHttpRequestInterceptor{
public static final String CSRF_TOKEN_HEADER_NAME = "X-CSRF-TOKEN";
public static final String csrfSessionToken = UUID.randomUUID().toString();
private static Logger LOG = LoggerFactory.getLogger(MyCsfrInterceptor.class);
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
LOG.info("My interceptor called!");
if (request.getMethod() == HttpMethod.DELETE ||
request.getMethod() == HttpMethod.POST ||
request.getMethod() == HttpMethod.PATCH ||
request.getMethod() == HttpMethod.PUT){
LOG.info("Setting csrf token...");
request.getHeaders().add(CSRF_TOKEN_HEADER_NAME, csrfSessionToken);
}
return execution.execute(request, body);
}
}
Here is the log output on client side:
23:24:40.605 [main] DEBUG c.m.l.w.client.StatefulRestTemplate - Created POST request for "http://localhost:8080/login"
23:24:40.610 [main] DEBUG c.m.l.w.client.StatefulRestTemplate - Writing [{username=[user], password=[user]}] using [org.springframework.http.converter.support.AllEncompassingFormHttpMessageConverter#618b19ad]
23:24:40.615 [main] INFO c.m.l.w.client.MyCsfrInterceptor - My interceptor called!
23:24:40.615 [main] INFO c.m.l.w.client.MyCsfrInterceptor - Setting csrf token...
23:24:40.650 [main] DEBUG o.a.h.c.protocol.RequestAddCookies - CookieSpec selected: best-match
23:24:40.670 [main] DEBUG o.a.h.c.protocol.RequestAuthCache - Auth cache not set in the context
23:24:40.675 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection request: [route: {}->http://localhost:8080][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
23:24:40.705 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection leased: [id: 0][route: {}->http://localhost:8080][total kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]
23:24:40.710 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Opening connection {}->http://localhost:8080
23:24:40.715 [main] DEBUG o.a.h.i.c.HttpClientConnectionOperator - Connecting to localhost/127.0.0.1:8080
23:24:40.715 [main] DEBUG o.a.h.i.c.HttpClientConnectionOperator - Connection established 127.0.0.1:54712<->127.0.0.1:8080
23:24:40.715 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Executing request POST /login HTTP/1.1
23:24:40.715 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Target auth state: UNCHALLENGED
23:24:40.720 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Proxy auth state: UNCHALLENGED
23:24:40.720 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> POST /login HTTP/1.1
23:24:40.720 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Type: application/x-www-form-urlencoded
23:24:40.720 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> X-CSRF-TOKEN: 99ca8171-b8e0-4b95-8d06-3759a874c64b
23:24:40.720 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Length: 27
23:24:40.720 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Host: localhost:8080
23:24:40.720 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Connection: Keep-Alive
23:24:40.720 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.3.4 (java 1.5)
23:24:40.720 [main] DEBUG org.apache.http.headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "POST /login HTTP/1.1[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Content-Type: application/x-www-form-urlencoded[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "X-CSRF-TOKEN: 99ca8171-b8e0-4b95-8d06-3759a874c64b[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Content-Length: 27[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Host: localhost:8080[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.3.4 (java 1.5)[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "[\r][\n]"
23:24:40.720 [main] DEBUG org.apache.http.wire - http-outgoing-0 >> "username=user&password=user"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "HTTP/1.1 403 Expected CSRF token not found. Has your session expired?[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "Date: Sat, 15 Aug 2015 21:24:40 GMT[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "Pragma: no-cache[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "X-XSS-Protection: 1; mode=block[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "X-Frame-Options: DENY[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "X-Content-Type-Options: nosniff[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "Set-Cookie: JSESSIONID=1bvmexep1lv9h1qja44hflx0wg;Path=/[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Type: text/html;charset=iso-8859-1[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Length: 409[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "Server: Jetty(9.3.0.RC1)[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "[\r][\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "<html>[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "<head>[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "<title>Error 403 Expected CSRF token not found. Has your session expired?</title>[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "</head>[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "<body><h2>HTTP ERROR 403</h2>[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "<p>Problem accessing /login. Reason:[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "<pre> Expected CSRF token not found. Has your session expired?</pre></p><hr>Powered by Jetty:// 9.3.0.RC1<hr/>[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "</body>[\n]"
23:24:40.831 [main] DEBUG org.apache.http.wire - http-outgoing-0 << "</html>[\n]"
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << HTTP/1.1 403 Expected CSRF token not found. Has your session expired?
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << Date: Sat, 15 Aug 2015 21:24:40 GMT
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << Pragma: no-cache
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << X-XSS-Protection: 1; mode=block
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << X-Frame-Options: DENY
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << X-Content-Type-Options: nosniff
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << Set-Cookie: JSESSIONID=1bvmexep1lv9h1qja44hflx0wg;Path=/
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << Content-Type: text/html;charset=iso-8859-1
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << Cache-Control: must-revalidate,no-cache,no-store
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << Content-Length: 409
23:24:40.836 [main] DEBUG org.apache.http.headers - http-outgoing-0 << Server: Jetty(9.3.0.RC1)
23:24:40.846 [main] DEBUG o.a.h.impl.execchain.MainClientExec - Connection can be kept alive indefinitely
23:24:40.861 [main] DEBUG o.a.h.c.p.ResponseProcessCookies - Cookie accepted [JSESSIONID="1bvmexep1lv9h1qja44hflx0wg", version:0, domain:localhost, path:/, expiry:null]
23:24:40.861 [main] DEBUG c.m.l.w.client.StatefulRestTemplate - POST request for "http://localhost:8080/login" resulted in 403 (Expected CSRF token not found. Has your session expired?); invoking error handler
23:24:40.866 [main] DEBUG o.a.h.i.c.PoolingHttpClientConnectionManager - Connection [id: 0][route: {}->http://localhost:8080] can be kept alive indefinitely
In spring security config i didn't do any csrf configuration.
So what's wrong? Did i set the header wrong? Any config missing?
Best regards and thank you for your time.

Looking at your code, it seems that you are generating the CSRF token yourself. But, as I understand, Spring Security CSRF handling would work this way:
Spring Security would generate the CSRF token.
Whenever a request comes (say a GET request) Spring Security will attach the token as a request parameter. This helps rendering JSP forms with the token as a hidden field, like this:
<form action="/foo/5/update" method="post">
<input type="text" ... />
<input type="submit" value="Update" />
<input type="hidden" name="${_csrf.parameterName}" value="${_csrf.token}"/>
</form>
Whenever a POST like request comes (say a form getting submitted), Spring Security will match its token with the one submitted as a parameter or header.
(2) above works well when we are using JSP etc. But, when we are calling an API, we will first need to fetch the token. A common practice to do that is to have a filter at the server side, attaching the token as a cookie. This is my filter code in a project:
public class CsrfCookieFilter extends OncePerRequestFilter {
public static final String XSRF_TOKEN_COOKIE_NAME = "XSRF-TOKEN";
#Override
protected void doFilterInternal(HttpServletRequest request,
HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
log.debug("Inside CsrfCookieFilter ...");
CsrfToken csrf = (CsrfToken)
request.getAttribute(CsrfToken.class.getName()); // Or "_csrf" (See CSRFFilter.doFilterInternal).
if (csrf != null) {
Cookie cookie = WebUtils.getCookie(
request, XSRF_TOKEN_COOKIE_NAME);
String token = csrf.getToken();
if (cookie==null ||
token!=null && !token.equals(cookie.getValue())) {
cookie = new Cookie(XSRF_TOKEN_COOKIE_NAME, token);
cookie.setPath("/");
response.addCookie(cookie);
}
}
filterChain.doFilter(request, response);
}
}
So, before a POST request, a GET request first should come and fetch the token as a cookie. The token then should be sent back as a header in subsequent requests.
My code for wiring this filter and altering the header name looks like this inside inside my security configuration class:
#Override
protected void configure(HttpSecurity http) throws Exception {
http
...
.addFilterAfter(csrfCookieFilter(), CsrfFilter.class)
...
}
protected Filter csrfCookieFilter() {
return new CsrfCookieFilter();
}
Also, note that Spting Security changes the token after certain events like login and logout. So, we need to again fetch the token with a GET request.
The official Spring Angular guide has elaborated it in details.

Related

nodemailer multiple email sending using SMTP causes connection close errors

I am having problems when I do a looped call to Mailer send email function.I am testing sending of over 50 emails and the most number of successfully sent were 30 so far. The rest is stopped by the following errors. I redacted the complete email address for the robot for privacy reasons.
[2018-09-03 05:18:32] INFO [0rULO04aig] User "xxx.robot#xxxxxx.com"
authenticated
[2018-09-03 05:18:32] INFO [#1] Sending message
<3299d22a-e0c8-00b8-4f7d-4d0455f85b31#xxxxx.com> using #1 to
[2018-09-03 05:18:32] DEBUG [0rULO04aig] C: MAIL FROM:
[2018-09-03 05:18:32] DEBUG [PFhdqz3yHs] S:
250 2.1.0 OK v72-v6sm36109733pfj.22 - gsmtp
[2018-09-03 05:18:32] DEBUG [PFhdqz3yHs] C: RCPT
TO:
[2018-09-03 05:18:32] DEBUG [0rULO04aig] S: 250 2.1.0 OK
v186-v6sm24113856pgd.25 - gsmtp
[2018-09-03 05:18:32] DEBUG [0rULO04aig] C: RCPT
TO:
[2018-09-03 05:18:32] DEBUG [PFhdqz3yHs] S: 250 2.1.5 OK
v72-v6sm36109733pfj.22 - gsmtp
[2018-09-03 05:18:32] DEBUG [PFhdqz3yHs] C: DATA
[2018-09-03 05:18:32] DEBUG [BlXN9zcX8Ak] S: 421 4.7.0 Temporary
System Problem. Try again later (10). x65-v6sm28504837pfk.140 - gsmtp
[2018-09-03 05:18:32] DEBUG [BlXN9zcX8Ak] Closing connection to the
server using "end"
[2018-09-03 05:18:32] INFO [#1] Connection #1 was closed
[2018-09-03 05:18:32] ERROR [#1] Pool Error for #1: Data command
failed: 421 4.7.0 Temporary System Problem. Try again later (10).
x65-v6sm28504837pfk.140 - gsmtp
[2018-09-03 05:18:32] ERROR Send Error: Data command failed: 421 4.7.0
Temporary System Problem. Try again later (10).
x65-v6sm28504837pfk.140 - gsmtp
[2018-09-03 05:18:32] DEBUG [gflzBwS1fc] S: 421 4.7.0 Temporary System
Problem. Try again later (10). b203-v6sm1382852pfb.174 - gsmtp
I am unable to post the complete code since the checker here keeps telling me that it is not properly formatted. The code follows ESLint rules.
Let me post a pseudocode:
forEach(users, key) =>
getSQL
create Users if no result found
sendEmail(user)
So the settings:
function Mailer(options) {
this.options = options;
that = this;
this.transporter = nodemailer.createTransport({
pool: true,
maxConnections: 50,
maxMessages: Infinity,
debug: true,
logger: true,
connectionTimeout: 3000,
host: config.mail.host,
port: config.mail.port,
secure: false,
auth: {
user: config.mail.username,
pass: config.mail.password,
},
});
}

WSO2 Not a valid protocol version: <head> For

I'm trying to call a http-EndPoint with POST method in WSO2 Enterprise Integrator. This EndPoint gets a JSON object as input and return a JSON object as response.
Until now I've used various types of http-EndPoints in Enterprise Integrator without any problem but this particular EndPoint returns an error message which I'm not able to find any clue about it in google.
This is my api:
<resource methods="POST" uri-template="/userInfo">
<inSequence>
<payloadFactory media-type="json">
<format>{"jsonrpc": "2.0", "params": {"auth_type": "ADMIN", "auth_name": "someName", "normal_username": "$1", "auth_pass": "myPassword"},"method": "user.MethodName", "id": 0}
</format>
<args>
<arg evaluator="json" expression="$.user"/>
</args>
</payloadFactory>
<property name="messageType" scope="axis2" type="STRING" value="application/json"/>
<property name="FORCE_HTTP_1.0" scope="axis2" type="STRING" value="true"/>
<send description="">
<endpoint>
<http method="post" uri-template="http://192.168.1.50:1237"/>
</endpoint>
</send>
</inSequence>
<outSequence>
<send/>
</outSequence>
<faultSequence/>
</resource>
This is my error message in console:
[EI-Core] ERROR - TargetHandler HTTP protocol violation : Not a valid protocol version: <head> For : 192.168.1.50:1237
UPDATE
I enabled Wire Logs in WSO2 to investigate in depth about this issue. Here is log output:
[2018-01-15 09:30:46,621] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "POST HTTP/1.0[\r][\n]"
[2018-01-15 09:30:46,622] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "Expect: 100-continue[\r][\n]"
[2018-01-15 09:30:46,622] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "Content-Type: application/json[\r][\n]"
[2018-01-15 09:30:46,622] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "Content-Length: 196[\r][\n]"
[2018-01-15 09:30:46,622] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "Host: 192.168.1.50:1237[\r][\n]"
[2018-01-15 09:30:46,622] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "Connection: Keep-Alive[\r][\n]"
[2018-01-15 09:30:46,622] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "User-Agent: Synapse-PT-HttpComponents-NIO[\r][\n]"
[2018-01-15 09:30:46,622] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "[\r][\n]"
[2018-01-15 09:30:46,622] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 << "{"jsonrpc": "2.0", "params": {"auth_type": "ADMIN", "auth_name": "someName", "normal_username": "myUserName", "auth_pass": "myPassword", "remote_addr": "127.0.0.1"},"method": "user.MethodName", "id": 0}"
[2018-01-15 09:30:46,623] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "<head>[\n]"
[2018-01-15 09:30:46,623] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "<title>Error response</title>[\n]"
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "</head>[\n]"
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "<body>[\n]"
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "<h1>Error response</h1>[\n]"
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "<p>Error code 400.[\n]"
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "<p>Message: Bad HTTP/0.9 request type ('POST').[\n]"
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "<p>Error code explanation: 400 = Bad request syntax or unsupported method.[\n]"
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O dispatcher-1 >> "</body>[\n]"
Although wire log returned this response: Message: Bad HTTP/0.9 request type ('POST'), but I can send post requests to Endpoint directly without any problem.
Here is direct post request to endpoint:
curl -v -H "Content-Type: application/json" -X POST -d '{"jsonrpc": "2.0", "params": {"auth_type": "ADMIN", "auth_name": "someName", "normal_username": "myUserName", "auth_pass": "myPassword", "remote_addr": "127.0.0.1"}, "method": "user.MethodName", "id": 0}' 192.168.1.50:1237
And here is output result:
STATE: INIT => CONNECT handle 0x600057950; line 1423 (connection #-5000)
Rebuilt URL to: http://192.168.1.50:1237/
Added connection 0. The cache now contains 1 members
Trying 5.202.129.107...
TCP_NODELAY set
STATE: CONNECT => WAITCONNECT handle 0x600057950; line 1475 (connection #0)
Connected to 192.168.1.50 (192.168.1.50) port 1237 (#0)
STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x600057950; line 1592 (connection #0)
Marked for [keep alive]: HTTP default
STATE: SENDPROTOCONNECT => DO handle 0x600057950; line 1610 (connection #0)
POST / HTTP/1.1
Host: http://192.168.1.50:1237
User-Agent: curl/7.56.1
Accept: /
Content-Type: application/json
Content-Length: 197
upload completely sent off: 197 out of 197 bytes
STATE: DO => DO_DONE handle 0x600057950; line 1689 (connection #0)
STATE: DO_DONE => WAITPERFORM handle 0x600057950; line 1814 (connection #0)
STATE: WAITPERFORM => PERFORM handle 0x600057950; line 1824 (connection #0)
HTTP 1.0, assume close after body
Marked for [closure]: HTTP/1.0 close after body < HTTP/1.0 200 OK < Server: BaseHTTP/0.3 Python/2.7.9 < Date: Sun, 14 Jan 2018 07:53:37
GMT < Content-type: application/json < Content-length: 2105 <
STATE: PERFORM => DONE handle 0x600057950; line 1993 (connection #0)
multi_done
Closing connection 0
The cache now contains 0 members
Expire cleared
When I compare headers from WSO2 to headers from curl I cant figure out what are differences. Why WSO2 request failed while curl request succeed?
Try adding this before the <send> mediator.
<property name="FORCE_HTTP_1.0" value="true" scope="axis2"/>
By comparing headers from WSO2 and curl I noticed this line in curl header Rebuilt URL to: http://192.168.1.50:1237/. Hence I changed this:
<endpoint>
<http method="post" uri-template="http://192.168.1.50:1237"/>
</endpoint>
To this:
<endpoint>
<http method="post" uri-template="http://192.168.1.50:1237/"/>
</endpoint>
Add the following properties to the sequence:
FORCE_HTTP_CONTENT_LENGTH
COPY_CONTENT_LENGTH_FROM_INCOMING
This solves the problem, but the reason still unknown.
from your wire log:
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O
dispatcher-1 >> "Message: Bad HTTP/0.9 request type ('POST').[\n]"
[2018-01-15 09:30:46,624] [EI-Core] DEBUG - wire HTTP-Sender I/O
dispatcher-1 >> "Error code explanation: 400 = Bad request syntax
or unsupported method.[\n]" [2018-01-15 09:30:46,624] [EI-Core] DEBUG
- wire HTTP-Sender I/O dispatcher-1 >> "[\n]"
Note: The official HTTP 0.9 specification included only one request method: GET.
you are using post which is not supported in HTTP 0.9
For more information about HTTP 0.9, refer to the original W3C HTTP specification.

javax.mail.sendfailedexception sending failed nested exception is class javax.mail.MessagingException

It give error about some problem with STARTTLS command .
it gives the error like this
javax.mail.sendfailedexception sending failed nested exception is class javax.mail.MessagingException
So please help me out of it.
Java Code
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
public class MailDemo {
public static void main(String[] args) {
Properties properties=new Properties();
properties.put("mail.smtp.auth", "true");
properties.put("mail.smtp.starttls.enable","true");
properties.put("mail.smtp.host","smtp.gmail.com");
properties.put("mail.smtp.port", "587");
Scanner scn=new Scanner(System.in);
System.out.println("Username for Authentication :");
final String username=scn.nextLine();
System.out.println("Password for Authentication :");
final String password=scn.nextLine();
System.out.println("From Email..");
String fromEmailAddrs=scn.nextLine();
System.out.println("To Email..");
String toEmail=scn.nextLine();
System.out.println("Subject..");
String subject=scn.nextLine();
System.out.println("Message..");
String textMessage=scn.nextLine();
Session session=Session.getDefaultInstance(properties,new Authenticator() {
#Override
protected javax.mail.PasswordAuthentication getPasswordAuthentication(){
return new javax.mail.PasswordAuthentication(username, password);
}
});
try{
Message msg=new MimeMessage(session);
msg.setFrom(new InternetAddress(fromEmailAddrs));
msg.setRecipients(Message.RecipientType.TO, InternetAddress.parse(toEmail));
msg.setSubject(subject);
msg.setText(textMessage);
Transport.send(msg);
System.out.println("/n Your Message Delivered Succesfully");
}
catch(MessagingException m){
throw new RuntimeException(m);
}
}
}
Console
Username for Authentication :
bijaybhaskar01
Password for Authentication :
abcdef#456789
From Email..
bijaybhaskar01#gmail.com
To Email..
bbswain1001#gmail.com
Subject..
Hello
Message..
hai
Output
Exception in thread "main" java.lang.RuntimeException: javax.mail.SendFailedException: Sending failed;
nested exception is:
class javax.mail.MessagingException: 530 5.7.0 Must issue a STARTTLS command first. fh14sm3131583pab.31 - gsmtp
at com.mail.bhaskar.MailDemo.main(MailDemo.java:52)
Caused by: javax.mail.SendFailedException: Sending failed;
nested exception is:
class javax.mail.MessagingException: 530 5.7.0 Must issue a STARTTLS command first. fh14sm3131583pab.31 - gsmtp
at javax.mail.Transport.send0(Transport.java:218)
at javax.mail.Transport.send(Transport.java:80)
at com.mail.bhaskar.MailDemo.main(MailDemo.java:48)
Debug output
DEBUG: JavaMail version 1.3.1
DEBUG: java.io.FileNotFoundException: C:\Program Files\Java\jre1.8.0_77\lib\javamail.providers (The system cannot find the file specified)
DEBUG: URL jar:file:/E:/Study/Java/Jars/MailJars/gimap.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: jar:file:/E:/Study/Java/Jars/MailJars/gimap.jar!/META-INF/javamail.providers
DEBUG: URL jar:file:/E:/Study/Java/Jars/MailJars/imap.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: jar:file:/E:/Study/Java/Jars/MailJars/imap.jar!/META-INF/javamail.providers
DEBUG: URL jar:file:/E:/Study/Java/Jars/MailJars/pop3.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: jar:file:/E:/Study/Java/Jars/MailJars/pop3.jar!/META-INF/javamail.providers
DEBUG: URL jar:file:/E:/Study/Java/Jars/MailJars/smtp.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: jar:file:/E:/Study/Java/Jars/MailJars/smtp.jar!/META-INF/javamail.providers
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Sun Microsystems, Inc], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.gimap.GmailSSLStore=javax.mail.Provider[STORE,gimaps,com.sun.mail.gimap.GmailSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc], com.sun.mail.gimap.GmailStore=javax.mail.Provider[STORE,gimap,com.sun.mail.gimap.GmailStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Sun Microsystems, Inc]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], gimap=javax.mail.Provider[STORE,gimap,com.sun.mail.gimap.GmailStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], gimaps=javax.mail.Provider[STORE,gimaps,com.sun.mail.gimap.GmailSSLStore,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: URL jar:file:/E:/Study/Java/Jars/MailJars/smtp.jar!/META-INF/javamail.address.map
DEBUG: successfully loaded resource: jar:file:/E:/Study/Java/Jars/MailJars/smtp.jar!/META-INF/javamail.address.map
DEBUG: java.io.FileNotFoundException: C:\Program Files\Java\jre1.8.0_77\lib\javamail.address.map (The system cannot find the file specified)
DEBUG: getProvider() returning javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]
DEBUG SMTP: useEhlo true, useAuth true
DEBUG SMTP: useEhlo true, useAuth true
DEBUG SMTP: trying to connect to host "smtp.gmail.com", port 587
220 smtp.gmail.com ESMTP n6sm3627811pfa.2 - gsmtp
DEBUG SMTP: connected to host "smtp.gmail.com", port: 587
EHLO Bhaskar
250-smtp.gmail.com at your service, [119.82.116.106]
250-SIZE 35882577
250-8BITMIME
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-PIPELINING
250 SMTPUTF8
DEBUG SMTP: Found extension "SIZE", arg "35882577"
DEBUG SMTP: Found extension "8BITMIME", arg ""
DEBUG SMTP: Found extension "STARTTLS", arg ""
DEBUG SMTP: Found extension "ENHANCEDSTATUSCODES", arg ""
DEBUG SMTP: Found extension "PIPELINING", arg ""
DEBUG SMTP: Found extension "SMTPUTF8", arg ""
DEBUG SMTP: use8bit false
MAIL FROM:<bijaybhaskar01#gmail.com>
530 5.7.0 Must issue a STARTTLS command first. n6sm3627811pfa.2 - gsmtp
QUIT
You're using JavaMail 1.3.1, which is ancient and does not support STARTTLS.
Upgrade to a newer version.

Apache Drill connect from code

I have start embedded apache drill instance following this tutorial https://drill.apache.org/docs/using-the-jdbc-driver/ I can query drill from console. But now I need to connect it from my code.
Connection conn = null;
conn = DriverManager.getConnection("jdbc:drill:zk=localhost:31010");
When I run it I get following output in my IDE:
18:01:04.970 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:31010. Will not attempt to authenticate using SASL (unknown error)
18:01:04.970 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/0:0:0:0:0:0:0:1:31010, initiating session
18:01:04.971 [main-SendThread(localhost:31010)] DEBUG org.apache.zookeeper.ClientCnxn - Session establishment request sent on localhost/0:0:0:0:0:0:0:1:31010
18:01:04.972 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
18:01:05.331 [main] DEBUG org.apache.curator.RetryLoop - Retrying operation
18:01:06.721 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:31010. Will not attempt to authenticate using SASL (unknown error)
18:01:06.721 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:31010, initiating session
18:01:06.721 [main-SendThread(localhost:31010)] DEBUG org.apache.zookeeper.ClientCnxn - Session establishment request sent on localhost/127.0.0.1:31010
18:01:06.722 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
18:01:07.495 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:31010. Will not attempt to authenticate using SASL (unknown error)
18:01:07.495 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/0:0:0:0:0:0:0:1:31010, initiating session
18:01:07.495 [main-SendThread(localhost:31010)] DEBUG org.apache.zookeeper.ClientCnxn - Session establishment request sent on localhost/0:0:0:0:0:0:0:1:31010
18:01:07.496 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
18:01:09.269 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:31010. Will not attempt to authenticate using SASL (unknown error)
18:01:09.269 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:31010, initiating session
18:01:09.269 [main-SendThread(localhost:31010)] DEBUG org.apache.zookeeper.ClientCnxn - Session establishment request sent on localhost/127.0.0.1:31010
18:01:09.270 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
18:01:09.392 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:31010. Will not attempt to authenticate using SASL (unknown error)
18:01:09.392 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/0:0:0:0:0:0:0:1:31010, initiating session
18:01:09.392 [main-SendThread(localhost:31010)] DEBUG org.apache.zookeeper.ClientCnxn - Session establishment request sent on localhost/0:0:0:0:0:0:0:1:31010
18:01:09.393 [main-SendThread(localhost:31010)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
18:01:10.355 [main] ERROR org.apache.curator.ConnectionState - Connection timed out for connection string (localhost:31010) and timeout (5000) / elapsed (6199)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:198) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:88) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.utils.EnsurePath$InitialHelper$1.call(EnsurePath.java:148) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.utils.EnsurePath$InitialHelper.ensure(EnsurePath.java:140) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.utils.EnsurePath.ensure(EnsurePath.java:99) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:74) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.imps.NamespaceImpl.newNamespaceAwareEnsurePath(NamespaceImpl.java:87) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.imps.CuratorFrameworkImpl.newNamespaceAwareEnsurePath(CuratorFrameworkImpl.java:468) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.<init>(PathChildrenCache.java:223) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.<init>(PathChildrenCache.java:182) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.x.discovery.details.ServiceCacheImpl.<init>(ServiceCacheImpl.java:65) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.x.discovery.details.ServiceCacheBuilderImpl.build(ServiceCacheBuilderImpl.java:47) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.drill.exec.coord.zk.ZKClusterCoordinator.<init>(ZKClusterCoordinator.java:104) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:185) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.drill.jdbc.DrillConnectionImpl.<init>(DrillConnectionImpl.java:126) [drill-jdbc-1.0.0.jar:1.0.0]
at org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection.<init>(DrillJdbc41Factory.java:97) [drill-jdbc-1.0.0.jar:1.0.0]
at org.apache.drill.jdbc.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:60) [drill-jdbc-1.0.0.jar:1.0.0]
at org.apache.drill.jdbc.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:46) [drill-jdbc-1.0.0.jar:1.0.0]
at org.apache.drill.jdbc.DrillFactory.newConnection(DrillFactory.java:54) [drill-jdbc-1.0.0.jar:1.0.0]
at net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126) [drill-jdbc-all-1.0.0.jar:na]
at java.sql.DriverManager.getConnection(DriverManager.java:571) [na:1.7.0_67]
at java.sql.DriverManager.getConnection(DriverManager.java:233) [na:1.7.0_67]
at DrillConnectTest.testConnect(DrillConnectTest.java:34) [classes/:na]
at DrillConnectTest.main(DrillConnectTest.java:55) [classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_67]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_67]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_67]
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) [idea_rt.jar:na]
18:01:10.355 [main] DEBUG org.apache.curator.RetryLoop - Retry-able exception received
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = ConnectionLoss
at org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:198) ~[drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:88) ~[drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115) ~[drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.utils.EnsurePath$InitialHelper$1.call(EnsurePath.java:148) ~[drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) ~[drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.utils.EnsurePath$InitialHelper.ensure(EnsurePath.java:140) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.utils.EnsurePath.ensure(EnsurePath.java:99) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:74) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.imps.NamespaceImpl.newNamespaceAwareEnsurePath(NamespaceImpl.java:87) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.imps.CuratorFrameworkImpl.newNamespaceAwareEnsurePath(CuratorFrameworkImpl.java:468) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.<init>(PathChildrenCache.java:223) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.framework.recipes.cache.PathChildrenCache.<init>(PathChildrenCache.java:182) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.x.discovery.details.ServiceCacheImpl.<init>(ServiceCacheImpl.java:65) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.curator.x.discovery.details.ServiceCacheBuilderImpl.build(ServiceCacheBuilderImpl.java:47) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.drill.exec.coord.zk.ZKClusterCoordinator.<init>(ZKClusterCoordinator.java:104) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.drill.exec.client.DrillClient.connect(DrillClient.java:185) [drill-jdbc-all-1.0.0.jar:na]
at org.apache.drill.jdbc.DrillConnectionImpl.<init>(DrillConnectionImpl.java:126) [drill-jdbc-1.0.0.jar:1.0.0]
at org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection.<init>(DrillJdbc41Factory.java:97) [drill-jdbc-1.0.0.jar:1.0.0]
at org.apache.drill.jdbc.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:60) [drill-jdbc-1.0.0.jar:1.0.0]
at org.apache.drill.jdbc.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:46) [drill-jdbc-1.0.0.jar:1.0.0]
at org.apache.drill.jdbc.DrillFactory.newConnection(DrillFactory.java:54) [drill-jdbc-1.0.0.jar:1.0.0]
at net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126) [drill-jdbc-all-1.0.0.jar:na]
at java.sql.DriverManager.getConnection(DriverManager.java:571) [na:1.7.0_67]
at java.sql.DriverManager.getConnection(DriverManager.java:233) [na:1.7.0_67]
at DrillConnectTest.testConnect(DrillConnectTest.java:34) [classes/:na]
at DrillConnectTest.main(DrillConnectTest.java:55) [classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_67]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_67]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_67]
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) [idea_rt.jar:na]
18:01:10.860 [main] DEBUG org.apache.curator.RetryLoop - Retrying operation
And this repeats in cycle. And in server logs also I see following:
2015-06-22 17:53:20,465 [UserServer-1] INFO o.a.d.exec.rpc.ProtobufLengthDecoder - Channel is closed, discarding remaining 48 byte(s) in buffer.
2015-06-22 17:53:21,232 [UserServer-1] ERROR o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication. Connection: /127.0.0.1:31010 <--> /127.0.0.1:55704 (user client). Closing connection.
io.netty.handler.codec.CorruptedFrameException: Received a message of length 0.
at org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:74) ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0]
at org.apache.drill.exec.rpc.user.UserProtobufLengthDecoder.decode(UserProtobufLengthDecoder.java:37) ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315) ~[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229) ~[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) [netty-common-4.0.27.Final.jar:4.0.27.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
2015-06-22 17:53:21,232 [UserServer-1] INFO o.a.d.exec.rpc.ProtobufLengthDecoder - Channel is closed, discarding remaining 48 byte(s) in buffer.
2015-06-22 17:53:23,324 [UserServer-1] ERROR o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication. Connection: /0:0:0:0:0:0:0:1:31010 <--> /0:0:0:0:0:0:0:1:55706 (user client). Closing connection.
io.netty.handler.codec.CorruptedFrameException: Received a message of length 0.
at org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:74) ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0]
at org.apache.drill.exec.rpc.user.UserProtobufLengthDecoder.decode(UserProtobufLengthDecoder.java:37) ~[drill-java-exec-1.0.0-rebuffed.jar:1.0.0]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315) ~[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229) ~[netty-codec-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) [netty-common-4.0.27.Final.jar:4.0.27.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
2015-06-22 17:53:23,325 [UserServer-1] INFO o.a.d.exec.rpc.ProtobufLengthDecoder - Channel is closed, discarding remaining 48 byte(s) in buffer.
2015-06-22 17:53:23,460 [UserServer-1] ERROR o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication. Connection: /127.0.0.1:31010 <--> /127.0.0.1:55707 (user client). Closing connection.
io.netty.handler.codec.CorruptedFrameException: Received a message of length 0.
Any suggestion why this can happen? May be I need some extra setup of local drill?
You've used zk= in your JDBC URL, but the right hand side of that portion of the URL is pointing to 31010, which is the Drillbit's "user" port, per the docs:
http://drill.apache.org/docs/ports-used-by-drill/
For embedded mode you can try either:
jdbc:drill:zk=local
or
jdbc:drill:drillbit=localhost:31010
Hope that helps.
Other way to execute query by Calling Apache Drill REST API through code
Prerequisite:
1.Start Apache Drill
2.Add/enable Storage Plugin in http://localhost:8047/storage
3.Execute below code
public class RestAPI {
private static Log logger = LogFactory.getLog(RestAPI.class);
public static void main(String[] args) {
getQueryResponce();
}
private static void getQueryResponce(){
Client client = null;
WebTarget target = null;
try {
logger.info("---------Started execution-----------");
RequestQuery query = new RequestQuery();
query.setQueryType("SQL");
//MySQL
//query.setQuery("SELECT * FROM MYSQL.foodmart.collections");
//query.setQuery("SELECT * FROM MYSQL.foodmart.collections limit 100");
//query.setQuery("SELECT payment_due_from,NumItems FROM MYSQL.foodmart.collections limit 10");
//query.setQuery("SELECT count(SPORTS_PREFERENCE) FROM MYSQL.foodmart.collections group by SPORTS_PREFERENCE");
//query.setQuery("SELECT DISTINCT payment_due_from FROM MYSQL.foodmart.collections ");
//Mongo DB
//query.setQuery("select * from mongo.apache_drill.pt_BMS_preferences_data where SPORTS_PREFERENCE = 'Cricket' limit 10");
//query.setQuery("SELECT COUNT(SPORTS_PREFERENCE) FROM mongo.apache_drill.pt_BMS_preferences_data GROUP BY SPORTS_PREFERENCE");
query.setQuery("SELECT DISTINCT(SPORTS_PREFERENCE) FROM mongo.apache_drill.pt_BMS_preferences_data ");
client = ClientBuilder.newClient();
target = client.target("http://localhost:8047/query.json");
//target = target.path(path);
Response response = target.request().accept(MediaType.APPLICATION_JSON)
.post(Entity.json(query), Response.class);
if (response.getStatus() != 200) {
throw new RuntimeException("Failed : HTTP error code : " + response.getStatus());
}
String string = response.readEntity(String.class);
logger.info(query.getQueryType()+"->"+query.getQuery());
logger.info("Responce:\n"+string);
logger.info("---------End execution-----------");
} catch (Exception e) {
e.printStackTrace();
logger.error(e.getMessage(),e);
}
}
Hope this help...

How to use the Fluent API of Apache HttpClient to read UTF-8 coded website?

String html = Request.Get("https://kokos.pl/")
.execute().returnContent().asString();
System.out.println(html);
What I obtain in the 12th line is:
<title>Szybkie po??yczki got??wkowe, po??yczki spo??eczno??ciowe - Kokos.pl</title>
while it should be:
<title>Szybkie pożyczki gotówkowe, pożyczki społecznościowe - Kokos.pl</title>
[DEBUG] DefaultClientConnection - Sending request: GET / HTTP/1.1
[DEBUG] headers - >> GET / HTTP/1.1
[DEBUG] headers - >> Host: kokos.pl
[DEBUG] headers - >> Connection: Keep-Alive
[DEBUG] headers - >> User-Agent: Apache-HttpClient/4.2.5 (java 1.5)
[DEBUG] DefaultClientConnection - Receiving response: HTTP/1.1 200 OK
[DEBUG] headers - << HTTP/1.1 200 OK
[DEBUG] headers - << Server: nginx
[DEBUG] headers - << Date: Thu, 01 Aug 2013 12:04:12 GMT
[DEBUG] headers - << Content-Type: text/html
[DEBUG] headers - << Connection: keep-alive
...
The response message returned by the server for this URI does not explicitly specify the charset of the content. In such cases HttpClient is forced to use the default charset encoding for HTTP content, which is ISO-8859-1 and not UTF-8.
Unfortunately the only way to override the default content charset used by fluent API is by using a custom response handler
ResponseHandler<String> myHandler = new ResponseHandler<String>() {
#Override
public String handleResponse(
final HttpResponse response) throws IOException {
return EntityUtils.toString(response.getEntity(), Consts.UTF_8);
}
};
String html = Request.Get("https://kokos.pl/").execute().handleResponse(myHandler);
System.out.println(html);