Camel route loop not working - json

I am trying to insert json data in mySQL database using camel and hibernate.
Everything is working.
for (Module module : modules) {
from("timer://foo?delay=10000")
.loop(7)//not working
.to(module.getUrl() + "/api/json")
.convertBodyTo(String.class)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
int index = (Integer)exchange.getProperty("CamelLoopIndex"); // not working
ObjectMapper mapper = new ObjectMapper();
JsonNode root = mapper.readTree(exchange.getIn().getBody().toString());
String[] lijst = {"lastBuild", "lastCompletedBuild", "lastFailedBuild", "lastStableBuild", "lastSuccessfulBuild", "lastUnstableBuild", "lastUnsuccessfulBuild"};
JSONObject obj = new JSONObject();
JsonNode node = root.get(lijst[index]);
JsonNode build = node.get("number");
obj.put("description", lijst[index]);
obj.put("buildNumber", build);
exchange.getIn().setBody(obj.toString());
}
})
.unmarshal(moduleDetail)
.to("hibernate:be.kdg.teamf.model.ModuleDetail")
.end();
}
When I debug, my CamelLoopIndex remains 0 so it is not incremented every time it goes through the loop.
All help is welcome!

In your case the only first instruction is processed in scope of the loop: .to(module.getUrl() + "/api/json"). You can add more instructions into a loop using Spring DSL, but I don't know how to declare a loop scope using Java DSL explicitly. I hope experts will explain more about a loop scope in Java DSL.
As a workaround I suggest to move all iteration instructions to a separate direct: route.

I can't reproduce your problem. This works:
from("restlet:http://localhost:9010}/loop?restletMethod=get")
.loop(7)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
int index = (int) exchange.getProperty("CamelLoopIndex");
exchange.getIn().setBody("index=" + index);
}
})
.convertBodyTo(String.class)
.end();
Output:
index=6

Related

how to fix NullPointerException while loading document to Elasticsearch 7.3

I want to load a JSON string to Elasticsearch version 7.3.
Following is the code i am using for this.
private RestHighLevelClient restHighLevelClient;
String jsonString="//here the complete JSON string";
JSONObject jsonObject = new JSONObject(cojsonStringntent1.toString());
HashMap<String, Object> hashMap = new Gson().fromJson(jsonObject.toString(), HashMap.class);
IndexRequest indexRequest = new IndexRequest("index", "type").source(hashMap);
restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
Exception :
Exception in thread "main" java.lang.NullPointerException
at line restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
If I post the same jsonString via POSTMEN than it is being loaded to ELASTICSEARCH perfectly.
If you are not using spring(as it's not mentioned), you can use below simple code to create a resthighlevelclient.
In below code, I am reading the elasticsearch configuration from a config file, feel free to change it to the way you read the properties or config, or if you just want to quickly test it hardcode the values of host and port
RestHighLevelClient restHighLevelClient = new RestHighLevelClient(
RestClient.builder(new HttpHost(configuration.getElasticsearchConfig().getHost(),
configuration.getElasticsearchConfig().getPort(),
"http")));
Based on your sample code, your restHighLevelClient hasn't been initialized indeed at all. Please find snippet of code below how you could solve this:
#Bean
public RestHighLevelClient elasticRestClient () {
String[] httpHosts = httpHostsProperty.split(";");
HttpHost[] httpHostsAsArray = new HttpHost[httpHosts.length];
int index = 0;
for (String httpHostAsString : httpHosts) {
HttpHost httpHost = new HttpHost(httpHostAsString.split(":")[0], new Integer(httpHostAsString.split(":")[1]), "http");
httpHostsAsArray[index++] = httpHost;
}
RestClientBuilder restClientBuilder = RestClient.builder(httpHostsAsArray)
.setRequestConfigCallback(builder -> builder
.setConnectTimeout(connectTimeOutInMs)
.setSocketTimeout(socketTimeOutInMs)
);
return new RestHighLevelClient(restClientBuilder);
}
and your impl class uses the autowired RestHighLevelClient bean:
#Autowired
private RestHighLevelClient restClient;

Camel - CSV Headers setting not working

I have CSV files without headers. Since I'm using 'useMaps' I want to specify the headers dynamically. If I set headers statically and then use in route it works fine as below Approach 1 -
#Component
public class BulkActionRoutes extends RouteBuilder {
#Override
public void configure() throws Exception {
CsvDataFormat csv = new CsvDataFormat(",");
csv.setUseMaps(true);
ArrayList<String> list = new ArrayList<String>();
list.add("DeviceName");
list.add("Brand");
list.add("status");
list.add("type");
list.add("features_c");
list.add("battery_c");
list.add("colors");
csv.setHeader(list);
from("direct:bulkImport")
.convertBodyTo(String.class)
.unmarshal(csv)
.split(body()).streaming()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
GenericObjectModel model = null;
HashMap<String, String> csvRecord = (HashMap<String, String>)exchange.getIn().getBody();
}
});
}
}
However, if the list is passed via Camel headers as below then it does not work Approach 2 -
#Component
public class BulkActionRoutes extends RouteBuilder {
#Override
public void configure() throws Exception {
CsvDataFormat csv = new CsvDataFormat(",");
csv.setUseMaps(true);
from("direct:bulkImport")
.convertBodyTo(String.class)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
ArrayList<String> fileHeaders = (ArrayList<String>)headers.get(Constants.FILE_HEADER_LIST);
if (fileHeaders != null && fileHeaders.size() > 0) {
csv.setHeader(fileHeaders);
}
}
})
.unmarshal(csv)
.split(body()).streaming()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
GenericObjectModel model = null;
HashMap<String, String> csvRecord = (HashMap<String, String>)exchange.getIn().getBody();
}
});
}
}
What could be missing in the Approach 2?
The big difference between approach 1 and 2 is the scope.
In approach 1 you fully configure the CSV data format. This is all done when the Camel Context is created, since the data format is shared within the Camel Context. When messages are processed, it is the same config for all messages.
In approach 2 you just configure the basics globally. The header configuration is within the route and therefore can change for every single message. Every message would overwrite the header configuration of the context-global data format instance.
Without being sure about this, I guess that it is not possible to change a context-global DataFormat inside the routes.
What would you expect (just for example) when messages are processed in parallel? They would overwrite the header config against each other.
As an alternative, you could use a POJO where you can do your dynamic marshal / unmarshal from Java code.

Apache HttpComponents CookieStore Not Storing Cookies

I'm using HttpComponents 4.5.2 and I'm trying to store cookies as I need to use them for login and other requests. The code works fine whilst the application is still running, but the problem here is when I restart it, the cookies that were supposed to be stored in CookieStore are not there. Here's what I've written:
public static void main( String[] args ) throws InterruptedException
{
RequestConfig globalConfig = RequestConfig.custom()
.setCookieSpec(CookieSpecs.STANDARD).build();
BasicCookieStore cookieStore = new BasicCookieStore();
HttpClientContext context = HttpClientContext.create();
context.setCookieStore(cookieStore);
CloseableHttpAsyncClient httpclient = HttpAsyncClients.custom()
.setDefaultRequestConfig(globalConfig)
.setDefaultCookieStore(cookieStore)
.build();
httpclient.start();
login(httpclient, context);
}
public static void login(CloseableHttpAsyncClient httpClient, HttpClientContext context) throws InterruptedException
{
JSONObject json = new JSONObject("{ email : blahblahblah1, password : blahblahblah2 }");
StringEntity requestEntity = new StringEntity(
json.toString(),
ContentType.APPLICATION_JSON);
HttpPost postMethod = new HttpPost("http://localhost:8080/login");
postMethod.setEntity(requestEntity);
final CountDownLatch latch = new CountDownLatch(1);
httpClient.execute(postMethod, context, new FutureCallback<HttpResponse>() {
public void completed(final HttpResponse response) {
latch.countDown();
System.out.println(postMethod.getRequestLine() + "->" + response.getStatusLine());
//System.out.println(context.getCookieStore().getCookies().size());
}
public void failed(final Exception ex) {
latch.countDown();
System.out.println(postMethod.getRequestLine() + "->" + ex);
}
public void cancelled() {
latch.countDown();
System.out.println(postMethod.getRequestLine() + " cancelled");
}
});
latch.await();
}
I've read the HttpComponents documentation and the section 3.5 about cookies says:
HttpClient can work with any physical representation of a persistent cookie store that implements the CookieStore interface. The default CookieStore implementation called BasicCookieStore is a simple implementation backed by a java.util.ArrayList. Cookies stored in an BasicClientCookie object are lost when the container object get garbage collected. Users can provide more complex implementations if necessary
So I'm wondering if it's left to it's users to implement some kind of structure that can effectively store cookies or if I'm missing something.
Yes, using BasicCookieStore backed by ArrayList means that when your jvm exists, the data there is being lost just like any ArrayList in memory.
BasicCookieStore class also implements Serializable so you can use that to persist it to disk and restore back on your app startup if the file was there.
You can borrow some code from the tests verifying that flow TestBasicCookieStore#testSerialization.

Simple camel cxfrs consumer that consumes json and creates a map

I am struggling on a simple task. I want to create a cxfrs consumer that simply consumes json.
The json should be converted to a simple map (key->value): I created a simple test:
#Test
public final void test() throws Exception {
MockEndpoint mockOut = context.getEndpoint(MOCK_OUT, MockEndpoint.class);
mockOut.expectedMessageCount(1);
context.addRoutes(createRouteBuilder());
context.start();
context.createProducerTemplate().sendBody(DIRECT_A, "{ \"ussdCode\":\"101#\",\"msisdn\":\"491234567\"}");
mockOut.assertIsSatisfied();
}
private RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from(DIRECT_A).to("cxfrs://http://localhost:8085/ussd");
from("cxfrs://http://localhost:8085/ussd")
.unmarshal().json(JsonLibrary.Jackson)
.process(to).to(MOCK_OUT);
}
};
}
The problem is on context.start() i get ServiceConstructionException: No resource classes found. I also tried to create the consumer this way (setting binding style):
private Endpoint fromCxfRsEndpoint() {
CxfRsEndpoint cxfRsEndpoint = context.getEndpoint("cxfrs://http://localhost:8085/ussd", CxfRsEndpoint.class);
cxfRsEndpoint.setBindingStyle(BindingStyle.SimpleConsumer);
return cxfRsEndpoint;
}
This didn't helped neither. So how to create a simple rest/json consumer and unmarshal to a simple map?

Apache Camel Integration Test - NotifyBuilder

I am writing integration tests to test existing Routes. The recommended way of getting the response looks something like this (via Camel In Action section 6.4.1):
public class TestGetClaim extends CamelTestSupport {
#Produce(uri = "seda:getClaimListStart")
protected ProducerTemplate producer;
#Test
public void testNormalClient() {
NotifyBuilder notify = new NotifyBuilder(context).whenDone(1).create();
producer.sendBody(new ClientRequestBean("TESTCLIENT", "Y", "A"));
boolean matches = notify.matches(5, TimeUnit.SECONDS);
assertTrue(matches);
BrowsableEndpoint be = context.getEndpoint("seda:getClaimListResponse", BrowsableEndpoint.class);
List<Exchange> list = be.getExchanges();
assertEquals(1, list.size());
System.out.println("***RESPONSE is type "+list.get(0).getIn().getBody().getClass().getName());
}
}
The test runs but I get nothing back. The assertTrue(matches) fails after the 5 second timeout.
If I rewrite the test to look like this I get a response:
#Test
public void testNormalClient() {
producer.sendBody(new ClientRequestBean("TESTCLIENT", "Y", "A"));
Object resp = context.createConsumerTemplate().receiveBody("seda:getClaimListResponse");
System.out.println("***RESPONSE is type "+resp.getClass().getName());
}
The documentation is a little light around this so can anyone tell me what I am doing wrong with the first approach? Is there anything wrong with following the second approach instead?
Thanks.
UPDATE
I have broken this down and it looks like the problem is with the mix of seda as the start endpoint in combination with the use of a recipientList in the Route. I've also changed the construction of the NotifyBuilder (I had the wrong endpoint specified).
If I change the start endpoint to
direct instead of seda then the test will work; or
If I comment out the recipientList
then the test will work.
Here's a stripped down version of my Route that reproduces this issue:
public class TestRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
// from("direct:start") //works
from("seda:start") //doesn't work
.recipientList(simple("exec:GetClaimList.bat?useStderrOnEmptyStdout=true&args=${body.client}"))
.to("seda:finish");
}
}
Note that if I change the source code of the NotifyTest from the "Camel In Action" source to have a route builder like this then it also fails.
Try to use "seda:getClaimListResponse" in the getEndpoint to be sure the endpoint uri is 100% correct
FWIW: It appears that notifyBuilder in conjunction with seda queues are not quite working: a test class to illustrate:
public class NotifyBuilderTest extends CamelTestSupport {
// Try these out!
// String inputURI = "seda:foo"; // Fails
// String inputURI = "direct:foo"; // Passes
#Test
public void testNotifyBuilder() {
NotifyBuilder b = new NotifyBuilder(context).from(inputURI)
.whenExactlyCompleted(1).create();
assertFalse( b.matches() );
template.sendBody(inputURI, "Test");
assertTrue( b.matches() );
b.reset();
assertFalse( b.matches() );
template.sendBody(inputURI, "Test2");
assertTrue( b.matches() );
}
#Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from(inputURI).to("mock:foo");
}
};
}
}