Camel Fuse Route from JSON to JSON not working - json

I am playing with the Camel Fuse tooling to convert from JSON to JSON through a data mapper. I have been able to do conversions from XML to JSON using data mappers.
However, when I try to receive a json object and then data map it and then send it the data mapping fails with the below message.
Note that I am building something that will run on a Tomcat server, that is why I am using the camel-config.xml file.
Any thoughts on what might be amiss?
Patrik
java.lang.NullPointerException
at org.apache.camel.component.dozer.DozerProducer.process(DozerProducer.java:78)
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:141)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:460)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:121)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.component.jetty.CamelContinuationServlet.service(CamelContinuationServlet.java:162)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at org.eclipse.jetty.servlets.MultiPartFilter.doFilter(MultiPartFilter.java:146)
at org.apache.camel.component.jetty.CamelFilterWrapper.doFilter(CamelFilterWrapper.java:43)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Unknown Source)java.lang.NullPointerException
at org.apache.camel.component.dozer.DozerProducer.process(DozerProducer.java:78)
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:141)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:460)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:121)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.component.jetty.CamelContinuationServlet.service(CamelContinuationServlet.java:162)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at org.eclipse.jetty.servlets.MultiPartFilter.doFilter(MultiPartFilter.java:146)
at org.apache.camel.component.jetty.CamelFilterWrapper.doFilter(CamelFilterWrapper.java:43)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Unknown Source)
Here is my input .json
{
"id": "138784",
"fields":
{
"description": "Maxed",
"summary": "Max is my name",
"created": "2015-09-28",
"duedate": "2015-09-28",
"updated": "2015-09-28"
}
}
Here is my expected output:
{
"theType" : "Transaction",
"theId" : "0",
"attributes" : {
"valuationDate" : "",
"amount" : "108.15",
"valueDate" : "",
"description" : "description 0",
"type" : "withdrawal",
"verificationId" : "verificationId 0"
},
"type" : "Transaction",
"id" : "0"
}
Here is my transformation map:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<mappings xmlns="http://dozer.sourceforge.net" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://dozer.sourceforge.net http://dozer.sourceforge.net/schema/beanmapping.xsd">
<configuration>
<wildcard>false</wildcard>
</configuration>
<mapping>
<class-a>input.Input</class-a>
<class-b>transaction1.Transaction1</class-b>
<field>
<a>id</a>
<b>attributes.verificationId</b>
</field>
</mapping>
<mapping>
<class-a>input.Fields</class-a>
<class-b>transaction1.Attributes</class-b>
<field>
<a>description</a>
<b>description</b>
</field>
<field>
<a>created</a>
<b>valuationDate</b>
</field>
<field>
<a>duedate</a>
<b>valueDate</b>
</field>
<field>
<a>summary</a>
<b>type</b>
</field>
<field>
<a>updated</a>
<b>amount</b>
</field>
</mapping>
<mapping>
<class-a>org.apache.camel.component.dozer.ExpressionMapper</class-a>
<class-b>transaction1.Transaction1</class-b>
<field custom-converter-id="_expressionMapping" custom-converter-param="constant:Transaction">
<a>expression</a>
<b>theType</b>
</field>
<field custom-converter-id="_expressionMapping" custom-converter-param="constant:0">
<a>expression</a>
<b>theId</b>
</field>
<field custom-converter-id="_expressionMapping" custom-converter-param="constant:2">
<a>expression</a>
<b>id</b>
</field>
</mapping>
</mappings>
And finally here is my route config in the file: camel-config.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- here we have the Camel route(s). -->
<!-- we must still use the http://camel.apache.org/schema/spring namespace so Camel can load the routes
though Spring JARs is not required -->
<!-- incoming requests from the servlet is routed -->
<!-- is there a header with the key name? -->
<!-- yes so return back a message to the user -->
<!-- if no name parameter then output a syntax to the user -->
<routes xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="jetty:http://localhost:8091/camel/input/input"/>
<log message="Log1 ${body}"/>
<to uri="dozer:transformation11?sourceModel=input.Input&targetModel=transaction0.Transaction0&mappingFile=transformation.xml"/>
<log message="Log2 ${body}"/>
<marshal>
<json library="Jackson"/>
</marshal>
<to uri="jetty:http://localhost:8088/camel/transaction/output?bridgeEndpoint=true&throwExceptionOnFailure=false"/>
</route>
</routes>

Although it is great that JBoss Fuse is getting a graphical mapper I think since it is a new component you should probably give it some time before they fix eventual bugs and optimise the performance. At least for me when I was testing it, it ran quit slowly.
For pure json-to-json mapping perhaps you can look at the component camel-jolt.
http://camel.apache.org/jolt.html

Related

Mail sending through gmail in Mule Flow

I am struggling with mail sending flow in mule, Getting below error:
Failed to route event via endpoint: DefaultOutboundEndpoint{endpointUri=smtps://dummy.DevGroup:<password>#smtp.gmail.com, connector=SmtpsConnector
I have below configuration in my xml file (Updated with namespace declarations in the Mule XML):
<mule
xmlns="http://www.mulesoft.org/schema/mule/core"
xmlns:twilio="http://www.mulesoft.org/schema/mule/twilio"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:jee="http://www.springframework.org/schema/jee"
xmlns:stdio="http://www.mulesoft.org/schema/mule/stdio"
xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
xmlns:file="http://www.mulesoft.org/schema/mule/file"
xmlns:cxf="http://www.mulesoft.org/schema/mule/cxf"
xmlns:jms="http://www.mulesoft.org/schema/mule/jms"
xmlns:jaxws="http://cxf.apache.org/jaxws"
xmlns:smtp="http://www.mulesoft.org/schema/mule/smtp"
xmlns:smtps="http://www.mulesoft.org/schema/mule/smtps"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd
http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-2.5.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/stdio http://www.mulesoft.org/schema/mule/stdio/current/mule-stdio.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/cxf http://www.mulesoft.org/schema/mule/cxf/current/mule-cxf.xsd
http://www.mulesoft.org/schema/mule/jms http://www.mulesoft.org/schema/mule/jms/current/mule-jms.xsd
http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd
http://www.mulesoft.org/schema/mule/smtp http://www.mulesoft.org/schema/mule/smtp/3.3/mule-smtp.xsd
http://www.mulesoft.org/schema/mule/smtps http://www.mulesoft.org/schema/mule/smtps/current/mule-smtps.xsd
http://www.mulesoft.org/schema/mule/twilio http://www.mulesoft.org/schema/mule/twilio/1.0/mule-twilio.xsd">
<service name="tool.muleservice.emailServiceintegration">
<inbound>
<inbound-endpoint ref="tool.endpoint.emailNotification"/>
</inbound>
<outbound>
<pass-through-router>
<outbound-endpoint ref="tool.smtp.endpoint.emailNotification"/>
</pass-through-router>
</outbound>
</service>
<smtps:endpoint name="tool.smtp.endpoint.emailNotification"
user="#[message.inboundProperties['userName']]"
password="#[message.inboundProperties['password']]"
host="smtp.gmail.com"
port="465"
from="dummy.DevGroup%40gmail.com"
transformer-refs="customEmailTransformer"
connector-ref="myTSLSMTPConnnector">
</smtps:endpoint>
<smtps:connector name="myTSLSMTPConnnector">
<smtps:tls-client path="../cacerts" storePassword="changeit" />
<smtps:tls-trust-store path="../cacerts" storePassword="changeit" />
</smtps:connector>
I updated the xml file as below content, to use smtp-gmail-connector given here
<service name="tool.muleservice.emailServiceintegration">
<inbound>
<inbound-endpoint
ref="tool.endpoint.emailNotification" />
</inbound>
<outbound>
<pass-through-router>
<!-- <outbound-endpoint ref="tool.smtp.endpoint.emailNotification"/> -->
<smtp:outbound-endpoint connector-ref="smtpGmailConnector"
subject="test msg"
address="smtps://#[message.inboundProperties['userName']]:#[message.inboundProperties['password']]#smtp.gmail.com"
responseTimeout="10000" doc:name="Send notification email" />
</pass-through-router>
</outbound>
</service>
<smtp:gmail-connector name="smtpGmailConnector"
contentType="text/html" fromAddress="iip.claims.test#gmail.com"
replyToAddresses="iip.claims.test#gmail.com" >
<smtp:header key="foo" value="bar" />
<smtp:header key="baz" value="boz" />
</smtp:gmail-connector>
But after this I am getting below error:
Caused by: org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Failed to import bean definitions from URL location [classpath*:/myApp-component-config/component-config.xml]
Offending resource: mule-bootstrap-config.xml; nested exception is org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Failed to import bean definitions from URL location [classpath*:/myApp-integration-config/tool-mule-bootstrap-config.xml]
Offending resource: URL [vfs:/D:/Software/JBoss/jboss-eap-7.2/standalone/deployments/myApp.war/WEB-INF/lib/tool-intrg.jar/myApp-component-config/component-config.xml]; nested exception is org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Failed to import bean definitions from URL location [classpath*:/myApp-integration-config/tool-intrg-flows.xml]
Offending resource: URL [vfs:/D:/Software/JBoss/jboss-eap-7.2/standalone/deployments/myApp.war/WEB-INF/lib/tool-intrg.jar/myApp-integration-config/tool-mule-bootstrap-config.xml]; nested exception is org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException: Line 91 in XML document from URL [vfs:/D:/Software/JBoss/jboss-eap-7.2/standalone/deployments/myApp.war/WEB-INF/lib/tool-intrg.jar/myApp-integration-config/tool-intrg-flows.xml] is invalid; nested exception is org.xml.sax.SAXParseException; lineNumber: 91; columnNumber: 67; The prefix "doc" for attribute "doc:name" associated with an element type "smtp:outbound-endpoint" is not bound.
at deployment.myApp.war//org.springframework.beans.factory.parsing.FailFastProblemReporter.error(FailFastProblemReporter.java:70)
at deployment.myApp.war//org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:85)
at deployment.myApp.war//org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:76)
at deployment.myApp.war//org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.importBeanDefinitionResource(DefaultBeanDefinitionDocumentReader.java:235)
Any help will be appreciated.
There are many details missing however on rereading the issue I suspect you are not following the deprecated syntax <service> correctly by trying to use a current example. Since that syntax has been deprecated for 6+ years, I would advice to move to the more current <flow> definitions.
In case you are unable or unwilling to migrate, you need to define an endpoint outside the service and reference it you could try the following steps:
Ensure that the namespaces at the beginning of the file are correct. The error mentions that doc: is not recognized. Maybe the namespaces are outdated.
Ensure that the dependencies in the pom reference your Mule 3.9.x release and not an older version that may not define doc.
Try moving <smtp:outbound-endpoint> to a separate endpoint definition and reference it from the <pass-through-router> as in your original implementation.
Example:
<service name="tool.muleservice.emailServiceintegration">
<inbound>
<inbound-endpoint ref="tool.endpoint.emailNotification"/>
</inbound>
<outbound>
<pass-through-router>
<outbound-endpoint ref="tool.smtp.endpoint.gmailNotification"/>
</pass-through-router>
</outbound>
</service>
<smtps:endpoint name="tool.smtp.endpoint.gmailNotification"
connector-ref="smtpGmailConnector"
subject="test msg"
address="smtps://#[message.inboundProperties['userName']]:#[message.inboundProperties['password']]#smtp.gmail.com"
responseTimeout="10000" doc:name="Send notification email" >
</smtps:endpoint>
<smtp:gmail-connector name="smtpGmailConnector"
contentType="text/html" fromAddress="iip.claims.test#gmail.com"
replyToAddresses="iip.claims.test#gmail.com" >
<smtp:header key="foo" value="bar" />
<smtp:header key="baz" value="boz" />
</smtp:gmail-connector>
Finally if everything else fails you could try removing the doc:name attribute and see what happens.

Adding custom field to log4j json layout breaks spring logging structure

I have a Spring Boot (2.2.6) application that uses Log4j2 (with Slf4j). Log4j is configured to use the json layout and in the end I want to ingest the logs in Datadog. For that the 'serviceName' is important as a field in the json.
Now according to the log4j docu (https://logging.apache.org/log4j/2.x/manual/layouts.html#JSONLayout) one can add a custom field with the 'KeyValuePair' tags and that works. Unfortunately this breaks the normal structure of the spring logs.
Log4j2.xml config:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<!-- Write logs to stdout, JSON, one line per log event -->
<Console name="Console" target="SYSTEM_OUT">
<JSONLayout compact="true" eventEol="true" includeStacktrace="true" locationInfo="true"
stacktraceAsString="true" properties="true">
<KeyValuePair key="serviceName" value="$${env:APPLICATION_NAME:-local}-sidecar"/> <!-- fine w/o this line -->
</JSONLayout>
</Console>
</Appenders>
<Loggers>
<Logger name="my.service" level="debug" additivity="false">
<AppenderRef ref="Console"/>
</Logger>
<Logger name="org.springframework" level="info" additivity="false">
<AppenderRef ref="Console"/>
</Logger>
<Logger name="org.apache" level="info" additivity="false">
<AppenderRef ref="Console"/>
</Logger>
<Root level="info">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
Log w/o custom field:
{
"thread": "main",
"level": "INFO",
"loggerName": "com.nginxtrafficsidecar.ApplicationKt",
"message": "Starting ApplicationKt on xxx with PID 17300 (C:\\Users\\Felix\\Documents\\code\\nginx-traffic-sidecar\\build\\classes\\kotlin\\main started by Felix in C:\\Users\\Felix\\Documents\\code\\nginx-traffic-sidecar)",
"endOfBatch": false,
"loggerFqcn": "org.apache.commons.logging.LogAdapter$Log4jLog",
"threadId": 1,
"instant": {
"epochSecond": 1587975181,
"nanoOfSecond": 331370300
},
"source": {
"class": "org.springframework.boot.StartupInfoLogger",
"method": "logStarting",
"file": "StartupInfoLogger.java",
"line": 55,
"classLoaderName": "app"
},
"contextMap": {},
"threadPriority": 5
}
log w/ custom field:
{
"logEvent": "Starting ApplicationKt on xxx with PID 9732 (C:\\Users\\Felix\\Documents\\code\\nginx-traffic-sidecar\\build\\classes\\kotlin\\main started by Felix in C:\\Users\\Felix\\Documents\\code\\nginx-traffic-sidecar)",
"serviceName": "local-sidecar"
}
Spring docu mentions how this might work with logback, but not log4j (https://docs.spring.io/spring-boot/docs/current/reference/html/spring-boot-features.html#boot-features-custom-log-configuration, end of chapter)
I've searched but couldn't find anything useful. Any ideas how i can add a custom field to the json log while still preserving all fields coming from Spring?
Thanks,
Felix
download Jackson Annotations and import it because JsonLayout use Jackson Annotations internal code.
in my case, download and import these
jackson-annotations-2.10.3.jar
jackson-core-2.10.3.jar
jackson-databind-2.10.3.jar
add <JsonLayout>...</JsonLayout> in your config(in my case log4j2.xml) like below
<JsonLayout>
<KeyValuePair key="testKey" value="testValue"/>
</JsonLayout>
Replace key and value to what you want to use. In my case key is "testKey" and value is "testValue"
run and check your log!
it it my full sample code and xml configuration info.
code
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
public class LogTest{
private static final Logger logger = LogManager.getLogger(LogTest.class.getName());
public static void main(String[] args) {
ThreadContext.put("logFileName", "testFile1" );
logger.info("log printed! - testFile1");
ThreadContext.put("logFileName", "testFile2" );
logger.info("log printed! - testFile2");
ThreadContext.remove("logFileName");
}
}
log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Routing name="RoutingAppender">
<Routes pattern="${ctx:logFileName}">
<Route>
<RollingFile name="Rolling-${ctx:logFileName}"
fileName="./logs/${ctx:logFileName}.log"
filePattern="./logs/${ctx:logFileName}.%i.log.gz">
<JsonLayout>
<KeyValuePair key="testKey" value="testValue"/>
</JsonLayout>
<SizeBasedTriggeringPolicy size="512" />
</RollingFile>
</Route>
</Routes>
</Routing>
</Appenders>
<Loggers>
<Root level="all">
<AppenderRef ref="RoutingAppender" />
</Root>
</Loggers>
</Configuration>
output
{
"instant" : {
"epochSecond" : 1588590944,
"nanoOfSecond" : 469000000
},
"thread" : "main",
"level" : "INFO",
"loggerName" : "com.test.LogTest",
"message" : "log printed! - testFile2",
"endOfBatch" : false,
"loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
"threadId" : 1,
"threadPriority" : 5,
"testKey" : "testValue"
}

Camel DataFormat Jackson using blueprint XML DSL throws context exception

No matter where I place the dataformats in XML DSL blueprint, I get this error just starting at different places. if I remove it, it works but of course I can't convert JSON to POJO. ??? any help or tell me what I'm doing wrong, what i'm missing. thanks!
Error
Unable to start blueprint container for bundle passthrumt1.core/1.0.1.SNAPSHOT
Caused by: org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'endpoint'. One of '{"http://camel.apache.org/schema/blueprint":redeliveryPolicyProfile, "http://camel.apache.org/schema/blueprint":onException, "http://camel.apache.org/schema/blueprint":onCompletion, "http://camel.apache.org/schema/blueprint":intercept, "http://camel.apache.org/schema/blueprint":interceptFrom, "http://camel.apache.org/schema/blueprint":interceptSendToEndpoint, "http://camel.apache.org/schema/blueprint":restConfiguration, "http://camel.apache.org/schema/blueprint":rest, "http://camel.apache.org/schema/blueprint":route}' is expected.
XML DSL
<camelContext
id="com.passthru.coreCamelContext"
trace="true"
xmlns="http://camel.apache.org/schema/blueprint"
allowUseOriginalMessage="false"
streamCache="true"
errorHandlerRef="deadLetterErrorHandler" >
<properties>
<property key="http.proxyHost" value="PITC-Zscaler-Americas.proxy.corporate.com"/>
<property key="http.proxyPort" value="80"/>
</properties>
<streamCaching id="CacheConfig"
spoolUsedHeapMemoryThreshold="70"
anySpoolRules="true"/>
<!-- -->
<dataFormats>
<json id="Json2Pojo" library="Jackson" unmarshalTypeName="com.passthru.core.entities.TokenEntities">
</json>
</dataFormats>
<endpoint id="predixConsumer" uri="direct:preConsumer" />
<endpoint id="predixProducer" uri="direct:preProducer" />
<endpoint id="getToken" uri="direct:getToken" />
<onException>
<exception>com.passthru.dataservice.PDXDataServiceInvalidDataException</exception>
<redeliveryPolicy maximumRedeliveries="3" />
<handled>
<constant>true</constant>
</handled>
<log
message="Invalid Data From Data Service"
loggingLevel="ERROR" />
<setBody>
<simple>${body.toString}</simple>
</setBody>
<to uri="file:{{errorArchive}}" />
</onException>
If I place the dataformats above properties, it complains, I have to remove properties and streamcache statements in order for it to work. but I need the proxy properties. any suggestions??? thanks again
If the
<camelContext
id="com.ge.digital.passthru.coreCamelContext"
trace="true"
xmlns="http://camel.apache.org/schema/blueprint"
allowUseOriginalMessage="false"
streamCache="true"
errorHandlerRef="deadLetterErrorHandler" >
<dataFormats>
<json id="Json2Pojo" library="Jackson" unmarshalTypeName="com.passthru.core.entities.TokenEntities"/>
</dataFormats>
<properties>
<property key="http.proxyHost" value="PITC-Zscaler-Americas-Cincinnati3PR.proxy.corporate.com"/>
<property key="http.proxyPort" value="80"/>
</properties>
i get this
Caused by: org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'properties'. One of '{"http://camel.apache.org/schema/blueprint":redeliveryPolicyProfile, "http://camel.apache.org/schema/blueprint":onException, "http://camel.apache.org/schema/blueprint":onCompletion, "http://camel.apache.org/schema/blueprint":intercept, "http://camel.apache.org/schema/blueprint":interceptFrom, "http://camel.apache.org/schema/blueprint":interceptSendToEndpoint, "http://camel.apache.org/schema/blueprint":restConfiguration, "http://camel.apache.org/schema/blueprint":rest, "http://camel.apache.org/schema/blueprint":route}' is expected.
what am I missing?
Camel blueprint XML is validated against camel-blueprint.xsd.
You are interested in complex type with name camelContextFactoryBean which contains sequence of available elements with fixed order.
Correct order of camelContext elements defined in this sequence is:
properties
globalOptions
propertyPlaceholder
package
packageScan
contextScan
jmxAgent
streamCaching
export
defaultServiceCallConfiguration
serviceCallConfiguration
defaultHystrixConfiguration
hystrixConfiguration
routeBuilder
routeContextRef
restContextRef
threadPoolProfile
threadPool
endpoint
dataFormats
transformers
validators
redeliveryPolicyProfile
onException
onCompletion
intercept
interceptFrom
interceptSendToEndpoint
restConfiguration
rest
route
To solve your problem move all endpoint declarations right above dataFormats.

using Nlog and writing to file as json

I think I'm missing something as I can't seem to figure out how to have it write to a log file in json format using NLog setup in configuration file. The straight rolling file works fine, but not the json. The json target only outputs the message (not in json).
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets async="true">
<target xsi:type="File" name="rollingFile" fileName="${basedir}/logs/${shortdate}.log" archiveFileName="${basedir}/logs/{shortdate}_Archive{###}.log" archiveAboveSize="1000000" archiveNumbering="Sequence" layout="${longdate} ${uppercase:${level}} ${callsite} ${message}" />
<target xsi:type="File"
name="rollingFileJson"
fileName="${basedir}/logs/${shortdate}.json"
archiveFileName="${basedir}/logs/{shortdate}_Archive{###}.json"
archiveAboveSize="1000000"
archiveNumbering="Sequence"
layout="${json-encode} ${message}">
</target>
</targets>
<rules>
<logger name="*" minlevel="Trace" writeTo="rollingFile" />
<logger name="*" minlevel="Trace" writeTo="rollingFileJson" />
</rules>
</nlog>
As of the release of NLog 4.0.0 it is possible to use a layout that renders events as structured JSON documents.
Example taken from NLog project site:
<target name="jsonFile" xsi:type="File" fileName="${logFileNamePrefix}.json">
<layout xsi:type="JsonLayout">
<attribute name="time" layout="${longdate}" />
<attribute name="level" layout="${level:upperCase=true}"/>
<attribute name="message" layout="${message}" />
</layout>
</target>
Edit:
You can also create it in code.
Here is the link to the specific part of documentation.
And here the copied example:
var jsonLayout = new JsonLayout
{
Attributes =
{
new JsonAttribute("type", "${exception:format=Type}"),
new JsonAttribute("message", "${exception:format=Message}"),
new JsonAttribute("innerException", new JsonLayout
{
Attributes =
{
new JsonAttribute("type", "${exception:format=:innerFormat=Type:MaxInnerExceptionLevel=1:InnerExceptionSeparator=}"),
new JsonAttribute("message", "${exception:format=:innerFormat=Message:MaxInnerExceptionLevel=1:InnerExceptionSeparator=}"),
}
},
//don't escape layout
false)
}
};
For more info read the docs.
As per NLog documentation: json-encode will only escape output of another layout using JSON rules. It will not "convert" the output to JSON. You'll have to do that yourself.
'{ "date":"${longdate}","level":"${level}","message":${message}}'
Take a look at this question for more details.

Strange Breeze error when query DOES return data (update: Metadata not loading correctly)

I'm using a ODataController to return data to the breeze dataservice, this is the dataservice
app.dataservice = (function (breeze) {
breeze.config.initializeAdapterInstances({ dataService: "OData" });
var manager = new breeze.EntityManager('/api/v1/');
return {
getRecipePage: getRecipePage
};
function getRecipePage(skip, take, searchText) {
var query = breeze.EntityQuery
.from("Recipes")
.orderBy("Name")
.skip(skip).take(take)
.inlineCount(true);
if (searchText) {
query = query.where("Name", "contains", searchText);
}
return manager.executeQuery(query);
}
})(breeze);
when calling the getRecipePage function in my controller, it seems to return data properly, but the exceptions is strange
getDataFunction(skip, take)
.then(function (largeLoad) {
$scope.setPagedData(largeLoad, currentPage, pageSize);
})
.fail(function (e) {
debugger;
});
The e variables has the message "; ", which makes no sense. The status is "200 OK" which is good.
The body contains my two entities and the url appers correct "/api/v1/Recipes?$orderby=Name&$top=2&$inlinecount=allpages", if I navigate to it, the json looks good:
{
"$id": "1",
"$type": "Breeze.WebApi2.QueryResult, Breeze.WebApi2",
"Results": [
{
"$id": "2",
"$type": "RecipeBook.Web.Angular.Models.RecipeBook.Recipe, RecipeBook.Web.Angular",
"Name": "1 Boiled Water",
"Description": "6 Steamy goodness!",
"Id": 1
},
{
"$id": "3",
"$type": "RecipeBook.Web.Angular.Models.RecipeBook.Recipe, RecipeBook.Web.Angular",
"Name": "2 Hot Chocolate",
"Description": "5 Chocolatey Chocolateness!",
"Id": 2
}
],
"InlineCount": 6
}
...what is this error? Lastly, here is the stack:
Error
at createError (http://localhost:62576/Scripts/breeze.debug.js:15182:22)
at http://localhost:62576/Scripts/breeze.debug.js:14971:40
at http://localhost:62576/Scripts/datajs-1.1.1.js:1671:17
at XMLHttpRequest.odata.defaultHttpClient.request.xhr.onreadystatechange (http://localhost:62576/Scripts/datajs-1.1.1.js:2587:25)
Thoughts as to what is going on???
EDIT:
After a ton of digging, I've somewhat narrowed down the issue to be related to the handler that reads the response. In datajs-1.1.1.js ~line 8100 there is a dispatchHandler function. I have a requestOrResponse that came back from the OData call:
It has a body property with the above json text. The data property is undefined however, but I think that's what it's trying to translate the body into...and is looking for a handler to do so. It's statusCode is 200, and statusText is OK. But the method doesn't find an appropriate handler and throws:
throw { message: "no handler for data" };
...this appears to be where the error originates. I just have no clue what's not setup correctly so that I can remedy the situation.
EDIT2:
It might actually be caused because the metadata (xml) isn't being parsed correctly..., this is what it looks like (taken from the datajs handlerRead function while debugging)
<?xml version="1.0" encoding="utf-8"?>
<edmx:Edmx Version="1.0" xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx">
<edmx:DataServices m:DataServiceVersion="3.0" m:MaxDataServiceVersion="3.0" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
<Schema Namespace="RecipeBook.Web.Angular.Models.Recipe" xmlns="http://schemas.microsoft.com/ado/2009/11/edm">
<EntityType Name="Recipe">
<Key>
<PropertyRef Name="Id" />
</Key>
<Property Name="Id" Type="Edm.Int32" Nullable="false" />
<Property Name="Name" Type="Edm.String" />
<Property Name="Description" Type="Edm.String" />
<Property Name="Steps" Type="Collection(Edm.String)" />
<NavigationProperty Name="Ingredients" Relationship="RecipeBook.Web.Angular.Models.Recipe.RecipeBook_Web_Angular_Models_Recipe_Recipe_Ingredients_RecipeBook_Web_Angular_Models_Recipe_RecipeIngredient_IngredientsPartner" ToRole="Ingredients" FromRole="IngredientsPartner" />
</EntityType>
<EntityType Name="RecipeIngredient">
<Key>
<PropertyRef Name="Id" />
</Key>
<Property Name="Id" Type="Edm.Int32" Nullable="false" />
<Property Name="IngredientId" Type="Edm.Int32" Nullable="false" />
<Property Name="Quantity" Type="Edm.Int32" Nullable="false" />
<Property Name="UnitOfMeasureId" Type="Edm.Int32" Nullable="false" />
<Property Name="Notes" Type="Edm.String" />
</EntityType>
<Association Name="RecipeBook_Web_Angular_Models_Recipe_Recipe_Ingredients_RecipeBook_Web_Angular_Models_Recipe_RecipeIngredient_IngredientsPartner">
<End Type="RecipeBook.Web.Angular.Models.Recipe.RecipeIngredient" Role="Ingredients" Multiplicity="*" />
<End Type="RecipeBook.Web.Angular.Models.Recipe.Recipe" Role="IngredientsPartner" Multiplicity="0..1" />
</Association>
<EntityContainer Name="Container" m:IsDefaultEntityContainer="true">
<EntitySet Name="Recipes" EntityType="RecipeBook.Web.Angular.Models.Recipe.Recipe" />
</EntityContainer>
</Schema>
</edmx:DataServices>
</edmx:Edmx>
EDIT3:
...so, if I use OData, as my dataService I need json metadata found at $metadata. If I use WebAPI it looks for metadata at /Metadata, and this can be Edmx or json. However, my metadata is returned as Edmx at $metadata ... which is the one thing not supported?
http://www.breezejs.com/documentation/breeze-metadata-details
I'm about to throw all this out (angular, breeze, odata) and just do it the old way.
Edit4:
This isn't a fix, but if I turn off metadata it "works"...so my issue is definitely related to Metadata not loading properly.
var dataService = new breeze.DataService({
serviceName: "/api/v1/",
hasServerMetadata: false
});
It looks like you are using Web API. Consider that you could decorate your controllers with the [BreezeController] attribute and instead of specifying OData you could use webApi which extends the configuration a bit. It's worth trying.
Also you probably need to configure the Breeze adapter to use backingStore which pairs well with Angular. - http://www.breezejs.com/samples/todo-dataservice (this link has some helpful tips to guide you through setting up Breeze to work well with Angular)
Last, remember that setting up any library the first time you use it always seems more difficult than it really is. Once you get it configured you almost never touch configuration again and instead Breeze just works and is really awesome.
EDIT
Check this link which has a brief walk-thru on Breeze, Angular, OData, and Web API - http://sravi-kiran.blogspot.com/2013/11/UsingBreezeJsToConsumeAspNetWebApiODataInAnAngularJsApplication.html
Another good 'How do I...' answer here by Ward -
How to consume OData service with Html/Javascript?