Mule ESB dynamically set Endpoint Path - esb

I'm trying to build a web service that accepts a number of variables used to build the path for Bing query API. For some reason, if I build the URI my flow recognizes that the new URI is assigned appropriately to the payload, but when I assign the payload to the out-bound HTTP endpoint, it fails.
<flow name="BingQuery" doc:name="BingQuery">
<http:inbound-endpoint exchange-pattern="one-way" host="localhost" port="8082" doc:name="HTTP" />
<set-variable variableName="query" doc:name="Variable" value="#[header:INBOUND:query]"/>
<set-variable variableName="numResults" doc:name="Variable" value="#[header:INBOUND:top]"/>
<set-variable variableName="offsetResults" doc:name="Variable" value="#[header:INBOUND:offset]"/>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[def query = message.getInvocationProperty("query")
def numResults = message.getInvocationProperty("numResults")
def offsetResults = message.getInvocationProperty("offsetResults")
def path = "Data.ashx/Bing/Search/v1/Web?Query=" + "$query" + "&WebFileType=%27PDF%27&\$top=" + "$numResults" + "&\$skip=" + "$offsetResults" + "&\$format=Json"
println "$path"
message.setProperty("pathVar","$path")
return null;]]></scripting:text>
</scripting:script>
</scripting:component>
<logger level="INFO" doc:name="Logger" message="#[header:OUTBOUND:pathVar]"/>
<https:outbound-endpoint exchange-pattern="request-response"
host="api.datamarket.azure.com" port="443" path="#[header:OUTBOUND:pathVar]"
user="*****" password="******"
doc:name="Bing" /> ...
The exception I get is
ERROR 2012-10-25 14:14:26,250 [[poll_directory].BingQuery.stage1.02] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : Failed to transform from "json" to "java.util.Map"
Code : MULE_ERROR-65110
--------------------------------------------------------------------------------
Exception stack is:
1. Unexpected character ('T' (code 84)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: java.io.InputStreamReader#73b62d67; line: 1, column: 2] (org.codehaus.jackson.JsonParseException)
org.codehaus.jackson.JsonParser:1291 (null)
2. Failed to transform from "json" to "java.util.Map" (org.mule.api.transformer.TransformerException)
org.mule.module.json.transformers.JsonToObject:136 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/transformer/TransformerException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
org.codehaus.jackson.JsonParseException: Unexpected character ('T' (code 84)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: java.io.InputStreamReader#73b62d67; line: 1, column: 2]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385)
at org.codehaus.jackson.impl.JsonParserMinimalBase._reportUnexpectedChar(JsonParserMinimalBase.java:306)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
There has to be a simple reason it isn't working and I'm sure it's just because I don't know the proper syntax yet in Mule. I also know there is probably a better way to do this using MEL versus Groovy, but I'm not familiar enough with MEL to do it yet.
SOLUTION
Turns out some of the characters of the User property and Password property were being encoded before being sent. Putting that key into a variable before passing to the User & Password property prevents that and led to success.
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:json="http://www.mulesoft.org/schema/mule/json"
xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns:https="http://www.mulesoft.org/schema/mule/https" xmlns:file="http://www.mulesoft.org/schema/mule/file"
xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" xmlns:core="http://www.mulesoft.org/schema/mule/core"
version="CE-3.3.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/https http://www.mulesoft.org/schema/mule/https/current/mule-https.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd ">
<configuration doc:name="Configuration">
<expression-language>
<import class="org.mule.util.StringUtils" />
</expression-language>
</configuration>
<flow name="BingQuery" doc:name="BingQuery">
<http:inbound-endpoint exchange-pattern="one-way" host="localhost" port="8082" doc:name="HTTP" />
<set-variable variableName="query" doc:name="Variable" value="#[header:INBOUND:query]"/>
<set-variable variableName="numResults" doc:name="Variable" value="#[header:INBOUND:top]"/>
<set-variable variableName="offsetResults" doc:name="Variable" value="#[header:INBOUND:offset]"/>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[def query = message.getInvocationProperty("query")
def numResults = message.getInvocationProperty("numResults")
def offsetResults = message.getInvocationProperty("offsetResults")
def path = "Data.ashx/Bing/Search/v1/Web?Query=" + "$query" + "&WebFileType=%27PDF%27&\$top=" + "$numResults" + "&\$skip=" + "$offsetResults" + "&\$format=Json"
println "$path"
message.setProperty("pathVar","$path")
message.setProperty("APIIdentifier","*****")
return null;]]></scripting:text>
</scripting:script>
</scripting:component>
<https:outbound-endpoint exchange-pattern="request-response"
host="api.datamarket.azure.com" port="443" path="#[header:OUTBOUND:pathVar]"
user="#[header:OUTBOUND:APIIdentifier]" password="#[header:OUTBOUND:APIIdentifier]"
doc:name="Bing" />
<json:json-to-object-transformer
returnClass="java.util.Map" doc:name="JSON to Object" />
<expression-transformer expression="#[message.payload.d.results]"
doc:name="Expression" />
<collection-splitter doc:name="Collection Splitter" />
<vm:outbound-endpoint exchange-pattern="one-way"
path="fileWriter" doc:name="VM" />
</flow>
<flow name="RestProcessor" doc:name="RestProcessor">
<vm:inbound-endpoint exchange-pattern="one-way"
path="fileWriter" doc:name="VM" />
<set-variable variableName="fileName" value="#[message.payload.ID].pdf"
doc:name="Variable" />
<http:rest-service-component
serviceUrl="#[joinChar=message.payload.Url.contains('?')?'&':'?' ; StringUtils.join(new String[]{message.payload.Url,(String)joinChar,'followRedirects=true'})]"
httpMethod="GET">
<http:error-filter>
<expression-filter expression="#[Integer.valueOf(message.inboundProperties['http.status']) >= 400]"></expression-filter>
</http:error-filter>
</http:rest-service-component>
<file:outbound-endpoint path="/home/administrator/Documents"
outputPattern="#[flowVars.fileName]" responseTimeout="10000"
mimeType="application/pdf" doc:name="File" />
</flow>
</mule>

Here is the beginning of the flow rewritten with MEL (which works for me, although I tested against a test endpoint not Bing).
Note that I also set the method to GET on the outbound HTTPS endpoint:
<flow name="BingQuery">
<http:inbound-endpoint exchange-pattern="one-way"
host="localhost" port="8082" />
<set-variable variableName="query"
value="#[message.inboundProperties.query]" />
<set-variable variableName="numResults"
value="#[message.inboundProperties.top]" />
<set-variable variableName="offsetResults"
value="#[message.inboundProperties.offset]" />
<expression-component>
flowVars['bingQuery'] =
'Data.ashx/Bing/Search/v1/Web?Query=' + query
+ '&WebFileType=%27PDF%27&$top=' + numResults
+ '&$skip=' + offsetResults
+ '&$format=Json';
payload = null;
</expression-component>
<logger level="ERROR" message="#[bingQuery]" />
<http:outbound-endpoint exchange-pattern="request-response"
method="GET"
host="api.datamarket.azure.com" port="443" path="#[bingQuery]"
user="*****" password="******" />
If the query, numResults and offsetResults variables are not used elsewhere in the flow, I suggest inlining the expressions used to create them right in the expression-component.

Related

insert into database with anypoint studio

I would like to save the flow coming from this json file: https://gist.githubusercontent.com/Rajeun/3dbbe8ca240d479dbabe/raw/aba5d0ec9345b886426ab767376b7c9cb60c251f/person.json in a database using anypoint studio.
For this i'm using a http connector where i mentioned the link of the json file. and then i used a json-to-object transformer and a database connector.
In my db i have a table with id, token, tel "int" and email is a varchar
-(I tested the connection and its working well).
when i run my code:
ERROR 2015-03-20 13:14:05,522 [main] org.mule.module.launcher.application.DefaultMuleApplication: null
java.lang.NullPointerException
at org.mule.module.db.internal.config.domain.query.QueryTemplateBeanDefinitionParser.parseParameterizedQuery(QueryTemplateBeanDefinitionParser.java:136) ~[mule-module-db-3.6.1.jar:3.6.1]
at org.mule.module.db.internal.config.domain.query.QueryTemplateBeanDefinitionParser.doParse(QueryTemplateBeanDefinitionParser.java:62) ~[mule-module-db-3.6.1.jar:3.6.1]
at org.mule.config.spring.parsers.AbstractMuleBeanDefinitionParser.parseInternal(AbstractMuleBeanDefinitionParser.java:295) ~[mule-module-spring-config-3.6.1.jar:3.6.1]
at org.springframework.beans.factory.xml.AbstractBeanDefinitionParser.parse(AbstractBeanDefinitionParser.java:59) ~[spring-beans-3.2.10.RELEASE.jar:3.2.10.RELEASE]
at org.springframework.beans.factory.xml.NamespaceHandlerSupport.parse(NamespaceHandlerSupport.java:73) ~[spring-beans-3.2.10.RELEASE.jar:3.2.10.RELEASE]
at org.mule.config.spring.MuleHierarchicalBeanDefinitionParserDelegate.parseCustomElement(MuleHierarchicalBeanDefinitionParserDelegate.java:98) ~[mule-module-spring-config-3.6.1.jar:3.6.1]
at org.mule.config.spring.MuleHierarchicalBeanDefinitionParserDelegate.parseCustomElement(MuleHierarchicalBeanDefinitionParserDelegate.java:140) ~[mule-module-spring-config-3.6.1.jar:3.6.1]
******************************************************************************
Config:
<mule xmlns:db="http://www.mulesoft.org/schema/mule/db" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.6.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/db http://www.mulesoft.org/schema/mule/db/current/mule-db.xsd">
<http:listener-config name="HTTP_Listener_Configuration" host="www.gist.githubusercontent.com" port="80" doc:name="HTTP Listener Configuration"/>
<db:mysql-config name="MySQL_Configuration" host="localhost" port="3306" user="root" database="mulesoft" doc:name="MySQL Configuration"/>
<db:template-query name="Template_Query" doc:name="Template Query">
<db:parameterized-query/>
</db:template-query>
<flow name="testFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="Rajeun/3dbbe8ca240d479dbabe/raw/aba5d0ec9345b886426ab767376b7c9cb60c251f/person.json" doc:name="HTTP"/>
<json:json-to-object-transformer returnClass="myclass" doc:name="JSON to Object"/>
<response>
<logger message="c bon" level="INFO" doc:name="Logger"/>
</response>
<response>
<db:insert config-ref="MySQL_Configuration" doc:name="Database">
<db:parameterized-query><![CDATA[INSERT INTO push(id, token, tel, email) VALUES (2,2,3,#[payload['userImage']])]]></db:parameterized-query>
</db:insert>
</response>
</flow>
</mule>
Remove the empty template query:
<db:template-query name="Template_Query" doc:name="Template Query">
<db:parameterized-query />
</db:template-query>
Also, does 'myclass' exist? if not try just using a map:
<json:json-to-object-transformer returnClass="java.util.HashMap" doc:name="JSON to Object"/>
Also It looks like you want to retrieve that JSON file from the specified URL. For that you want the http:request not http:listener. http:listener is used as a message source waiting for requests to your app.
You should use the http:request to request the file and trigger it using a polling message source:
<http:request-config name="HTTP_Request_Configuration" host="www.gist.githubusercontent.com" port="80" doc:name="HTTP Request Configuration"/>
<flow name="testFlow">
<poll>
<http:request config-ref="HTTP_Request_Configuration" path="Rajeun/3dbbe8ca240d479dbabe/raw/aba5d0ec9345b886426ab767376b7c9cb60c251f/person.json" method="GET" doc:name="HTTP"/>
</poll>
</flow>
See: http://www.mulesoft.org/documentation/display/current/HTTP+Request+Connector
And: http://www.mulesoft.org/documentation/display/current/Poll+Reference

How to enrich payload and insert objects into MongoDB using Mule

I have a message enricher in my Mule flow that looks like this:
<set-variable doc:name="Variable" value="#[payload['MeterUID']]" variableName="#['theKey']"/>
<enricher target="#[payload]" doc:name="Message Enricher">
<mongo:find-objects-using-query-map config-ref="Mongo_DB1" collection="meterentity" doc:name="Mongo DB">
<mongo:query-attributes>
<mongo:query-attribute key="_id">#[theKey]</mongo:query-attribute>
</mongo:query-attributes>
<mongo:fields>
<mongo:field>IpAddress</mongo:field>
<mongo:field>LastSetTime</mongo:field>
<mongo:field>LastReadGsmData</mongo:field>
</mongo:fields>
</mongo:find-objects-using-query-map>
and this is the payload that the message enricher recieves from my inbound-endpoint:
{TimeStamp=2013-12-16 08:48:33,270, MeterUID=4B414D000000011613CF, SignalStrengthIndication=15, CellID=4938, LocationAreaCode=280, MobileCountryCode=238, MobileNetworkCode=1}
at the Mongo-collection to json my payload looks like this:
[ { "_id" : "4B414D000000011613CC" , "IpAddress" : "10.12.189.12" , "LastSetTime" : { "$date" : "2014-03-11T14:40:36.987Z"} , "LastReadGsmData" : { "$date" : "2014-03-11T14:40:47.253Z"}}]
This seems wrong to me. First, where did my my first payload go ?
And second I get this this error when the flow tryes to inserted into my mongodb:
Exception stack is:
1. BasicBSONList can only work with numeric keys, not: [_id] (java.lang.IllegalArgumentException)
org.bson.types.BasicBSONList:161 (null)
2. Failed to invoke insertObject. Message payload is of type: String
How can I make this work?
I am still vary new to mule so i hope you guys can help me here.
My entire flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:mulexml="http://www.mulesoft.org/schema/mule/xml" xmlns:mongo="http://www.mulesoft.org/schema/mule/mongo" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns:data-mapper="http://www.mulesoft.org/schema/mule/ee/data-mapper" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.4.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/ee/data-mapper http://www.mulesoft.org/schema/mule/ee/data-mapper/current/mule-data-mapper.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/mongo http://www.mulesoft.org/schema/mule/mongo/2.0/mule-mongo.xsd
http://www.mulesoft.org/schema/mule/xml http://www.mulesoft.org/schema/mule/xml/current/mule-xml.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd">
<data-mapper:config name="csv_to_xml_7" transformationGraphPath="csv_to_xml_7.grf" doc:name="csv_to_xml_7"/>
<data-mapper:config name="csv_to_xml_8" transformationGraphPath="csv_to_xml_8.grf" doc:name="csv_to_xml_8"/>
<mongo:config name="Mongo_DB" username="$[admin]" doc:name="Mongo DB">
<mongo:connection-pooling-profile initialisationPolicy="INITIALISE_ONE" exhaustedAction="WHEN_EXHAUSTED_GROW"/>
</mongo:config>
<mongo:config name="Mongo_DB1" username="$[admin]" doc:name="Mongo DB">
<mongo:connection-pooling-profile initialisationPolicy="INITIALISE_ONE" exhaustedAction="WHEN_EXHAUSTED_GROW"/>
</mongo:config>
<data-mapper:config name="xml_to_json_2" transformationGraphPath="xml_to_json_2.grf" doc:name="xml_to_json_2"/>
<data-mapper:config name="xml_to_json_3" transformationGraphPath="xml_to_json_3.grf" doc:name="xml_to_json_3"/>
<mongo:config name="Mongo_DB2" username="$[admin]" doc:name="Mongo DB">
<mongo:connection-pooling-profile initialisationPolicy="INITIALISE_ONE" exhaustedAction="WHEN_EXHAUSTED_GROW"/>
</mongo:config>
<flow name="p2pcontrollerloganalyserFlow1" doc:name="p2pcontrollerloganalyserFlow1">
<file:inbound-endpoint path="C:\Users\Simon\Desktop\CSVFile" responseTimeout="10000" doc:name="File" moveToDirectory="C:\Users\Simon\Desktop\CSVPros" />
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<choice doc:name="Choice">
<when expression="#[message.outboundProperties.originalFilename=='gsmdata.log']">
<data-mapper:transform config-ref="csv_to_xml_7" doc:name="GSMV1 to xml"/>
<splitter expression="#[xpath('//Root')]" doc:name="Splitter"/>
<mulexml:dom-to-xml-transformer doc:name="DOM to XML"/>
<data-mapper:transform config-ref="xml_to_json_2" doc:name="XML To JSON"/>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<vm:outbound-endpoint exchange-pattern="one-way" path="json1" doc:name="VM"/>
</when>
<when expression="#[message.outboundProperties.originalFilename=='gsmdatav2.log']">
<data-mapper:transform config-ref="csv_to_xml_8" doc:name="GSMV2 to XML"/>
<splitter expression="#[xpath('//Root')]" doc:name="Splitter"/>
<mulexml:dom-to-xml-transformer doc:name="DOM to XML"/>
<data-mapper:transform config-ref="xml_to_json_3" doc:name="XML To JSON"/>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<vm:outbound-endpoint exchange-pattern="one-way" path="json2" doc:name="VM"/>
</when>
<otherwise>
<file:outbound-endpoint path="C:\Users\Simon\Desktop\CSVFile" responseTimeout="10000" doc:name="Failed GSM"/>
</otherwise>
</choice>
</flow>
<flow name="json1persistent" doc:name="json1persistent">
<vm:inbound-endpoint exchange-pattern="one-way" path="json1" doc:name="VM"/>
<json:json-to-object-transformer returnClass="java.lang.Object" doc:name="JSON to Object"/>
<set-variable doc:name="Variable" value="#[payload['MeterUID']]" variableName="#['theKey']"/>
<enricher target="#[payload]" doc:name="Message Enricher">
<mongo:find-objects-using-query-map config-ref="Mongo_DB1" collection="meterentity" doc:name="Mongo DB">
<mongo:query-attributes>
<mongo:query-attribute key="_id">#[theKey]</mongo:query-attribute>
</mongo:query-attributes>
<mongo:fields>
<mongo:field>IpAddress</mongo:field>
<mongo:field>LastSetTime</mongo:field>
<mongo:field>LastReadGsmData</mongo:field>
</mongo:fields>
</mongo:find-objects-using-query-map>
</enricher>
<mongo:mongo-collection-to-json doc:name="Mongo DB"/>
<mongo:insert-object config-ref="Mongo_DB2" collection="GSMdata" doc:name="Mongo DB"/>
<!-- <foreach collection="#[payload]" doc:name="For Each">
<mongo:insert-object config-ref="Mongo_DB" collection="GSMdata" doc:name="Mongo DB"/>
</foreach> -->
<!-- <enricher doc:name="Message Enricher">
<mongo:get-file-content config-ref="Mongo_DB" query-ref="#[payload['MeterUID']]" doc:name="Find Meter"/>
</enricher>-->
</flow>
</mule>
I am tying to read a CSV file, splitting it into smaller messages, enrich each of thise messages with the data from a MongoDB and finally insert each message into another MongoDB.
First, where did my my first payload go ?
enricher target="#[payload]" means that you are setting your current payload to whatever is returned by the enricher. You really want the target to be something else, such as a variable: target="#[variable:myVar]".
second I get this this error when the flow tryes to inserted into my
mongodb
You are not sharing any of your relevant configuration here, but I guess you are now sending your list of queried Mongo objects to some component that expects a single Map object. Please refer to your first problem.
If your aim is to add the fields from queried Mongo objects to your current payload, you should use a variable as the enrich target like in my above example. You can then later combine the payload and the enriched variable containing the Mongo object(s).
Further, if you are only getting a single object from Mongo, you can extract it from the Mongo list by using something like source="#[payload.toArray()[0]]" in the enricher component.
As your payload and the myVar variable are now both Java Maps, you can combine them with plain Java:
<expression-component doc:name="Expression">payload.putAll(myVar)</expression-component>
If you want to get rid of the _id field returned by Mongo, you can do it with
<expression-component doc:name="Expression">payload.remove('_id')</expression-component>
EDIT: Now that your payload is a Map, you can insert it into Mongo with
<mongo:insert-object-from-map config-ref="Mongo_DB2" collection="GSMdata">
<mongo:element-attributes ref="#[payload]"/>
</mongo:insert-object-from-map>

How to returned with custmise response to client using Mule ESB

I am new for the mule esb but i know other ESB patterns
my issue is i have done a sample which will insert a data into DB.its working fine but its not giving any response to client so client is getting nothing from server
my sample code is
<mule xmlns:json="http://www.mulesoft.org/schema/mule/json"
xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:jdbc-ee="http://www.mulesoft.org/schema/mule/ee/jdbc"
xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.4.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/ee/jdbc http://www.mulesoft.org/schema/mule/ee/jdbc/current/mule-jdbc-ee.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd">
<jdbc-ee:postgresql-data-source name="PostgreSQL_Data_Source" user="youtilitydba" password="45782dfff1" url="jdbc:postgresql://localhost:5432/sample" transactionIsolation="UNSPECIFIED" doc:name="PostgreSQL Data Source"/>
<jdbc-ee:connector name="Database" dataSource-ref="PostgreSQL_Data_Source" validateConnections="true" queryTimeout="-1" pollingFrequency="0" doc:name="Database"/>
<flow name="insertintoDBFlow1" doc:name="insertintoDBFlow1">
<http:inbound-endpoint exchange-pattern="request-response" host="localhost" port="8081" doc:name="HTTP" path="httpPost"/>
<logger message="log about input message: #[payload]" level="INFO" doc:name="Logger"/>
<json:json-to-object-transformer doc:name="JSON to Object" returnClass="java.util.Map"></json:json-to-object-transformer>
<jdbc-ee:outbound-endpoint exchange-pattern="one-way" queryKey="INSERT_TOKEN" queryTimeout="-1" connector-ref="Database" doc:name="Database">
<jdbc-ee:query key="INSERT_TOKEN" value="insert into users(FirstName,lastname) values(#[message.payload.name],#[message.payload.id]);"/>
</jdbc-ee:outbound-endpoint>
</flow>
</mule>
and i am calling to this using sample JSON client like this
curl -H "Content-Type: application/json" -d '{"name":"kk","id":"anil"}' http://localhost:8081/httpPost
when i am running this json its giving nothing response but i wish to give response like this {"ResponseJSON":{"Body":{"Datalist":{"Data":"Successfully Rows inserted"}},"Status":"200"}}
how would i format this above format i used http response builder but there is nothing to do like this..
you need to add an Echo component, set the payload to the value you are looking for and add a HTTP Response Builder with status="200" and contentType="application/json".
Add the following lines after </jdbc-ee:outbound-endpoint> and before </flow>
<response>
<http:response-builder status="200" contentType="application/json" doc:name="HTTP Response Builder"/>
</response>
<response>
<set-payload value="{"Body":{"Datalist":{"Data":"Successfully Rows inserted"}},"Status":"200"}}" doc:name="Set Payload"/>
</response>
<echo-component doc:name="Echo"/>
Cheers,
-Marco.

Mule ESB 3.3 "All" flow messes up payload for downstream components

EDIT: Simplified question.
Why on earth, the file written is garbage (binary serilized data) instead of the payload that was set in the Groovy component? This only happens if there is an "All" Flow component.
Flow:
XML:
<file:connector name="OutputFile" autoDelete="true" streaming="true" validateConnections="true" doc:name="File" writeToDirectory="#{systemProperties['user.home']}"/>
<flow name="AllProblemFlow1" doc:name="AllProblemFlow1">
<vm:inbound-endpoint exchange-pattern="one-way" path="in" doc:name="VM"/>
<all doc:name="All">
<processor-chain>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[return payload + 1]]></scripting:text>
</scripting:script>
</scripting:component>
</processor-chain>
<processor-chain>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[return payload + 2]]></scripting:text>
</scripting:script>
</scripting:component>
</processor-chain>
</all>
<logger message="All payload: #[payload]" level="INFO" doc:name="Logger"/>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[return "new payload"]]></scripting:text>
</scripting:script>
</scripting:component>
<file:outbound-endpoint responseTimeout="10000" connector-ref="OutputFile" doc:name="File" outputPattern="output.txt" path="#{systemProperties['user.home']}"/>
</flow>
This is very tricky: the all message processor changes the very nature of the in-flight Mule message from a MuleMessage to a MuleMessageCollection. Changing the payload on a MuleMessageCollection is basically ineffective.
You need to forcefully replace the current MuleMessageCollection with a brand new MuleMessage. Use the following code for your last Groovy component:
<scripting:text><![CDATA[
return new org.mule.DefaultMuleMessage("new payload", muleContext)
]]></scripting:text>

Execute action after all split messages have been processed

I have a Mule 3.3.0 flow which splits a file into records. I need to execute an action (stored procedure) AFTER ALL records have finished processing.
The problem is that sometimes the action gets executed before all records have been processed by Mule. I think this is due to the fact that Mule process stuff in parallel, which is great, so sometimes the final action gets called too early.
If I set the flow as synchronous things appear to work, but I'm not taking advantage of parallel execution.
I think I could also use a Foreach scope (haven't tried) but I guess that stuff will still not be parallelized.
Is there a way to "wait" until all records finish processing?
I'm attaching a very simple flow which exhibits this behaviour. If you run it you will see that the loggers don't print stuff in order. Actually, the "DONE" message gets logged before the rest.
The flow processes a simple csv file auntil it matches a field with value "end". There is a choice component which loggs "DONE" when such field is found. The rest of the fields simply get logged.
Any help will be greatly appreciated.
Flow:
Flow xml
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting"
xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:file="http://www.mulesoft.org/schema/mule/file"
xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.3.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd ">
<file:connector name="inputFileConnector" autoDelete="true"
streaming="false" validateConnections="true" doc:name="File" fileAge="60000"
readFromDirectory="#{systemProperties['user.home']}" />
<flow name="flow1" doc:name="flow1" processingStrategy="synchronous">
<file:inbound-endpoint path="#{systemProperties['user.home']}"
responseTimeout="10000" doc:name="Input File" fileAge="100"
connector-ref="inputFileConnector">
<file:filename-regex-filter pattern="input.csv"
caseSensitive="false" />
</file:inbound-endpoint>
<byte-array-to-string-transformer
doc:name="Byte Array to String" />
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[return payload.split('\n');]]></scripting:text>
</scripting:script>
</scripting:component>
<collection-splitter doc:name="Collection Splitter" />
<choice doc:name="Choice">
<when expression="#[groovy:payload != 'end']">
<processor-chain>
<logger message="." level="INFO" doc:name="Process"/>
<vm:outbound-endpoint path="toFlow2" doc:name="VM"/>
</processor-chain>
</when>
<otherwise>
<processor-chain>
<logger message="|||| DONE" level="INFO" doc:name="DONE"/>
</processor-chain>
</otherwise>
</choice>
</flow>
<flow name="flow2" doc:name="flow2" >
<vm:inbound-endpoint path="toFlow2" doc:name="VM"/>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[return payload.split(',');]]></scripting:text>
</scripting:script>
</scripting:component>
<collection-splitter doc:name="Collection Splitter" />
<logger message="|||||| #[payload]" level="INFO" doc:name="Logger"/>
<vm:outbound-endpoint path="toFlow3" doc:name="VM"/>
</flow>
One option is to use a collection-aggregator to act as an accumulator, blocking the final flow action until all the messages have been processed. The trick is that the collection-splitters will set up a correlation group size that is only good for either the number of lines in the file or the number of columns in the file. But we want to accumulate until all columns of all lines have been processed. The solution consists in computing first this value (ie total number of expected messages) and overriding whatever correlation group size had been calculated the collection-splitters with the total value.
Here is how I've done this (you'll note that I replaced all Groovy snippets with more Mule-3-esque MEL expressions):
<file:connector name="inputFileConnector" autoDelete="true"
streaming="false" validateConnections="true" fileAge="60000"
readFromDirectory="#{systemProperties['user.home']}" />
<flow name="flow1" processingStrategy="synchronous">
<file:inbound-endpoint path="#{systemProperties['user.home']}"
responseTimeout="10000" fileAge="100"
connector-ref="inputFileConnector">
<file:filename-regex-filter pattern="input.csv"
caseSensitive="false" />
</file:inbound-endpoint>
<byte-array-to-string-transformer />
<set-session-variable variableName="expectedMessageCount"
value="#[org.mule.util.StringUtils.countMatches(message.payload, '\n') + org.mule.util.StringUtils.countMatches(message.payload, ',') - 1]" />
<expression-transformer expression="#[message.payload.split('\n')]" />
<collection-splitter enableCorrelation="IF_NOT_SET" />
<set-property propertyName="MULE_CORRELATION_GROUP_SIZE"
value="#[sessionVars.expectedMessageCount]" />
<choice>
<when expression="#[message.payload != 'end']">
<processor-chain>
<logger message="." level="INFO" />
<vm:outbound-endpoint path="toFlow2" />
</processor-chain>
</when>
<otherwise>
<processor-chain>
<logger message="|||| END" level="INFO" />
</processor-chain>
</otherwise>
</choice>
</flow>
<flow name="flow2">
<vm:inbound-endpoint path="toFlow2"/>
<expression-transformer expression="#[message.payload.split(',')]" />
<collection-splitter />
<set-property propertyName="MULE_CORRELATION_GROUP_SIZE"
value="#[sessionVars.expectedMessageCount]" />
<logger message="|||||| #[message.payload]" level="INFO"/>
<vm:outbound-endpoint path="toFinalizer" />
<vm:outbound-endpoint path="toFlow3" />
</flow>
<flow name="finalizer">
<vm:inbound-endpoint path="toFinalizer" />
<collection-aggregator />
<logger message="|||| DONE" level="INFO" />
</flow>
NB. Alternatively, if using a collection-aggregator is an issue because it uses too much memory, you could use an expression component to decrement sessionVars.expectedMessageCount and filter to let a message hit the final message processor when the counter is back to 0.