Mule-Setting Status code in HTTP response - exception

I need to set Status code and Reason Phrase in mule HTTP using Mule Choice exception strategy under that is Catch exception strategy. Following the Mulesoft documentation https://docs.mulesoft.com/mule-user-guide/v/3.7/http-listener-connector#http-response-status-code-and-reason-phrase
I tried doing something like this :
<choice-exception-strategy name="MyExptn">
<rollback-exception-strategy when="#[exception.causedBy(org.mule.component.ComponentException)]" doc:name="ComponentException Strategy">
<set-variable variableName="errorTableNotPresent" value="400" doc:name="Set status code"/>
<set-variable variableName="errorReasonPhrase" value="Table name does not exist" doc:name="Set reason phrase"/>
<message-properties-transformer doc:name="Message Properties">
<add-message-property key="timestamp" value="#[server.dateTime.format('yyyy-MM-dd hh:mm:ss.SS')]"/>
<add-message-property key="messageID" value="#[message.id]"/>
<add-message-property key="status" value="Error"/>
<add-message-property key="executionPoint" value="Error Handling"/>
<add-message-property key="ip" value="#[server.ip]"/>
<add-message-property key="serverName" value="#[server.host]"/>
<add-message-property key="domainname" value="xyz.com"/>
<add-message-property key="errorCode" value="123"/>
<add-message-property key="errorType" value="Bad request"/>
<add-message-property key="errorText" value="ComponentException"/>
<add-message-property key="integrationName" value="${integrationName}"/>
<add-message-property key="resourceType" value="HTTP Request"/>
<add-message-property key="resourceName" value="Payload"/>
</message-properties-transformer>
<logger level="INFO" message="#[LoggingAppendString] - Failure" doc:name="Logger"/>
</rollback-exception-strategy>
</choice-exception-strategy>
Which is working fine. But when I'm replacing it with Catch Exception Strategy as shown below :
<choice-exception-strategy name="MyExptn">
<catch-exception-strategy when="#[exception.causedBy(org.mule.component.ComponentException)]" doc:name="ComponentException Strategy">
<set-variable variableName="errorTableNotPresent" value="400" doc:name="Set status code"/>
<set-variable variableName="errorReasonPhrase" value="Table name does not exist" doc:name="Set reason phrase"/>
<message-properties-transformer doc:name="Message Properties">
<add-message-property key="timestamp" value="#[server.dateTime.format('yyyy-MM-dd hh:mm:ss.SS')]"/>
<add-message-property key="messageID" value="#[message.id]"/>
<add-message-property key="status" value="Error"/>
<add-message-property key="executionPoint" value="Error Handling"/>
<add-message-property key="ip" value="#[server.ip]"/>
<add-message-property key="serverName" value="#[server.host]"/>
<add-message-property key="domainname" value="xyz.com"/>
<add-message-property key="errorCode" value="123"/>
<add-message-property key="errorType" value="Bad request"/>
<add-message-property key="errorText" value="ComponentException"/>
<add-message-property key="integrationName" value="${integrationName}"/>
<add-message-property key="resourceType" value="HTTP Request"/>
<add-message-property key="resourceName" value="Payload"/>
</message-properties-transformer>
<logger level="INFO" message="#[LoggingAppendString] - Failure" doc:name="Logger"/>
</catch-exception-strategy>
</choice-exception-strategy>
Will only log the message but HTTP status always showing 200 OK.
My basic requirement is to send status code based on the error. Say for example if request is not correct then Bad request with code 400.

Define two outbound properties: http.status and http.reason. These will be sent to the client as the actual status. Without setting these values Mule will default to 200 - OK.
I do not see these fields defined in your example.
Example from the documentation link you provided:
<set-property propertyName="http.status" value="500" doc:name="Property"/>
<set-property propertyName="http.reason" value="Request successfully executed!" doc:name="Property"/>
Make sure you scope these as outbound properties.
Keep in mind that on certain versions of the run time there is a bug where http.reason is not mapped into the response correctly resulting in something like Status 400 - OK rather than Status: 400 - Bad Request.
https://www.mulesoft.org/jira/browse/MULE-9045

Related

Rollback strategy with maxRedeliveryAttempts and VM inbound configured does not redeliver messages

My flow looks like the below, I expect the message to be redelivered 2 times and then the
Redelivery is exhaused SAD
logger to be printed. But after the component throws the exception the re delivery mechanism does not kick in
<vm:connector name="VM" validateConnections="true" doc:name="VM" />
<flow name="TriggerFlow" >
<http:listener config-ref="Orders_HTTP_Listener_Configuration" path="/rollback" allowedMethods="GET" doc:name="1080/rollback" />
<vm:outbound-endpoint exchange-pattern="request-response" path="txFlow" doc:name="VM" connector-ref="VM" responseTimeout="60000">
</vm:outbound-endpoint>
</flow>
<flow name="TxFlow" >
<vm:inbound-endpoint exchange-pattern="request-response" path="txFlow" doc:name="case1" connector-ref="VM" responseTimeout="60000">
<xa-transaction action="ALWAYS_BEGIN"/>
</vm:inbound-endpoint>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy"><![CDATA[throw new RuntimeException();]]></scripting:script>
</scripting:component>
<rollback-exception-strategy maxRedeliveryAttempts="3" doc:name="Rollback Exception Strategy">
<logger message="Will rollback #[payload]" level="INFO" doc:name="Logger"/>
<on-redelivery-attempts-exceeded>
<logger message="Redelivery is exhaused SAD " level="INFO" doc:name="Logger"/>
</on-redelivery-attempts-exceeded>
</rollback-exception-strategy>
</flow>
I finally got the answer after speaking with MuleSoft. The updated flow should look like the below, both the VM's should be one-way and the processing strategy of the TxFlow should be synchronous. If the VM is request-reponse then it behaves more like a flow-ref with no queue involved and hence no redelivery ...

How to enrich payload and insert objects into MongoDB using Mule

I have a message enricher in my Mule flow that looks like this:
<set-variable doc:name="Variable" value="#[payload['MeterUID']]" variableName="#['theKey']"/>
<enricher target="#[payload]" doc:name="Message Enricher">
<mongo:find-objects-using-query-map config-ref="Mongo_DB1" collection="meterentity" doc:name="Mongo DB">
<mongo:query-attributes>
<mongo:query-attribute key="_id">#[theKey]</mongo:query-attribute>
</mongo:query-attributes>
<mongo:fields>
<mongo:field>IpAddress</mongo:field>
<mongo:field>LastSetTime</mongo:field>
<mongo:field>LastReadGsmData</mongo:field>
</mongo:fields>
</mongo:find-objects-using-query-map>
and this is the payload that the message enricher recieves from my inbound-endpoint:
{TimeStamp=2013-12-16 08:48:33,270, MeterUID=4B414D000000011613CF, SignalStrengthIndication=15, CellID=4938, LocationAreaCode=280, MobileCountryCode=238, MobileNetworkCode=1}
at the Mongo-collection to json my payload looks like this:
[ { "_id" : "4B414D000000011613CC" , "IpAddress" : "10.12.189.12" , "LastSetTime" : { "$date" : "2014-03-11T14:40:36.987Z"} , "LastReadGsmData" : { "$date" : "2014-03-11T14:40:47.253Z"}}]
This seems wrong to me. First, where did my my first payload go ?
And second I get this this error when the flow tryes to inserted into my mongodb:
Exception stack is:
1. BasicBSONList can only work with numeric keys, not: [_id] (java.lang.IllegalArgumentException)
org.bson.types.BasicBSONList:161 (null)
2. Failed to invoke insertObject. Message payload is of type: String
How can I make this work?
I am still vary new to mule so i hope you guys can help me here.
My entire flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:mulexml="http://www.mulesoft.org/schema/mule/xml" xmlns:mongo="http://www.mulesoft.org/schema/mule/mongo" xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking" xmlns:data-mapper="http://www.mulesoft.org/schema/mule/ee/data-mapper" xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="EE-3.4.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/ee/data-mapper http://www.mulesoft.org/schema/mule/ee/data-mapper/current/mule-data-mapper.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/mongo http://www.mulesoft.org/schema/mule/mongo/2.0/mule-mongo.xsd
http://www.mulesoft.org/schema/mule/xml http://www.mulesoft.org/schema/mule/xml/current/mule-xml.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd">
<data-mapper:config name="csv_to_xml_7" transformationGraphPath="csv_to_xml_7.grf" doc:name="csv_to_xml_7"/>
<data-mapper:config name="csv_to_xml_8" transformationGraphPath="csv_to_xml_8.grf" doc:name="csv_to_xml_8"/>
<mongo:config name="Mongo_DB" username="$[admin]" doc:name="Mongo DB">
<mongo:connection-pooling-profile initialisationPolicy="INITIALISE_ONE" exhaustedAction="WHEN_EXHAUSTED_GROW"/>
</mongo:config>
<mongo:config name="Mongo_DB1" username="$[admin]" doc:name="Mongo DB">
<mongo:connection-pooling-profile initialisationPolicy="INITIALISE_ONE" exhaustedAction="WHEN_EXHAUSTED_GROW"/>
</mongo:config>
<data-mapper:config name="xml_to_json_2" transformationGraphPath="xml_to_json_2.grf" doc:name="xml_to_json_2"/>
<data-mapper:config name="xml_to_json_3" transformationGraphPath="xml_to_json_3.grf" doc:name="xml_to_json_3"/>
<mongo:config name="Mongo_DB2" username="$[admin]" doc:name="Mongo DB">
<mongo:connection-pooling-profile initialisationPolicy="INITIALISE_ONE" exhaustedAction="WHEN_EXHAUSTED_GROW"/>
</mongo:config>
<flow name="p2pcontrollerloganalyserFlow1" doc:name="p2pcontrollerloganalyserFlow1">
<file:inbound-endpoint path="C:\Users\Simon\Desktop\CSVFile" responseTimeout="10000" doc:name="File" moveToDirectory="C:\Users\Simon\Desktop\CSVPros" />
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<choice doc:name="Choice">
<when expression="#[message.outboundProperties.originalFilename=='gsmdata.log']">
<data-mapper:transform config-ref="csv_to_xml_7" doc:name="GSMV1 to xml"/>
<splitter expression="#[xpath('//Root')]" doc:name="Splitter"/>
<mulexml:dom-to-xml-transformer doc:name="DOM to XML"/>
<data-mapper:transform config-ref="xml_to_json_2" doc:name="XML To JSON"/>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<vm:outbound-endpoint exchange-pattern="one-way" path="json1" doc:name="VM"/>
</when>
<when expression="#[message.outboundProperties.originalFilename=='gsmdatav2.log']">
<data-mapper:transform config-ref="csv_to_xml_8" doc:name="GSMV2 to XML"/>
<splitter expression="#[xpath('//Root')]" doc:name="Splitter"/>
<mulexml:dom-to-xml-transformer doc:name="DOM to XML"/>
<data-mapper:transform config-ref="xml_to_json_3" doc:name="XML To JSON"/>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<vm:outbound-endpoint exchange-pattern="one-way" path="json2" doc:name="VM"/>
</when>
<otherwise>
<file:outbound-endpoint path="C:\Users\Simon\Desktop\CSVFile" responseTimeout="10000" doc:name="Failed GSM"/>
</otherwise>
</choice>
</flow>
<flow name="json1persistent" doc:name="json1persistent">
<vm:inbound-endpoint exchange-pattern="one-way" path="json1" doc:name="VM"/>
<json:json-to-object-transformer returnClass="java.lang.Object" doc:name="JSON to Object"/>
<set-variable doc:name="Variable" value="#[payload['MeterUID']]" variableName="#['theKey']"/>
<enricher target="#[payload]" doc:name="Message Enricher">
<mongo:find-objects-using-query-map config-ref="Mongo_DB1" collection="meterentity" doc:name="Mongo DB">
<mongo:query-attributes>
<mongo:query-attribute key="_id">#[theKey]</mongo:query-attribute>
</mongo:query-attributes>
<mongo:fields>
<mongo:field>IpAddress</mongo:field>
<mongo:field>LastSetTime</mongo:field>
<mongo:field>LastReadGsmData</mongo:field>
</mongo:fields>
</mongo:find-objects-using-query-map>
</enricher>
<mongo:mongo-collection-to-json doc:name="Mongo DB"/>
<mongo:insert-object config-ref="Mongo_DB2" collection="GSMdata" doc:name="Mongo DB"/>
<!-- <foreach collection="#[payload]" doc:name="For Each">
<mongo:insert-object config-ref="Mongo_DB" collection="GSMdata" doc:name="Mongo DB"/>
</foreach> -->
<!-- <enricher doc:name="Message Enricher">
<mongo:get-file-content config-ref="Mongo_DB" query-ref="#[payload['MeterUID']]" doc:name="Find Meter"/>
</enricher>-->
</flow>
</mule>
I am tying to read a CSV file, splitting it into smaller messages, enrich each of thise messages with the data from a MongoDB and finally insert each message into another MongoDB.
First, where did my my first payload go ?
enricher target="#[payload]" means that you are setting your current payload to whatever is returned by the enricher. You really want the target to be something else, such as a variable: target="#[variable:myVar]".
second I get this this error when the flow tryes to inserted into my
mongodb
You are not sharing any of your relevant configuration here, but I guess you are now sending your list of queried Mongo objects to some component that expects a single Map object. Please refer to your first problem.
If your aim is to add the fields from queried Mongo objects to your current payload, you should use a variable as the enrich target like in my above example. You can then later combine the payload and the enriched variable containing the Mongo object(s).
Further, if you are only getting a single object from Mongo, you can extract it from the Mongo list by using something like source="#[payload.toArray()[0]]" in the enricher component.
As your payload and the myVar variable are now both Java Maps, you can combine them with plain Java:
<expression-component doc:name="Expression">payload.putAll(myVar)</expression-component>
If you want to get rid of the _id field returned by Mongo, you can do it with
<expression-component doc:name="Expression">payload.remove('_id')</expression-component>
EDIT: Now that your payload is a Map, you can insert it into Mongo with
<mongo:insert-object-from-map config-ref="Mongo_DB2" collection="GSMdata">
<mongo:element-attributes ref="#[payload]"/>
</mongo:insert-object-from-map>

Trouble using two different databases in Mule JDBC flow

I have two JDBC flows in Mule 3.2.0, one using MySQL database and other using SQLServer database.
<mule ...
<spring:bean id="MySQL-jdbcDataSource"
class="org.enhydra.jdbc.standard.StandardDataSource" destroy-method="shutdown">
<spring:property name="driverName"
value="com.mysql.jdbc.Driver" />
<spring:property name="url"
value="jdbc:mysql://host:port/schema" />
</spring:bean>
<jdbc:connector name="MySQL-jdbcConnector"
dataSource-ref="MySQL-jdbcDataSource" pollingFrequency="${MySQL.db.poll}"
transactionPerMessage="false">
<jdbc:query key="read" value="${MySQL.db.jdbc_query}" />
</jdbc:connector>
<flow name="MySQL-flow">
<jdbc:inbound-endpoint queryKey="read"
connector-ref="MySQL-jdbcConnector">
<jdbc:transaction action="ALWAYS_BEGIN"/>
<property key="receiveMessageInTransaction" value="true"/>
</jdbc:inbound-endpoint>
<vm:outbound-endpoint path="path" connector-ref="first-level">
<message-properties-transformer scope="outbound">
<add-message-property key="identifier" value="MySQL"/>
</message-properties-transformer>
<vm:transaction action="NONE"/>
</vm:outbound-endpoint>
</flow>
</mule>
And
<mule ...
<spring:bean id="SQLServer-jdbcDataSource"
class="org.enhydra.jdbc.standard.StandardDataSource" destroy-method="shutdown">
<spring:property name="driverName"
value="com.microsoft.sqlserver.jdbc.SQLServerDriver" />
<spring:property name="url"
value="jdbc:sqlserver://host:port;databaseName=schema" />
</spring:bean>
<jdbc:connector name="SQLServer-jdbcConnector"
dataSource-ref="SQLServer-jdbcDataSource" pollingFrequency="${SQLServer.db.poll}"
transactionPerMessage="false">
<jdbc:query key="read" value="${SQLServer.db.jdbc_query}" />
</jdbc:connector>
<flow name="SQLServer-flow">
<jdbc:inbound-endpoint queryKey="read"
connector-ref="SQLServer-jdbcConnector">
<jdbc:transaction action="ALWAYS_BEGIN"/>
<property key="receiveMessageInTransaction" value="true"/>
</jdbc:inbound-endpoint>
<vm:outbound-endpoint path="${sv.vm.queue.name}" connector-ref="first-level-xform">
<message-properties-transformer scope="outbound">
<add-message-property key="${sv.vm.msg.identifier}" value="SQLServer"/>
</message-properties-transformer>
<vm:transaction action="NONE"/>
</vm:outbound-endpoint>
</flow>
</mule>
When I deploy any one of these floes in mule-deploy.properties, it runs ok. But when I deploy both of these flows at the same time, none of them works. I do not get any error or exception, but it seems like none of these flows run.
Any ideas what might be wrong? Probably something related to JDBC transactions?
if you have 2 different flows in same application .. then remove :-
<spring:bean id="SQLServer-jdbcDataSource"
class="org.enhydra.jdbc.standard.StandardDataSource" destroy-method="shutdown">
<spring:property name="driverName"
value="com.microsoft.sqlserver.jdbc.SQLServerDriver" />
<spring:property name="url"
value="jdbc:sqlserver://host:port;databaseName=schema" />
</spring:bean>
<jdbc:connector name="SQLServer-jdbcConnector"
dataSource-ref="SQLServer-jdbcDataSource" pollingFrequency="${SQLServer.db.poll}"
transactionPerMessage="false">
<jdbc:query key="read" value="${SQLServer.db.jdbc_query}" />
</jdbc:connector>
from any of the flow ... Since it is declared as global ... it can be used from any of the flow in the application .. what I mean is .. just remove the above line of code from one of the flow ... since you have declared in both the flow I guess it's a duplicate and not required to place in each and every flow ... and yes just mention the reference connector-ref="SQLServer-jdbcConnector" in both the mule flow ... ex:-
<jdbc:inbound-endpoint queryKey="read"
connector-ref="SQLServer-jdbcConnector">

Mule JMS Max Redelivery Exceeded Sonic and WMQ

I have a mule flow which reads messages from a sonic topic and publish to a Websphere MQ Topic.
The flow is synchronous and transacted.
I have set maxRedelivery="10" in the inboud endpoint. but it keeps on trying to redeliver.
Here is the exception I get:
ERROR 2014-03-03 15:13:08,763 [JMS Session Delivery Thread - $TMPAPPID$544239$$SESSION$0:-3441129419112818566] org.mule.exception.DefaultMessagingExceptionStrategy:
********************************************************************************
Message : "Message with id "ID:1419c04f:da190005:1448992ACB1" has been redelivered 1,511 times on endpoint "jms://topic:testtopic", which exceeds the maxRedelivery setting of 10 on the connector "sonicMQConnectorSub". Message payload is of type: TextMessage
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. "Message with id "ID:1419c04f:da190005:1448992ACB1" has been redelivered 1,511 times on endpoint "jms://topic:testtopic", which exceeds the maxRedelivery setting of 10 on the connector "sonicMQConnectorSub". Message payload is of type: TextMessage (org.mule.transport.jms.redelivery.MessageRedeliveredException)
org.mule.transport.jms.redelivery.JmsXRedeliveryHandler:91 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/transport/jms/redelivery/MessageRedeliveredException.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
org.mule.transport.jms.redelivery.MessageRedeliveredException: "Message with id "ID:1419c04f:da190005:1448992ACB1" has been redelivered 1,511 times on endpoint "jms://topic:testtopic", which exceeds the maxRedelivery setting of 10 on the connector "sonicMQConnectorSub". Message payload is of type: TextMessage
at org.mule.transport.jms.redelivery.JmsXRedeliveryHandler.handleRedelivery(JmsXRedeliveryHandler.java:91)
at org.mule.transport.jms.MultiConsumerJmsMessageReceiver$JmsWorker.preProcessMessage(MultiConsumerJmsMessageReceiver.java:426)
at org.mule.transport.AbstractReceiverWorker$1$1.process(AbstractReceiverWorker.java:120)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
Here is the flow I use.
<jms:connector name="sonicMQConnectorSub" specification="1.1" validateConnections="true" maxRedelivery="10" connectionFactory-ref="soniqMQConnectionFactorySub" doc:name="JMS" clientId="${topic.sub.clientId}" durable="true"/>
<spring:beans>
<spring:bean id="soniqMQConnectionFactoryBeanSub" name="soniqMQConnectionFactorySub" class="progress.message.jclient.ConnectionFactory">
<spring:property name="connectionURLs" value="${topic.sub.providerUrls}" />
<spring:property name="defaultUser" value="${topic.sub.username}" />
<spring:property name="defaultPassword" value="${topic.sub.password}" />
</spring:bean>
</spring:beans>
<jms:connector name="wsMQConnector" specification="1.1" username="${topic.pub.username}" password="${topic.pub.password}" validateConnections="true" connectionFactory-ref="wsMQConnectionFactory" doc:name="JMS" clientId="${topic.pub.clientId}"/>
<spring:beans>
<spring:bean id="wsMQConnectionFactory" class="com.ibm.mq.jms.MQTopicConnectionFactory">
<spring:property name="transportType" value="1"/>
<spring:property name="hostName" value="${topic.pub.host}"/>
<spring:property name="port" value="${topic.pub.port}"/>
<spring:property name="channel" value="${topic.pub.channel}"/>
<spring:property name="queueManager" value="${topic.pub.queueManager}"/>
</spring:bean>
</spring:beans>
<flow name="Flow1" doc:name="Flow1" processingStrategy="synchronous">
<jms:inbound-endpoint doc:name="JMS" connector-ref="sonicMQConnectorSub" topic="${topic.sub.name}" >
<jms:transaction action="ALWAYS_BEGIN" />
</jms:inbound-endpoint>
<message-properties-transformer doc:name="Message Properties">
<add-message-property key="SENTTIMESTAMP" value="#[server.dateTime]"/>
</message-properties-transformer>
<logger message="Payload: #[message.payload]" level="INFO" doc:name="Logger"/>
<jms:outbound-endpoint doc:name="JMS" connector-ref="wsMQConnector" topic="${topic.pub.name}">
<jms:transaction action="ALWAYS_JOIN" />
</jms:outbound-endpoint>
</flow>
This is expected behavior, the above exception is raised when the maximum redelivery count defined in maxRedelivery has been exceeded by your JMS provider. See the answer by David Dossot here: jms Mule max redelivery exceeded

Execute action after all split messages have been processed

I have a Mule 3.3.0 flow which splits a file into records. I need to execute an action (stored procedure) AFTER ALL records have finished processing.
The problem is that sometimes the action gets executed before all records have been processed by Mule. I think this is due to the fact that Mule process stuff in parallel, which is great, so sometimes the final action gets called too early.
If I set the flow as synchronous things appear to work, but I'm not taking advantage of parallel execution.
I think I could also use a Foreach scope (haven't tried) but I guess that stuff will still not be parallelized.
Is there a way to "wait" until all records finish processing?
I'm attaching a very simple flow which exhibits this behaviour. If you run it you will see that the loggers don't print stuff in order. Actually, the "DONE" message gets logged before the rest.
The flow processes a simple csv file auntil it matches a field with value "end". There is a choice component which loggs "DONE" when such field is found. The rest of the fields simply get logged.
Any help will be greatly appreciated.
Flow:
Flow xml
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting"
xmlns:vm="http://www.mulesoft.org/schema/mule/vm" xmlns:file="http://www.mulesoft.org/schema/mule/file"
xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" version="CE-3.3.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.mulesoft.org/schema/mule/vm http://www.mulesoft.org/schema/mule/vm/current/mule-vm.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd
http://www.mulesoft.org/schema/mule/scripting http://www.mulesoft.org/schema/mule/scripting/current/mule-scripting.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd ">
<file:connector name="inputFileConnector" autoDelete="true"
streaming="false" validateConnections="true" doc:name="File" fileAge="60000"
readFromDirectory="#{systemProperties['user.home']}" />
<flow name="flow1" doc:name="flow1" processingStrategy="synchronous">
<file:inbound-endpoint path="#{systemProperties['user.home']}"
responseTimeout="10000" doc:name="Input File" fileAge="100"
connector-ref="inputFileConnector">
<file:filename-regex-filter pattern="input.csv"
caseSensitive="false" />
</file:inbound-endpoint>
<byte-array-to-string-transformer
doc:name="Byte Array to String" />
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[return payload.split('\n');]]></scripting:text>
</scripting:script>
</scripting:component>
<collection-splitter doc:name="Collection Splitter" />
<choice doc:name="Choice">
<when expression="#[groovy:payload != 'end']">
<processor-chain>
<logger message="." level="INFO" doc:name="Process"/>
<vm:outbound-endpoint path="toFlow2" doc:name="VM"/>
</processor-chain>
</when>
<otherwise>
<processor-chain>
<logger message="|||| DONE" level="INFO" doc:name="DONE"/>
</processor-chain>
</otherwise>
</choice>
</flow>
<flow name="flow2" doc:name="flow2" >
<vm:inbound-endpoint path="toFlow2" doc:name="VM"/>
<scripting:component doc:name="Groovy">
<scripting:script engine="Groovy">
<scripting:text><![CDATA[return payload.split(',');]]></scripting:text>
</scripting:script>
</scripting:component>
<collection-splitter doc:name="Collection Splitter" />
<logger message="|||||| #[payload]" level="INFO" doc:name="Logger"/>
<vm:outbound-endpoint path="toFlow3" doc:name="VM"/>
</flow>
One option is to use a collection-aggregator to act as an accumulator, blocking the final flow action until all the messages have been processed. The trick is that the collection-splitters will set up a correlation group size that is only good for either the number of lines in the file or the number of columns in the file. But we want to accumulate until all columns of all lines have been processed. The solution consists in computing first this value (ie total number of expected messages) and overriding whatever correlation group size had been calculated the collection-splitters with the total value.
Here is how I've done this (you'll note that I replaced all Groovy snippets with more Mule-3-esque MEL expressions):
<file:connector name="inputFileConnector" autoDelete="true"
streaming="false" validateConnections="true" fileAge="60000"
readFromDirectory="#{systemProperties['user.home']}" />
<flow name="flow1" processingStrategy="synchronous">
<file:inbound-endpoint path="#{systemProperties['user.home']}"
responseTimeout="10000" fileAge="100"
connector-ref="inputFileConnector">
<file:filename-regex-filter pattern="input.csv"
caseSensitive="false" />
</file:inbound-endpoint>
<byte-array-to-string-transformer />
<set-session-variable variableName="expectedMessageCount"
value="#[org.mule.util.StringUtils.countMatches(message.payload, '\n') + org.mule.util.StringUtils.countMatches(message.payload, ',') - 1]" />
<expression-transformer expression="#[message.payload.split('\n')]" />
<collection-splitter enableCorrelation="IF_NOT_SET" />
<set-property propertyName="MULE_CORRELATION_GROUP_SIZE"
value="#[sessionVars.expectedMessageCount]" />
<choice>
<when expression="#[message.payload != 'end']">
<processor-chain>
<logger message="." level="INFO" />
<vm:outbound-endpoint path="toFlow2" />
</processor-chain>
</when>
<otherwise>
<processor-chain>
<logger message="|||| END" level="INFO" />
</processor-chain>
</otherwise>
</choice>
</flow>
<flow name="flow2">
<vm:inbound-endpoint path="toFlow2"/>
<expression-transformer expression="#[message.payload.split(',')]" />
<collection-splitter />
<set-property propertyName="MULE_CORRELATION_GROUP_SIZE"
value="#[sessionVars.expectedMessageCount]" />
<logger message="|||||| #[message.payload]" level="INFO"/>
<vm:outbound-endpoint path="toFinalizer" />
<vm:outbound-endpoint path="toFlow3" />
</flow>
<flow name="finalizer">
<vm:inbound-endpoint path="toFinalizer" />
<collection-aggregator />
<logger message="|||| DONE" level="INFO" />
</flow>
NB. Alternatively, if using a collection-aggregator is an issue because it uses too much memory, you could use an expression component to decrement sessionVars.expectedMessageCount and filter to let a message hit the final message processor when the counter is back to 0.