Reading a .txt file in WSO2 Integration Studio - csv

Is it possible to read a .txt file which has "|" as seperator using smooks? The file is similar to a normal csv file, only the extension is .txt. For csv we make use of "csv-reader", what can I use for reading a .txt file? Or is there any way to convert the .txt file into .csv in WSO2 Integration Studio?
This is the content of my file:
SYS_CD|PRO_CD|LOC_CD|AVA_QTY|RESQTY|PRO_QTY
POS|15|90|00|0|067656400
This is how I am calling the Smooks in my Proxy.
<smooks config-key="Smooks-config-file">
<input type="text"/>
<output type="xml"/>
</smooks>
This is my smooks-config-file
<?xml version="1.0" encoding="UTF-8"?>
<localEntry key="Smooks-config-file" xmlns="http://ws.apache.org/ns/synapse">
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.5.xsd">
<csv:reader fields="SYS_CD,PRO_CD,LOC_CD,AVA_QTY,RESQTY,PRO_QTY" separator="|" skipLines="1"/>
</smooks-resource-list>
</localEntry>

You can use the Smooks Mediator for this. You can simply read the file and use the property separator="|" to split the records by "|" and parse it to an XML.
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.5.xsd">
<csv:reader fields="name,age,$ignore$,address" separator="|" skipLines="1"/>
</smooks-resource-list>
Following is a VFS working sample.
Consumer Proxy
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="Consumer" startOnLoad="true" transports="vfs" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<log description="Log student file records" level="full"/>
<smooks config-key="smooks">
<input type="text"/>
<output type="xml"/>
</smooks>
<log description="Log student file records After===========" level="full"/>
</inSequence>
<outSequence/>
<faultSequence/>
</target>
<parameter name="transport.PollInterval">3</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.FileURI">file:///home/ycr/workspace/wso2/file/in</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///home/ycr/workspace/wso2/file/out</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.FileNamePattern">.*\.txt</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///home/ycr/workspace/wso2/file/out</parameter>
</proxy>
Smooks Config
<?xml version="1.0" encoding="UTF-8"?>
<localEntry key="smooks" xmlns="http://ws.apache.org/ns/synapse">
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.5.xsd">
<csv:reader fields="SYS_CD,PRO_CD,LOC_CD,AVA_QTY,RESQTY,PRO_QTY" separator="|" skipLines="1"/>
</smooks-resource-list>
</localEntry>
Output
[2022-09-20 08:24:24,200] INFO {LogMediator} - {proxy:Consumer} To: , WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:1FE89F7256E5FF52E91663676664208, Direction: request, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><text xmlns="http://ws.apache.org/commons/ns/payload">SYS_CD|PRO_CD|LOC_CD|AVA_QTY|RESQTY|PRO_QTY
POS|15|90|00|0|067656400
</text></soapenv:Body></soapenv:Envelope>
[2022-09-20 08:24:24,204] INFO {LogMediator} - {proxy:Consumer} To: , WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:1FE89F7256E5FF52E91663676664208, Direction: request, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><csv-set><csv-record number="1"><SYS_CD>POS</SYS_CD><PRO_CD>15</PRO_CD><LOC_CD>90</LOC_CD><AVA_QTY>00</AVA_QTY><RESQTY>0</RESQTY><PRO_QTY>067656400</PRO_QTY></csv-record></csv-set></soapenv:Body></soapenv:Envelope>

Related

Could not initialize Start up controller. TaskManager not found

i have written a scheduled task which call an Api in sequence in wso2 ei 6.4.0. my task is :
<task xmlns="http://ws.apache.org/ns/synapse"
name="GetMunicipalityAPIToken"
class="org.apache.synapse.startup.tasks.MessageInjector"
group="synapse.simple.quartz">
<trigger interval="60"/>
<property xmlns:task="http://www.wso2.org/products/wso2commons/tasks" name="message">
<test xmlns=""/>
</property>
<property xmlns:task="http://www.wso2.org/products/wso2commons/tasks"
name="sequenceName"
value="GetTokenFromMunicipalityService"/>
<property xmlns:task="http://www.wso2.org/products/wso2commons/tasks"
name="injectTo"
value="sequence"/>
</task>
and my sequence is :
<?xml version="1.0" encoding="UTF-8"?>
<sequence name="GetTokenFromMunicipalityService" xmlns="http://ws.apache.org/ns/synapse">
<send>
<endpoint>
<http method="POST" uri-template="http://x.x.x.x/api/MunicipalityBill/connect/token"/>
</endpoint>
</send>
</sequence>
but when i run integrator.bat in wso2eiHome/bin I got this error : "Could not initialize Start up controller. TaskManager not found" and the Api dont be called. Do i forget something ?
The error can be caused when you do not have org.wso2.carbon.mediation.ntask.NTaskTaskManager defined in the server. Could you please check whether the following is added in the synapse.xml file located in the [EI_HOME]/repository/deployment/server/synapse-configs/default/ directory
<definitions xmlns="http://ws.apache.org/ns/synapse">
<taskManager provider="org.wso2.carbon.mediation.ntask.NTaskTaskManager"/>
</definitions>

WSO2: xml to json data mapping in wso2 esb : json message is wrapped in soap envelope

I am working on one request where the input message is xml and it has to be converted to json. I am using data mapper to do this.
Configured proxy service for the same and below is the code for it:
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="xmltojson_to_Partneronb" startOnLoad="true" transports="vfs
http https" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<log level="full">
<property name="property_name" value="Initial reading file"/>
</log>
<datamapper config="gov:datamapper/ProducerPartnerOnb.dmc" inputSchema="gov:datamapper/ProducerPartnerOnb_inputSchema.json" inputType="XML" outputSchema="gov:datamapper/ProducerPartnerOnb_outputSchema.json" outputType="JSON"/>
<log level="full">
<property name="property_name" value="after enriching"/>
</log>
<property name="messageType" scope="axis2" type="STRING" value="application/json"/>
</inSequence>
<outSequence/>
<faultSequence/>
</target>
<parameter name="transport.PollInterval">1</parameter>
<parameter name="transport.vfs.FileURI">file:///tmp/in</parameter>
<parameter name="transport.vfs.ContentType">application/xml</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///tmp/failure</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.FileNamePattern">.*\.xml</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///tmp/out</parameter>
</proxy>
From the logs which you have provided the SOAP format is not framed correctly, below is the formatted SOAP Response.
<soapenv:Envelope xmlns:soapenv="schemas.xmlsoap.org/soap/envelope/">
;
<soapenv:Body>
<jsonObject>
<FirstName>ABC</FirstName>
<MiddleInitial>m</MiddleInitial>
<LastName>LN</LastName>
<Email>test#test.com</Email>
<Company>Test</Company>
<Address1>200 West St</Address1>
<Address2>hjhjhj</Address2>
</jsonObject>
</soapenv:Body>
</soapenv:Envelope>
After SOAP envelop ends there is a ; due to which you are getting error, you need to rectify this.

WSO2 Smook response not write to file

I'm trying write the response from a proxy with Smooks Mediator on a file but nothing happens.
The mediator get a csv file with vfs option, convert to xml output and send the csv to a out directory correctely.
In console i see the ws02-esb-service.log and the xml is correctly.
But i not do get write to file.
In /u01/app/wso2esb-4.7.0/tmp/send/convert then result not appears.
And not have any error.
This is the code.
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="SmookTest"
transports="vfs"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<smooks config-key="smooks-registro">
<input type="text"/>
<output type="xml"/>
</smooks>
<log level="full"/>
<property name="transport.vfs.ReplyFileName"
value="test1.xml"
scope="transport"/>
<property name="OUT_ONLY" value="true"/>
<send>
<endpoint name="grabar">
<address uri="vfs:file:///u01/app/wso2esb-4.7.0/tmp/send/convert"/>
</endpoint>
</send>
</inSequence>
</target>
<parameter name="transport.vfs.ReplyFileURI">salida.xml</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">5</parameter>
<parameter name="transport.vfs.FileURI">/u01/app/wso2esb-4.7.0/tmp/send</parameter>
<parameter name="transport.vfs.MoveAfterProcess">/u01/app/wso2esb-4.7.0/tmp/send/out</parameter>
<parameter name="transport.vfs.MoveAfterFailure">/u01/app/wso2esb-4.7.0/tmp/send/fail</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.csv</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<description/>
</proxy>
Since the VFS read as a plain/text format it will not engage the formarter until you explicitly mention it. Therefore you can mention the content type before you call the endpoint
Like
<property name="OUT_ONLY" value="true"/>
<property name="messageType"
value="application/soap"
scope="axis2"
type="STRING"/>
<send>
<endpoint name="grabar">
<address uri="vfs:file:///u01/app/wso2esb-4.7.0/tmp/send/convert"/>
</endpoint>
</send>
I found the solution.
adding format to endpoint and the file content are write correctly.
<address uri="vfs:file:///u01/app/wso2esb-4.7.0/tmp/send/convert"
format="soap11"/>

WSO2 ESB - XML to MySQL Database

I need a solution for a problem:
I just tried this example for the WSO2 ESB and it is working:
http://wso2.org/library/articles/2011/01/wso2-esb-example-file-processing
But I like to use an XML file as input and write this to a MySQL-DB.
The XML-file looks like this:
<?xml version="1.0" encoding="utf-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<Name>word</Name>
<Surname>processor</Surname>
<Phone>12333</Phone>
</soapenv:Body>
</soapenv:Envelope>
If i put this into my "in"-folder the file will be moved to the "out"-folder, but nothing is written into the database. There is no error message.
My Proxy-Service:
<proxy xmlns="http://ws.apache.org/ns/synapse" name="FileProxy" transports="vfs" statistics="disable" trace="disable" startOnLoad="true">
<target>
<inSequence>
<log level="full"/>
<clone>
<target sequence="fileWriteSequence"/>
<target sequence="databaseSequence"/>
</clone>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.MoveAfterProcess">C:/wso2esb/testordner/out</parameter>
<parameter name="transport.vfs.FileURI">C:/wso2esb/testordner/in</parameter>
<parameter name="transport.vfs.MoveAfterFailure">C:/wso2esb/testordner/error</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.xml</parameter>
<parameter name="transport.vfs.ContentType">text/xml</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<description></description>
</proxy>
And my Sequence:
<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse"
name="databaseSequence"
trace="enable"
statistics="enable">
<log level="full">
<property name="sequence" value="before-smooks"/>
</log>
<smooks config-key="smooks">
<input type="xml"/>
<output type="xml"/>
</smooks>
<log level="full">
<property name="sequence" value="after-smooks"/>
</log>
<log level="full">
<property xmlns:ns="http://org.apache.synapse/xsd"
name="name"
expression="//csv-record/name/text()"/>
<property xmlns:ns="http://org.apache.synapse/xsd"
name="surname"
expression="//csv-record/surname/text()"/>
<property name="phone" value="//csv-record/phone/text()"/>
</log>
<iterate xmlns:ns2="http://org.apache.synapse/xsd"
xmlns:ns="http://org.apache.synapse/xsd"
xmlns:sec="http://secservice.samples.esb.wso2.org"
expression="//csv-set/csv-record">
<target>
<sequence>
<log level="full">
<property name="State" value="Iteration"/>
</log>
<log level="full" category="DEBUG"/>
<dbreport>
<connection>
<pool>
<password>passwort</password>
<user>esb</user>
<url>jdbc:mysql://localhost:3306/</url>
<driver>com.mysql.jdbc.Driver</driver>
</pool>
</connection>
<statement>
<sql>insert into info.info (name, surname, phone) values (?, ?, ?)</sql>
<parameter expression="//csv-record/name/text()" type="VARCHAR"/>
<parameter expression="//csv-record/surname/text()" type="VARCHAR"/>
<parameter expression="//csv-record/phone/text()" type="VARCHAR"/>
</statement>
</dbreport>
</sequence>
</target>
</iterate>
<log level="full" category="DEBUG"/>
</sequence>
I thought this would be a simple problem, but I'm not able to get it...
Like I said, it's working with a normal text-file, but not with XML. I tried different thinks and have no idea what else I could try...
Does someone know what's my mistake?
Update:
The log from the console:
[2013-04-12 09:35:55,687] INFO - DBReportMediator Successfully created data source for jdbc:mysql://localhost:3306/.
[2013-04-12 09:36:10,170] INFO - LogMediator To: , WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:
2CBD1753113A1373101365752170177, Direction: request, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope x
mlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body>
<Name>word</Name>
<Surname>processor</Surname>
<Phone>12333</Phone>
</soapenv:Body></soapenv:Envelope>
[2013-04-12 09:36:10,175] INFO - LogMediator To: , WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:
a9252cd8-fe73-4e85-9f41-0842872d3a73, Direction: request, sequence = before-smooks, Envelope: <?xml version='1.0' encodi
ng='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body>
<Name>word</Name>
<Surname>processor</Surname>
<Phone>12333</Phone>
</soapenv:Body></soapenv:Envelope>
[2013-04-12 09:36:10,178] INFO - LogMediator sequence = fileWriteSequence
[2013-04-12 09:36:10,246] INFO - LogMediator To: , WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:
a9252cd8-fe73-4e85-9f41-0842872d3a73, Direction: request, sequence = after-smooks, Envelope: <?xml version='1.0' encodin
g='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><csv-set/></soapenv
:Body></soapenv:Envelope>
[2013-04-12 09:36:10,249] INFO - LogMediator To: , WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:
a9252cd8-fe73-4e85-9f41-0842872d3a73, Direction: request, name = , surname = , phone = //csv-record/phone/text(), Envelo
pe: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><s
oapenv:Body><csv-set/></soapenv:Body></soapenv:Envelope>
[2013-04-12 09:36:11,254] WARN - TimeoutHandler Expiring message ID : urn:uuid:4c422f4e-aed5-4eaa-ad9b-e5407db20562; dr
opping message after global timeout of : 120 seconds
The fileWriteSequence looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse"
name="fileWriteSequence"
trace="enable"
statistics="enable">
<log level="custom">
<property name="sequence" value="fileWriteSequence"/>
</log>
<property xmlns:ns2="http://org.apache.synapse/xsd"
xmlns:ns="http://org.apache.synapse/xsd"
name="transport.vfs.ReplyFileName"
expression="fn:concat(fn:substring-after(get-property('MessageID'), 'urn:uuid:'), '.xml')"
scope="transport"
type="STRING"/>
<property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
<send>
<endpoint name="FileEpr">
<address uri="vfs:C:/wso2esb/testordner/out"/>
</endpoint>
</send>
</sequence>
Smooks config:
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.0.xsd">
<!--Configure the CSVParser to parse the message into a stream of SAX events. -->
<resource-config selector="org.xml.sax.driver">
<resource>org.milyn.csv.CSVParser</resource>
<param name="fields" type="string-list">name,surname,phone</param>
</resource-config>
</smooks-resource-list>
I also changed the code of the databaseSequence above.
Thank you.
Your proxy service looks ok. But I have few concerns.
Why have you put the sql query as CDATA. Please see the sample available here.
You have not mentioned the database name in the connection url. May be it is not a problem since you have used the database.table format in your sql query.
After fixing and 1 and 2, if the issue still exists, try your proxy with only one sequence. i.e. Avoid the file write sequence temporarily and try only with the dbwrite sequence.
You can also log the entries which you are going to insert before the DBReport mediator. i.e. Right after the smooks mediator, put a log mediator and log the name, surname and phone. This will make sure you are trying to insert values which are correctly extracted from the message.

Filtering in WSO2

I want to build one proxy that:
1. Call Service that do Authorize and give result OK or Fail (1st Service)
2. If Result ‘OK’ then call a Service
The Problem is, when The 1st Service give back the Message :
<?xml version='1.0' encoding='utf-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<result>
<status>OK</status>
<message></message>
</result>
</soapenv:Body>
</soapenv:Envelope>
And I give “filtering” at Out Sequence. Here is the XML :
<proxy xmlns="http://ws.apache.org/ns/synapse" name="TestProxy" transports="https,http" statistics="disable" trace="enable" startOnLoad="true">
<target endpoint="AuthorizationService">
<outSequence>
<log level="full" />
<filter xpath="/result/status='OK'">
<then>
<send>
<endpoint>
<address uri="http://192.168.1.140:8080/axis2/services/TaskService.TaskServiceHttpEndpoint/getTask" />
</endpoint>
</send>
</then>
<else>
<makefault version="soap11">
<code xmlns:soap11Env="http://schemas.xmlsoap.org/soap/envelope/" value="soap11Env:VersionMismatch" />
<reason value="1" />
<role>2</role>
<detail>3</detail>
</makefault>
</else>
</filter>
<log level="full" />
</outSequence>
</target>
</proxy>
When I run my Application, the ESB always give the message :
16:08:59,358 [-] [HttpClientWorker-4] INFO Start : Log mediator
16:08:59,361 [-] [HttpClientWorker-4] INFO To: http://www.w3.org/2005/08/addressing/anonymous, WSAction: , SOAPAction: , MessageID: urn:uuid:0bc33821-c4f1-448e-a7dc-be4194be8e99, Direction: response, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><result><status>OK</status><message></message></result></soapenv:Body></soapenv:Envelope>
16:08:59,361 [-] [HttpClientWorker-4] INFO End : Log mediator
16:08:59,361 [-] [HttpClientWorker-4] INFO Start : Filter mediator
16:08:59,361 [-] [HttpClientWorker-4] INFO XPath expression : /result/status='OK' evaluates to false - executing the else path child mediators
Seems like the condition of the filtering is always false.What is the correct statement for the XPath in the filter?
Your proxy configuration should be look as follows
<proxy xmlns="http://ws.apache.org/ns/synapse" name="TestProxy" transports="https,http" statistics="disable" trace="enable" startOnLoad="true">
<target endpoint="AuthorizationService">
<outSequence>
<log level="full"/>
<filter source="/result/status" regex="ok">
<then>
<send>
<endpoint>
<address uri="http://192.168.1.140:8080/axis2/services/TaskService.TaskServiceHttpEndpoint/getTask"/>
</endpoint>
</send>
</then>
<else>
<makefault version="soap11">
<code xmlns:soap11Env="http://schemas.xmlsoap.org/soap/envelope/" value="soap11Env:VersionMismatch"/>
<reason value="1"/>
<role>2</role>
<detail>3</detail>
</makefault>
<send/>
</else>
</filter>
</outSequence>
</target>
<description></description>
</proxy>
It seems that you may have given a wrong xpath expression. You can't give an xpath and a boolean expression both for the xpath value, i.e. it can't be "/result/status='OK'", but has to be "/result/status". Then, according to your sequence, it would fire the section after then, if this element is present. Since, you need to evaluate a boolean condition as well based on the xpath, I'll present an alternative based on the switch mediator (Same can be done for the filter by setting a property):
<proxy xmlns="http://ws.apache.org/ns/synapse" name="TestProxy" transports="https,http" statistics="disable" trace="enable" startOnLoad="true">
<target endpoint="AuthorizationService">
<outSequence>
<log level="full" />
<switch source="//result/status">
<case regex="OK">
<send>
<endpoint>
<address uri="http://192.168.1.140:8080/axis2/services/TaskService.TaskServiceHttpEndpoint/getTask" />
</endpoint>
</send>
</case>
<default>
<makefault version="soap11">
<code xmlns:soap11Env="http://schemas.xmlsoap.org/soap/envelope/" value="soap11Env:VersionMismatch" />
<reason value="1" />
<role>2</role>
<detail>3</detail>
</makefault>
</default>
</switch>
<log level="full" />
</outSequence>
</target>
</proxy>