Testing a POC and wondering what the quickest/simplest way to pretty print a JSON payload in the Anypoint Studio console. I'd rather not figure out how to use a Groovy component to accomplish this.
I've tried setting the application/json indent property to true as below, but it still prints the entire object on a single line (which is indented).
output application/json indent=true
Any lightweight solutions for quick POC work?
The console in Studio shows what it is being logged. If you are using DataWeave to create the JSON, as evidenced by the script you shared, then the default value for the indent writer property is true. So when logging it, the generated JSON will be pretty-printed by default.
Example:
<ee:transform doc:name="Transform Message" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
{
a:"aaaa",
b: {c:1, d:"dddd"},
e: "ccc"
}]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" message="Payload: #[payload]"/>
Output (omitting all the logging information for clarity):
Payload: {
"a": "aaaa",
"b": {
"c": 1,
"d": "dddd"
},
"e": "ccc"
}
Related
I have a json file which contains a collection of numerous JSON objects. A sample format is given below:
{"ID": 123,"Name": "TEST-1","val11": {},"url": "test1.com","val12": []}
{"ID": 456,"Name": "TEST-2","val21": {},"url": "test2.com","val22": []}
{"ID": 789,"Name": "TEST-3","val31": {},"url": "test3.com","val32": []}
As you see, it is not an array ([ ] and commas missing). I need to convert this into a valid JSON array.
The code that I tried is:
%dw 2.0
output application/json
var PayloadSplit = payload splitBy('\n')
var PayloadArray = (PayloadSplit map (value, index) -> read(value, 'application/json'))
---
PayloadArray
This works fine for a small sized payload. However, if I try to perform this on the entire file (size about 320 MB with ~20k JSON objects), it fails with a java.lang.OutOfMemoryError: Java heap space error. Is there a way to overcome this? Or can I split the main file into multiple files and then try this (in a ForEach Loop perhaps?). Please advise
Edit1 - Attaching the mule flow below:
<flow name="convert-object-to-array-test1Flow" doc:id="0645e9bd-7f77-4b1e-93d0-dedd9d154ef7" >
<http:listener doc:name="Listener" doc:id="551cd3b6-e4c8-4b7a-aff3-305effbe8a8b" config-ref="HTTP_Listener_config" path="/file"/>
<file:read doc:name="Read" doc:id="21a310c1-5887-4bc0-83b9-b8968e145f0d" path="C:\Desktop\NDJsonSample.json" outputMimeType="application/ndjson" />
<ee:transform doc:name="Transform Message" doc:id="95235c56-2f5a-4f39-ba96-8be7c4e501b5" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" doc:id="935530dd-17fd-41c9-8fe0-1561ba3de703" />
</flow>
DW already have support for this format. It is called ndjson. Please visit the documentation. You just need to set application/ndjson to the payload.
I need to convert Pipe Delimited text file to JSON in Mule 4. I tried with below code.
`<file:listener doc:name="Fetch Input from Geometry SFTP" doc:id="1151a602-6748-43b7-b491-2caedf6f7010" directory="C:\Vijay\My Projects\AM-Mexico\US\AIM-65\Input" autoDelete="true" outputMimeType="text/csv; separator=|" recursive="false">
<non-repeatable-stream />
<scheduling-strategy >
<fixed-frequency frequency="10" timeUnit="SECONDS"/>
</scheduling-strategy>
</file:listener>
<ee:transform doc:name="Transform Message" doc:id="79dcddf9-34a0-4005-88b4-3a395544be8c" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
input payload text/csv
---
payload]]></ee:set-payload>
</ee:message>
</ee:transform>`
If I try to execute the code i am getting exception as below.
Message : "Internal execution exception while executing the script this is most probably a bug.
Caused by:
java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:147)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at $java.io.InputStream$$FastClassByCGLIB$$31b19c4e.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.mule.extension.file.common.api.stream.AbstractFileInputStream.lambda$createLazyStream$0(AbstractFileInputStream.java:41)
at $java.io.InputStream$$EnhancerByCGLIB$$55e4687e.read(<generated>)
at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:98)
at org.mule.weave.v2.io.DefaultSeekableStream.readUntil(SeekableStream.scala:193)
at org.mule.weave.v2.io.DefaultSeekableStream.delegate$lzycompute(SeekableStream.scala:202)
at org.mule.weave.v2.io.DefaultSeekableStream.delegate(SeekableStream.scala:201)
at org.mule.weave.v2.io.DefaultSeekableStream.seek(SeekableStream.scala:234)
at org.mule.weave.v2.io.SeekableStream.resetStream(SeekableStream.scala:17)
at org.mule.weave.v2.io.SeekableStream.resetStream$(SeekableStream.scala:16)
at org.mule.weave.v2.io.DefaultSeekableStream.resetStream(SeekableStream.scala:186)
at org.mule.weave.v2.model.values.BinaryValue$.getBytesFromSeekableStream(BinaryValue.scala:84)
at org.mule.weave.v2.model.values.BinaryValue$.getBytes(BinaryValue.scala:68)
at org.mule.weave.v2.model.values.BinaryValue.equals(BinaryValue.scala:26)
at org.mule.weave.v2.model.values.BinaryValue.equals$(BinaryValue.scala:25)
at org.mule.weave.v2.module.pojo.reader.JavaBinaryValue.equals(JavaBinaryValue.scala:11)
at org.mule.weave.v2.model.values.wrappers.DelegateValue.equals(DelegateValue.scala:38)
at org.mule.weave.v2.model.values.wrappers.DelegateValue.equals$(DelegateValue.scala:37)
at org.mule.weave.v2.model.values.wrappers.LazyValue.equals(DelegateValue.scala:65)
at org.mule.weave.v2.model.types.Type.$anonfun$acceptsSchema$2(Type.scala:203)
at org.mule.weave.v2.model.types.Type.$anonfun$acceptsSchema$2$adapted(Type.scala:198)
any help on this would be higly appreciated.
You need to set the reader properties for separator:
%dw 2.0
input payload application/csv separator='|'
output application/json
---
payload
Relevant documentation here.
I am getting an error while using a dataweave transformation on a JSON Payload. The JSON Payload is
{
"requestId": "13431#1638a2abfb8",
"result": [
{
"batchId": 1028,
"importId": "1028",
"status": "Queued"
}
],
"success": true
}
The above payload is returned by a RESTful service and I have converted that to a object using byteArray to Object transformer before applying the following dataweave transformation
%dw 1.0
%output application/json
---
batchexecution:
{
batchid:payload.result[0].batchid,
status: payload.result[0].status,
success:payload.success
} when ((payload.result != null) and (sizeOf payload.result > 0))
otherwise
{
batchid: 0,
status:"Not queued",
success:false
}
I am expecting only one record for the result object and I have a check to see whether the array is null or its size is >0. I get the following error when I execute the transformation code. Not sure what is wrong here.
I am expecting the following output for the transformation but I am getting the error while executing the transformation code
{
"batchexecution": {
"batchId": 1028,
"status": "Queued",
"success": true
}
}
But I am getting the following error as You cannot compare a value of type ::array.
Message : Exception while executing:
{"requestId":"64b3#1638e55058c","result":[{"batchId":1037,"importId":"1037","status":"Queued"}],"success":true}
^
You cannot compare a value of type ::array.
Payload : {"requestId":"64b3#1638e55058c","result":[{"batchId":1037,"importId":"1037","status":"Queued"}],"success":true}
Payload Type : java.lang.String
Element : /marketing-dbmkt-etl-marketoFlow/processors/8 # marketing-dbmkt-etl-marketo:marketing-dbmkt-etl-marketo.xml:69 (Transform Message)
Element XML : <dw:transform-message doc:name="Transform Message" metadata:id="90448cfd-5884-441a-a989-e32e4877ac24">
<dw:input-payload mimeType="application/json" doc:sample="sample_data\batchreturnObject.dwl"></dw:input-payload>
<dw:set-payload>%dw 1.0%output application/json---batchexecution:{batchid:payload.result[0].batchid,status: payload.result[0].status,success:payload.success} when ((payload.result != null) and (sizeOf payload.result > 0))otherwise{batchid: 0,status:"Not queued",success:false}</dw:set-payload>
</dw:transform-message>
--------------------------------------------------------------------------------
Root Exception stack trace:
com.mulesoft.weave.mule.exception.WeaveExecutionException: Exception while executing:
{"requestId":"64b3#1638e55058c","result":[{"batchId":1037,"importId":"1037","status":"Queued"}],"success":true}
^
You cannot compare a value of type ::array.
at com.mulesoft.weave.mule.exception.WeaveExecutionException$.apply(WeaveExecutionException.scala:10)
at com.mulesoft.weave.mule.WeaveMessageProcessor.execute(WeaveMessageProcessor.scala:121)
at com.mulesoft.weave.mule.WeaveMessageProcessor.process(WeaveMessageProcessor.scala:67)
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:27)
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:108)
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88)
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:27)
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:98)
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
at org.mule.interceptor.AbstractEnvelopeInterceptor.processBlocking(AbstractEnvelopeInterceptor.java:58)
at org.mule.processor.AbstractRequestResponseMessageProcessor.process(AbstractRequestResponseMessageProcessor.java:47)
at org.mule.processor.AsyncInterceptingMessageProcessor.processNextTimed(AsyncInterceptingMessageProcessor.java:129)
at org.mule.processor.AsyncInterceptingMessageProcessor$AsyncMessageProcessorWorker$1.process(AsyncInterceptingMessageProcessor.java:213)
at org.mule.processor.AsyncInterceptingMessageProcessor$AsyncMessageProcessorWorker$1.process(AsyncInterceptingMessageProcessor.java:206)
at org.mule.execution.ExecuteCallbackInterceptor.execute(ExecuteCallbackInterceptor.java:16)
at org.mule.execution.CommitTransactionInterceptor.execute(CommitTransactionInterceptor.java:35)
at org.mule.execution.CommitTransactionInterceptor.execute(CommitTransactionInterceptor.java:22)
at org.mule.execution.HandleExceptionInterceptor.execute(HandleExceptionInterceptor.java:30)
at org.mule.execution.HandleExceptionInterceptor.execute(HandleExceptionInterceptor.java:14)
at org.mule.execution.BeginAndResolveTransactionInterceptor.execute(BeginAndResolveTransactionInterceptor.java:67)
at org.mule.execution.ResolvePreviousTransactionInterceptor.execute(ResolvePreviousTransactionInterceptor.java:44)
at org.mule.execution.SuspendXaTransactionInterceptor.execute(SuspendXaTransactionInterceptor.java:50)
at org.mule.execution.ValidateTransactionalStateInterceptor.execute(ValidateTransactionalStateInterceptor.java:40)
at org.mule.execution.IsolateCurrentTransactionInterceptor.execute(IsolateCurrentTransactionInterceptor.java:41)
at org.mule.execution.ExternalTransactionInterceptor.execute(ExternalTransactionInterceptor.java:48)
at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:28)
at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:13)
at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:110)
at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:30)
at org.mule.processor.AsyncInterceptingMessageProcessor$AsyncMessageProcessorWorker.doRun(AsyncInterceptingMessageProcessor.java:205)
at org.mule.work.AbstractMuleEventWork.run(AbstractMuleEventWork.java:53)
at org.mule.work.WorkerContext.run(WorkerContext.java:301)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
********************************************************************************
The problem here is not obvious, but I have come across the same issue before - it's related to the sizeOf function and the poor way that Mule applies precedence to some of it's operators. When you say (sizeOf payload.result > 0) it's first trying to attempt to resolve the payload.result > 0 expression - hence the error you're seeing (it's trying to compare an array to 0).
Please use ((sizeOf payload.result) > 0) instead (I always make a point of wrapping sizeOf in parentheses for this reason).
As a side note, you have batchid:payload.result[0].batchid - it should be batchId:payload.result[0].batchId (capitalisation in batchId)
whenever you are using any function like sizeOf in dataweave try to encapsulate it with round braces to avoid these kinds of errors.
#ghoshyTech in your case
when ((payload.result != null) and ((sizeOf payload.result) > 0))
I am using Anypoint Studio 6.1 and Mule 3.8.1 and have the following JSON response from a Mule workflow:
{
"Description": "Top 10 games:
- Mario Kart
- Legend of Zelda"
}
This fails validation with the following error:
Error: Parse error on line 2:
{ "Description": "Top 10 games: - M
-----------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'
I have tried escaping the special characters put it still does not work. How can I change the response to pass JSON v4 validation. The JSON validator I have been using is http://jsonlint.com/
Thanks
The root cause of the issue is carriage return in the JSON response. The valid JSON should be as below
{
"Description": "Top 10 games:\r\n- Mario Kart\r\n- Legend of Zelda"
}
Please provide more details on where you are receiving this description. is it from XML or any other source. Also it will be helpful if you post your DWL code as well.
As per my understanding "ffprobe" will provide file related data in JSON format. So, I have installed the ffprobe in my Ubuntu machine but I don't know how to access the ffprobe JSON response using Java/Grails.
Expected response format:
{
"format": {
"filename": "/Users/karthick/Documents/videos/TestVideos/sample.ts",
"nb_streams": 2,
"nb_programs": 1,
"format_name": "mpegts",
"format_long_name": "MPEG-TS (MPEG-2 Transport Stream)",
"start_time": "1.430800",
"duration": "170.097489",
"size": "80425836",
"bit_rate": "3782576",
"probe_score": 100
}
}
This is my groovy code
def process = "ffprobe -v quiet -print_format json -show_format -show_streams HelloWorld.mpeg ".execute()
println "Found ${process.text}"
render process as JSON
I m able to get the process object and i m not able to get the json response
Should i want to convert the process object to json object?
OUTPUT:
Found java.lang.UNIXProcess#75566697
org.codehaus.groovy.grails.web.converters.exceptions.ConverterException: Error converting Bean with class java.lang.UNIXProcess
Grails has nothing to do with this. Groovy can execute arbitrary shell commands in a very simplistic way:
"mkdir foo".execute()
Or for more advanced features, you might look into using ProcessBuilder. At the end of the day, you need to execute ffprobe and then capture the output stream of JSON to use in your app.
Groovy provides a simple way to execute command line processes. Simply
write the command line as a string and call the execute() method.
The execute() method returns a java.lang.Process instance.
println "ffprobe <options>".execute().text
[Source]