How to convert Json objects into arrays [(string, double)] in Scala JSON4s? - json

I don't know how to use CustomSerializer to convert JSON to Scala data
I don't need to serialize the parts, but deserialization is difficult
Can someone help me? Thank you very much
{
"parameters": {
"field_owner": true,
"sample_field": "gender",
"sample_strategy": "by_ratio",
"sample_config": [["male", 1500.0], ["female", 1500.0]],
"with_replacement": true,
"random_seed": 114514,
"least_sample_num": 100
},
"input_artifacts": {"dataset": {"object_id": "upload/ml_classifier_predict_result"}},
"output_artifacts": {"predict_result": {"object_id": "ingest_output.csv"}}
}
class TupleSerializer extends CustomSerializer[(String,Double)](tuple_formats => (
{
case JArray(List(JString(x), JDouble(y))) =>
(x,y)
},{
case x:Tuple2[String,Double] => null
}
))
implicit val formats: Formats = DefaultFormats + new TupleSerializer
val fileBuffer = Source.fromFile(path)
val jsonString = fileBuffer.mkString
parse(jsonString).extract[StratifiedSampleArgument]
}
error message:
o usable value for parameters
No usable value for sample_config
Expected object with 1 element but got JArray(List(JString(male), JDouble(1500.0)))
at com.alipay.morse.sgxengine.driver.StratifiedSampleDriverTest.main(StratifiedSampleDriverTest.scala:15)
Caused by: org.json4s.package$MappingException:
No usable value for sample_config
Expected object with 1 element but got JArray(List(JString(male), JDouble(1500.0)))
at com.alipay.morse.sgxengine.driver.StratifiedSampleDriverTest.main(StratifiedSampleDriverTest.scala:15)
Caused by: org.json4s.package$MappingException: Expected object with 1 element but got JArray(List(JString(male), JDouble(1500.0)))
at com.alipay.morse.sgxengine.driver.StratifiedSampleDriverTest.main(StratifiedSampleDriverTest.scala:15)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR] StratifiedSampleDriverTest.main:15 » Mapping No usable value for parameters
No...
[INFO]
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

Related

Play Framework WebSocket ArrayIndexOutOfBounds Error

I'm having a WebSocket endpoint exposed by the Play Framework like this:
def socket: WebSocket = WebSocket.acceptOrResult[JsValue, JsValue] { request =>
Future.successful(
if (acceptedSubProtocol(request.headers, appConfig.ocppServerCfg.supportedProtocols)) { // TODO: Get the Protocol via AppConfig
Right(ActorFlow.actorRef { actorRef =>
Props(new ProvisioningActor(actorRef))
})
} else {
logger.warn(s"Supported Protocol is not one of ${appConfig.ocppServerCfg.supportedProtocols.mkString(",")} " +
"in the Sec-WebSocket-Protocol header")
Left(Forbidden)
}
)
}
The following implicit conversion for the incoming Json which looks like this:
implicit val ocppCallRequestReads2: Reads[OCPPCallRequest] = Reads { jsValue =>
val messageTypeIdE = (jsValue \ 0).toEither
val messageIdE = (jsValue \ 1).toEither
val actionNameE = (jsValue \ 2).toEither
val payloadE = (jsValue \ 3).toEither
val yielded = for {
messageTypeId <- messageTypeIdE
messageId <- messageIdE
actionName <- actionNameE
payload <- payloadE
} yield {
OCPPCallRequest( // Here I know all 4 exists, so safe to call head
messageTypeId.head.as[Int],
messageId.head.as[String],
actionName.head.as[String],
payload
)
}
yielded match {
case Right(ocppCallRequest) => JsSuccess(ocppCallRequest)
case Left(errors) =>
println("****************+ ERRORS")
errors.messages.foreach(println)
println("****************+ ERRORS")
JsError(errors)
}
}
And the actual JSON:
[
"19223201",
"BootNotification",
{
"reason": "PowerUp",
"chargingStation": {
"serialNumber" : "12345",
"model" : "",
"vendorName" : "",
"firmwareVersion" : "",
"modem": {
"iccid": "",
"imsi": ""
}
}
}
]
What I'm trying to do is to validate the incoming JSON and propogate any errors to the client. When I tried to run my example., I'm not able to pass the error back as a JSON response, but the WebSocket endpoint closes with an internal server error as seen below:
[info] a.a.CoordinatedShutdown - Running CoordinatedShutdown with reason [ApplicationStoppedReason]
[info] a.e.s.Slf4jLogger - Slf4jLogger started
[info] play.api.Play - Application started (Dev) (no global state)
****************+ ERRORS
Array index out of bounds in ["19223201","BootNotification",{"reason":"PowerUp","chargingStation":{"serialNumber":"12345","model":"","vendorName":"","firmwareVersion":"","modem":{"iccid":"","imsi":""}}}]
****************+ ERRORS
[error] p.c.s.c.WebSocketFlowHandler - WebSocket flow threw exception
java.lang.ClassCastException: scala.Tuple2 cannot be cast to play.api.libs.json.JsValue
at akka.stream.impl.fusing.Map$$anon$1.onPush(Ops.scala:52)
at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:542)
at akka.stream.impl.fusing.GraphInterpreter.processEvent(GraphInterpreter.scala:496)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:390)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:650)
at akka.stream.impl.fusing.ActorGraphInterpreter$SimpleBoundaryEvent.execute(ActorGraphInterpreter.scala:61)
at akka.stream.impl.fusing.ActorGraphInterpreter$SimpleBoundaryEvent.execute$(ActorGraphInterpreter.scala:57)
at akka.stream.impl.fusing.ActorGraphInterpreter$BatchingActorInputBoundary$OnNext.execute(ActorGraphInterpreter.scala:104)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:625)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:800)
Any clues as to how I can control the WebSocket endpoint not to crash into an exception? What happens is that the connection gets closed and I do not want that.

JSON Object Property with space in Scala using Json4s(Jackson)

I have a JSON String in the given below form:
var jStr =
""" {
|"company":{
|"company name":"ABCD"
|},
|"person":[
|{"name":"john",
|"age":"28"
|},
|{
|"name":"AWQ",
|"age":"45"
|}
|]
|}
""".stripMargin
See the property "company name". Due to this I cannot extract it from its json form to its case class without changing the "company name" to "company_name"(or anything else). Below is the code:
import com.fasterxml.jackson.annotation.{JsonCreator, JsonProperty}
import org.json4s._
import org.json4s.native.JsonMethods._
val parseJson = parse(jStr)
var obj = parseJson.extract[Info] // Exception Here
case class Company(cname : String)
case class Info(company: Company,person: List[Person])
case class Person(name : String , age : String)
I tried using #JsonProperty and #JsonCreator but they too failed.
#JsonCreator
case class Company( #JsonProperty("company name") cname : String)
case class Info(company: Company, person: List[Person])
case class Person(name : String , age : String)
I need to map a json property having spaces to its respective case class(which obviously can't have a space!) using Json4s in Scala.
Stacktrace:
Exception in thread "main" org.json4s.package$MappingException: No usable value for company
No usable value for cname
Did not find value which can be converted into java.lang.String
at org.json4s.reflect.package$.fail(package.scala:95)
at org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:548)
at org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:572)
at org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:570)
at scala.PartialFunction.$anonfun$runWith$1$adapted(PartialFunction.scala:145)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.collect(TraversableLike.scala:407)
at scala.collection.TraversableLike.collect$(TraversableLike.scala:405)
at scala.collection.AbstractTraversable.collect(Traversable.scala:108)
at org.json4s.Extraction$ClassInstanceBuilder.instantiate(Extraction.scala:570)
at org.json4s.Extraction$ClassInstanceBuilder.result(Extraction.scala:630)
at org.json4s.Extraction$.$anonfun$extract$10(Extraction.scala:416)
at org.json4s.Extraction$.$anonfun$customOrElse$1(Extraction.scala:637)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
at scala.PartialFunction$$anon$1.applyOrElse(PartialFunction.scala:257)
at org.json4s.Extraction$.customOrElse(Extraction.scala:637)
at org.json4s.Extraction$.extract(Extraction.scala:408)
at org.json4s.Extraction$.extract(Extraction.scala:40)
at org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
at JsonTest.convertToJSON(JsonTest.scala:100)
at Main$.main(Main.scala:8)
at Main.main(Main.scala)
Caused by: org.json4s.package$MappingException: No usable value for cname
Did not find value which can be converted into java.lang.String
at org.json4s.reflect.package$.fail(package.scala:95)
at org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:548)
at org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:572)
at org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:570)
at scala.PartialFunction.$anonfun$runWith$1$adapted(PartialFunction.scala:145)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.collect(TraversableLike.scala:407)
at scala.collection.TraversableLike.collect$(TraversableLike.scala:405)
at scala.collection.AbstractTraversable.collect(Traversable.scala:108)
at org.json4s.Extraction$ClassInstanceBuilder.instantiate(Extraction.scala:570)
at org.json4s.Extraction$ClassInstanceBuilder.result(Extraction.scala:630)
at org.json4s.Extraction$.$anonfun$extract$10(Extraction.scala:416)
at org.json4s.Extraction$.$anonfun$customOrElse$1(Extraction.scala:637)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
at scala.PartialFunction$$anon$1.applyOrElse(PartialFunction.scala:257)
at org.json4s.Extraction$.customOrElse(Extraction.scala:637)
at org.json4s.Extraction$.extract(Extraction.scala:408)
at org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:534)
... 23 more
Caused by: org.json4s.package$MappingException: Did not find value which can be converted into java.lang.String
at org.json4s.reflect.package$.fail(package.scala:95)
at org.json4s.Extraction$.$anonfun$convert$2(Extraction.scala:735)
at scala.Option.getOrElse(Option.scala:189)
at org.json4s.Extraction$.convert(Extraction.scala:735)
at org.json4s.Extraction$.$anonfun$extract$10(Extraction.scala:410)
at org.json4s.Extraction$.$anonfun$customOrElse$1(Extraction.scala:637)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126)
at scala.PartialFunction$$anon$1.applyOrElse(PartialFunction.scala:257)
at org.json4s.Extraction$.customOrElse(Extraction.scala:637)
at org.json4s.Extraction$.extract(Extraction.scala:408)
at org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:534)
... 42 more
Process finished with exit code 1

how to convert from JSON to a parametric nested struct in julia?

stringtest="""{ "struct_A": {
"arg1": {
"arg_B": 90
},
"arg2": 100
}
}"""
jsonDict=JSON.parse(stringtest)
struct struct_B<: myStruct
arg
end
struct struct_A{T<:myStruct}
arg1::T
arg2
function BatteryStorage{T}(arg1,arg2) where {T}
return new{T}(arg1,arg2)
end
end
to_struct=ToStruct.tostruct(struct_A,jsonDict)
why I'm always getting :
LoadError: MethodError: no method matching struct_A(::Dict{String, Any})
caused by: MethodError: Cannot convert an object of type
Dict{String, Any} to an object of type
struct_A
Closest candidates are:
convert(::Type{T}, ::T) where T at essentials.jl:205

Error in Dataweave transformation involving a JSON Payload in Mule 3.8.5

I am getting an error while using a dataweave transformation on a JSON Payload. The JSON Payload is
{
"requestId": "13431#1638a2abfb8",
"result": [
{
"batchId": 1028,
"importId": "1028",
"status": "Queued"
}
],
"success": true
}
The above payload is returned by a RESTful service and I have converted that to a object using byteArray to Object transformer before applying the following dataweave transformation
%dw 1.0
%output application/json
---
batchexecution:
{
batchid:payload.result[0].batchid,
status: payload.result[0].status,
success:payload.success
} when ((payload.result != null) and (sizeOf payload.result > 0))
otherwise
{
batchid: 0,
status:"Not queued",
success:false
}
I am expecting only one record for the result object and I have a check to see whether the array is null or its size is >0. I get the following error when I execute the transformation code. Not sure what is wrong here.
I am expecting the following output for the transformation but I am getting the error while executing the transformation code
{
"batchexecution": {
"batchId": 1028,
"status": "Queued",
"success": true
}
}
But I am getting the following error as You cannot compare a value of type ::array.
Message : Exception while executing:
{"requestId":"64b3#1638e55058c","result":[{"batchId":1037,"importId":"1037","status":"Queued"}],"success":true}
^
You cannot compare a value of type ::array.
Payload : {"requestId":"64b3#1638e55058c","result":[{"batchId":1037,"importId":"1037","status":"Queued"}],"success":true}
Payload Type : java.lang.String
Element : /marketing-dbmkt-etl-marketoFlow/processors/8 # marketing-dbmkt-etl-marketo:marketing-dbmkt-etl-marketo.xml:69 (Transform Message)
Element XML : <dw:transform-message doc:name="Transform Message" metadata:id="90448cfd-5884-441a-a989-e32e4877ac24">
<dw:input-payload mimeType="application/json" doc:sample="sample_data\batchreturnObject.dwl"></dw:input-payload>
<dw:set-payload>%dw 1.0%output application/json---batchexecution:{batchid:payload.result[0].batchid,status: payload.result[0].status,success:payload.success} when ((payload.result != null) and (sizeOf payload.result > 0))otherwise{batchid: 0,status:"Not queued",success:false}</dw:set-payload>
</dw:transform-message>
--------------------------------------------------------------------------------
Root Exception stack trace:
com.mulesoft.weave.mule.exception.WeaveExecutionException: Exception while executing:
{"requestId":"64b3#1638e55058c","result":[{"batchId":1037,"importId":"1037","status":"Queued"}],"success":true}
^
You cannot compare a value of type ::array.
at com.mulesoft.weave.mule.exception.WeaveExecutionException$.apply(WeaveExecutionException.scala:10)
at com.mulesoft.weave.mule.WeaveMessageProcessor.execute(WeaveMessageProcessor.scala:121)
at com.mulesoft.weave.mule.WeaveMessageProcessor.process(WeaveMessageProcessor.scala:67)
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:27)
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:108)
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88)
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:27)
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:98)
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
at org.mule.interceptor.AbstractEnvelopeInterceptor.processBlocking(AbstractEnvelopeInterceptor.java:58)
at org.mule.processor.AbstractRequestResponseMessageProcessor.process(AbstractRequestResponseMessageProcessor.java:47)
at org.mule.processor.AsyncInterceptingMessageProcessor.processNextTimed(AsyncInterceptingMessageProcessor.java:129)
at org.mule.processor.AsyncInterceptingMessageProcessor$AsyncMessageProcessorWorker$1.process(AsyncInterceptingMessageProcessor.java:213)
at org.mule.processor.AsyncInterceptingMessageProcessor$AsyncMessageProcessorWorker$1.process(AsyncInterceptingMessageProcessor.java:206)
at org.mule.execution.ExecuteCallbackInterceptor.execute(ExecuteCallbackInterceptor.java:16)
at org.mule.execution.CommitTransactionInterceptor.execute(CommitTransactionInterceptor.java:35)
at org.mule.execution.CommitTransactionInterceptor.execute(CommitTransactionInterceptor.java:22)
at org.mule.execution.HandleExceptionInterceptor.execute(HandleExceptionInterceptor.java:30)
at org.mule.execution.HandleExceptionInterceptor.execute(HandleExceptionInterceptor.java:14)
at org.mule.execution.BeginAndResolveTransactionInterceptor.execute(BeginAndResolveTransactionInterceptor.java:67)
at org.mule.execution.ResolvePreviousTransactionInterceptor.execute(ResolvePreviousTransactionInterceptor.java:44)
at org.mule.execution.SuspendXaTransactionInterceptor.execute(SuspendXaTransactionInterceptor.java:50)
at org.mule.execution.ValidateTransactionalStateInterceptor.execute(ValidateTransactionalStateInterceptor.java:40)
at org.mule.execution.IsolateCurrentTransactionInterceptor.execute(IsolateCurrentTransactionInterceptor.java:41)
at org.mule.execution.ExternalTransactionInterceptor.execute(ExternalTransactionInterceptor.java:48)
at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:28)
at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:13)
at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:110)
at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:30)
at org.mule.processor.AsyncInterceptingMessageProcessor$AsyncMessageProcessorWorker.doRun(AsyncInterceptingMessageProcessor.java:205)
at org.mule.work.AbstractMuleEventWork.run(AbstractMuleEventWork.java:53)
at org.mule.work.WorkerContext.run(WorkerContext.java:301)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
********************************************************************************
The problem here is not obvious, but I have come across the same issue before - it's related to the sizeOf function and the poor way that Mule applies precedence to some of it's operators. When you say (sizeOf payload.result > 0) it's first trying to attempt to resolve the payload.result > 0 expression - hence the error you're seeing (it's trying to compare an array to 0).
Please use ((sizeOf payload.result) > 0) instead (I always make a point of wrapping sizeOf in parentheses for this reason).
As a side note, you have batchid:payload.result[0].batchid - it should be batchId:payload.result[0].batchId (capitalisation in batchId)
whenever you are using any function like sizeOf in dataweave try to encapsulate it with round braces to avoid these kinds of errors.
#ghoshyTech in your case
when ((payload.result != null) and ((sizeOf payload.result) > 0))

MongoDB: How to deal with invalid ObjectIDs

Here below is my code to find a document by ObjectID:
def find(selector: JsValue, projection: Option[JsValue], sort: Option[JsValue],
page: Int, perPage: Int): Future[Seq[JsValue]] = {
var query = collection.genericQueryBuilder.query(selector).options(
QueryOpts(skipN = page * perPage)
)
projection.map(value => query = query.projection(value))
sort.map(value => query = query.sort(value.as[JsObject]))
// this is the line where the call crashes
query.cursor[JsValue].collect[Vector](perPage).transform(
success => success,
failure => failure match {
case e: LastError => DaoException(e.message, Some(DATABASE_ERROR))
}
)
}
Now let's suppose we invoke this method with an invalid ObjectID:
// ObjectId 53125e9c2004006d04b605abK is invalid (ends with a K)
find(Json.obj("_id" -> Json.obj("$oid" -> "53125e9c2004006d04b605abK")), None, None, 0, 25)
The call above causes the following exception when executing query.cursor[JsValue].collect[Vector](perPage) in the find method:
Caused by: java.util.NoSuchElementException: JsError.get
at play.api.libs.json.JsError.get(JsResult.scala:11) ~[play-json_2.10.jar:2.2.1]
at play.api.libs.json.JsError.get(JsResult.scala:10) ~[play-json_2.10.jar:2.2.1]
at play.modules.reactivemongo.json.collection.JSONGenericHandlers$StructureBufferWriter$.write(jsoncollection.scala:44) ~[play2-reactivemongo_2.10-0.10.2.jar:0.10.2]
at play.modules.reactivemongo.json.collection.JSONGenericHandlers$StructureBufferWriter$.write(jsoncollection.scala:42) ~[play2-reactivemongo_2.10-0.10.2.jar:0.10.2]
at reactivemongo.api.collections.GenericQueryBuilder$class.reactivemongo$api$collections$GenericQueryBuilder$$write(genericcollection.scala:323) ~[reactivemongo_2.10-0.10.0.jar:0.10.0]
at reactivemongo.api.collections.GenericQueryBuilder$class.cursor(genericcollection.scala:342) ~[reactivemongo_2.10-0.10.0.jar:0.10.0]
at play.modules.reactivemongo.json.collection.JSONQueryBuilder.cursor(jsoncollection.scala:110) ~[play2-reactivemongo_2.10-0.10.2.jar:0.10.2]
at reactivemongo.api.collections.GenericQueryBuilder$class.cursor(genericcollection.scala:331) ~[reactivemongo_2.10-0.10.0.jar:0.10.0]
at play.modules.reactivemongo.json.collection.JSONQueryBuilder.cursor(jsoncollection.scala:110) ~[play2-reactivemongo_2.10-0.10.2.jar:0.10.2]
at services.common.mongo.MongoDaoComponent$MongoDao$$anon$1.find(MongoDaoComponent.scala:249) ~[classes/:na]
... 25 common frames omitted
Any idea? Thanks.