Scala : Retry with try/catch for exception handling - json

I am trying to add a retry logic for JSON conversion. When converting an object to json, I am retrying for 3 times if there is any exception. I am doing :
var mapper = new ObjectMapper() with ScalaObjectMapper
intializeMapper( )
def intializeMapper() = {
// jackson library does not support seralization and deserialization of
// of scala classes like List and Map, this is needed to support it
mapper.registerModule( DefaultScalaModule )
// enables parsing of NaN. Enabling it here as JsonUtil class currently in
// use supports it.
mapper.configure(JsonParser.Feature.ALLOW_NON_NUMERIC_NUMBERS, true )
mapper.setSerializationInclusion(Include.NON_NULL)
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
}
def getPersonRDD(result: DataFrame): RDD[(String, String)] = {
val finalValue = result.rdd.map({
r =>
val customerId = r.getAs[String](CUSTOMER_ID)
val itemId = r.getAs[Map[String, Int]](ITEM_ID)
val itemName = r.getAs[Map[String, Int]](ITEM_NAME)
val person = Person(itemId, itemName)
val jsonString = toJson(person)
(customerId, jsonString)
})
return finalValue
}
def fromJson(json: String, clazz: Class[_]) = {
mapper.readValue(json, clazz)
}
def toJson(value: Any): String = {
var jsonString: String = " "
jsonString = mapper.writeValueAsString(value)
try {
fromJson(jsonString, clazz)
return jsonString
} catch {
case Exception => {
publishMetrics(PARSING_EXCEPTION, 1.0)
val result = util.Try(retry() {
jsonString = mapper.writeValueAsString(value)
val features = fromJson(jsonString, clazz)
})
result match {
case util.Success(value) => jsonString
case util.Failure(error) => {
log.error("Error while parsing JSON " + jsonString)
return jsonString
}
}
}
}
}
// Returning T, throwing the exception on failure
#annotation.tailrec
def retry[T](n: Int = 3)(fn: => T): T = {
util.Try {
fn
} match {
case util.Success(x) => x
case _ if n > 1 => retry(n - 1)(fn)
case util.Failure(e) => throw e
}
}
case class Person(itemId: Map[String, Int], itemName: Map[String, Int]) extends Serializable
Is this correct ? I am new to Scala. Can someone suggest me if there is any better way for achieving this ? Is there predefined retry logic available in Scala ? The reason I am trying to add retry logic for JSON conversion is due to Jackson version I use(which I can't change for now), sometimes my writeValueAsString results in incomplete JSON.

You retry function seems correct. The only flaw I can think of is that if you expect something would fail it's better just make the return type Try[T], so you can handle it outside in the scala way.
Here is one of my implementation:
def retry[T](n: Int)(block: => T): Try[T] = {
val stream = Stream.fill(n)(Try(block))
stream find (_.isSuccess) getOrElse stream.head
}

Related

How to convert Scala Document to JSON in Scala

I want to convert variable message which is of type scala.Seq[Scala.Document] to JSON format in following code:
path("getMessages"){
get {
parameters('roomname.as[String]) {
(roomname) =>
try {
val messagesByGroupName = MongoDatabase.collectionForChat.find(equal("groupChatName",roomname)).toFuture()
val messages = Await.result(messagesByGroupName,60.seconds)
println("Messages:"+messages)
complete(messages)
}
catch {
case e:TimeoutException =>
complete("Reading file timeout.")
}
}
}
But it is giving me error on complete(messages) line. It is not accepting message of that type.
I tried to convert it into JSON by using following :
import play.api.libs.json._
object MyJsonProtocol{
implicit object ChatFormat extends Format[Chat] {
def writes(c: Chat) : JsValue = {
val chatSeq = Seq (
"sender" -> JsString(c.sender),
"receiver" -> JsString(c.receiver),
"message" -> JsString(c.message),
"groupChatName" -> JsString(c.groupChatName),
)
JsObject(chatSeq)
}
def reads(value: JsValue) = {
JsSuccess(Chat("","","",""))
}
}
}
But it is not working.
My Chat.scala class is as follows:
import play.api.libs.json.{Json, Reads, Writes}
class Chat(var sender:String,var receiver:String,var message:String, var groupChatName:String){
def setSenderName(senderName:String) = {
sender = senderName
}
def setReceiverName(receiverName:String) = {
receiver = receiverName
}
def setMessage(getMessage:String) = {
message = getMessage
}
def setGroupChatName(chatName:String) = {
groupChatName = chatName
}
}
object Chat {
def apply(sender: String, receiver: String, message: String, groupname: String): Chat
= new Chat(sender, receiver, message,groupname)
def unapply(arg: Chat): Option[(String, String, String,String)] = ???
implicit val requestReads: Reads[Chat] = Json.reads[Chat]
implicit val requestWrites: Writes[Chat] = Json.writes[Chat]
}
I am also not able to figure out what to write in unapply method.
I am new to scala and akka.
EDIT:
My MongoDatabase.scala which has collection is as follows:
object MongoDatabase {
val chatCodecProvider = Macros.createCodecProvider[Chat]()
val codecRegistry = CodecRegistries.fromRegistries(
CodecRegistries.fromProviders(chatCodecProvider),
DEFAULT_CODEC_REGISTRY
)
implicit val system = ActorSystem("Scala_jwt-App")
implicit val executor: ExecutionContext = system.dispatcher
val mongoClient: MongoClient = MongoClient()
val databaseName = sys.env("database_name")
// Getting mongodb database
val database: MongoDatabase = mongoClient.getDatabase(databaseName).withCodecRegistry(codecRegistry)
val registrationCollection = sys.env("register_collection_name")
val chatCollection = sys.env("chat_collection")
// Getting mongodb collection
val collectionForUserRegistration: MongoCollection[Document] = database.getCollection(registrationCollection)
collectionForUserRegistration.drop()
val collectionForChat: MongoCollection[Document] = database.getCollection(chatCollection)
collectionForChat.drop()
}
And if try to change val collectionForChat: MongoCollection[Document] = database.getCollection(chatCollection)
to
val collectionForChat: MongoCollection[Chat] = database.getCollection[Chat](chatCollection)
then I get error on in saveChatMessage() method below:
def saveChatMessage(sendMessageRequest: Chat) : String = {
val senderToReceiverMessage : Document = Document(
"sender" -> sendMessageRequest.sender,
"receiver" -> sendMessageRequest.receiver,
"message" -> sendMessageRequest.message,
"groupChatName" -> sendMessageRequest.groupChatName)
val chatAddedFuture = MongoDatabase.collectionForChat.insertOne(senderToReceiverMessage).toFuture()
Await.result(chatAddedFuture,60.seconds)
"Message sent"
}
on val chatAddedFuture = MongoDatabase.collectionForChat.insertOne(senderToReceiverMessage).toFuture() this line since it accepts data of type Seq[Document] and I am trying to add data of type Seq[Chat]
I am going to assume that MongoDatabase.collectionForChat.find(equal("groupChatName",roomname)) returns either Seq[Chat], or Chat. Both of them are the same for play.
You have 2 options:
Adding the default format on the companion object:
object Chat {
implicit val format: Format[Chat] = Json.format[Chat]
}
In this case you can delete the object MyJsonProtocol which is not used.
In case you want to keep your own serializers(i.e. MyJsonProtocol), you need to rename MyJsonProtocol into Chat. This way the complete route will be able to find the implicit Format.
create case class for the message object you want to send
for example:
case class MyMessage(sender: String, receiver: String, message: String, groupChatName: String)
You should create Format for the type of case class
implicit val MessageTypeFormat = Json.format[MyMessage]
if complete should get JSON type - then call complete myMessage when myMessage is an instance of MyMessage.
complete(Json.toJson(myMessage))

Jackson: (de)serialising Map with arbitrary non-string key

I have a Scala Map keyed by a type that itself needs serialising to JSON. Because of the nature of JSON that requires key names for objects to be strings, a simple mapping is not directly possible.
The work around I wish to implement is to convert the Map to a Set before serialising to JSON, and then from a Set back to a Map after deserialising.
I am aware of other methods using key serialisers on specific types, e.g. Serialising Map with Jackson, however, I require a solution that applies to arbitrary key types and in this regard, conversion to Set and back again looks to me like the best option.
I've had some success serialising to a Set with a wrapper object by modifying MapSerializerModule.scala from jackson-module-scala below, but I'm not familiar enough with the Jackson internals to get that JSON to deserialise back into the Map I started with.
I should add that I control both the serialisation and deserialisation side, so what the JSON looks like is not significant.
case class Wrapper[K, V](
value: Set[(K, V)]
)
class MapConverter[K, V](inputType: JavaType, config: SerializationConfig)
extends StdConverter[Map[K, V], Wrapper[K, V]] {
def convert(value: Map[K, V]): Wrapper[K, V] = {
val set = value.toSet
Wrapper(set)
}
override def getInputType(factory: TypeFactory) = inputType
override def getOutputType(factory: TypeFactory) =
factory.constructReferenceType(classOf[Wrapper[_, _]], inputType.getContentType)
.withTypeHandler(inputType.getTypeHandler)
.withValueHandler(inputType.getValueHandler)
}
object MapSerializerResolver extends Serializers.Base {
val MAP = classOf[Map[_, _]]
override def findMapLikeSerializer(
config: SerializationConfig,
typ: MapLikeType,
beanDesc: BeanDescription,
keySerializer: JsonSerializer[AnyRef],
elementTypeSerializer: TypeSerializer,
elementValueSerializer: JsonSerializer[AnyRef]): JsonSerializer[_] = {
val rawClass = typ.getRawClass
if (!MAP.isAssignableFrom(rawClass)) null
else new StdDelegatingSerializer(new MapConverter(typ, config))
}
}
object Main {
def main(args: Array[String]): Unit = {
val objMap = Map(
new Key("k1", "k2") -> "k1k2",
new Key("k2", "k3") -> "k2k3")
val om = new ObjectMapper()
om.registerModule(DefaultScalaModule)
om.registerModule(ConverterModule)
val res = om.writeValueAsString(objMap)
println(res)
}
}
I managed to find the solution:
case class MapWrapper[K, V](
wrappedMap: Set[MapEntry[K, V]]
)
case class MapEntry[K, V](
key: K,
value: V
)
object MapConverter extends SimpleModule {
addSerializer(classOf[Map[_, _]], new StdDelegatingSerializer(new StdConverter[Map[_, _], MapWrapper[_, _]] {
def convert(inMap: Map[_, _]): MapWrapper[_, _] = MapWrapper(inMap map { case (k, v) => MapEntry(k, v) } toSet)
}))
addDeserializer(classOf[Map[_, _]], new StdDelegatingDeserializer(new StdConverter[MapWrapper[_, _], Map[_, _]] {
def convert(mapWrapper: MapWrapper[_, _]): Map[_, _] = mapWrapper.wrappedMap map { case MapEntry(k, v) => (k, v) } toMap
}))
}
class MapKey(
val k1: String,
val k2: String
) {
override def toString: String = s"MapKey($k1, $k2) (str)"
}
object Main {
def main(args: Array[String]): Unit = {
val objMap = Map(
new MapKey("k1", "k2") -> "k1k2",
new MapKey("k2", "k3") -> "k2k3")
val om = setupObjectMapper
val jsonMap = om.writeValueAsString(objMap)
val deserMap = om.readValue(jsonMap, classOf[Map[_, _]])
}
private def setupObjectMapper = {
val typeResolverBuilder =
new DefaultTypeResolverBuilder(ObjectMapper.DefaultTyping.NON_FINAL) {
init(JsonTypeInfo.Id.CLASS, null)
inclusion(JsonTypeInfo.As.WRAPPER_OBJECT)
typeProperty("#CLASS")
override def useForType(t: JavaType): Boolean = !t.isContainerType && super.useForType(t)
}
val om = new ObjectMapper()
om.registerModule(DefaultScalaModule)
om.registerModule(MapConverter)
om.setDefaultTyping(typeResolverBuilder)
om
}
}
Interestingly, if the key type is a case class, the MapConverter is not necessary since the case class can be reconstituted from the string representation.
In my case, I had a nested Map. This required a small addition to the conversion back into a map:
addDeserializer(classOf[Map[_, _]], new StdDelegatingDeserializer(new StdConverter[MapWrapper[_, _], Map[_, _]] {
def convert(mapWrapper: MapWrapper[_, _]): Map[_, _] = {
mapWrapper.wrappedMap.map { case MapEntry(k, v) => {
v match {
case wm: MapWrapper[_, _] => (k, convert(wm))
case _ => (k, v)
}
}}.toMap
}
}))

Play ws how to map response in typesafe manner

How can I map the JSON response of a play-ws async web request in a typesafe manner?
private def webRequest(): Map[String, Any] = {
case class QuantileResult(val_05: Double, val_25: Double, val_50: Double, val_75: Double, val_90: Double)
object QuantileResult {
implicit val quantileResultFormat = Json.format[QuantileResult]
}
implicit val quantileReads = Json.reads[QuantileResult]
val payload = Json.obj(
"json" -> Json.parse("""{"type":1,"x":[1,2,3,4,5,6,7,8,9,10],"probs":[0.05,0.25,0.75,0.95]}"""),
"windowWidth" -> JsNumber(11)
)
wsClient.url("http://public.opencpu.org/ocpu/library/stats/R/quantile/json")
.withHeaders("Content-Type" -> "application/json")
.post(payload)
.map { wsResponse =>
if (!(200 to 299).contains(wsResponse.status)) {
sys.error(s"Received unexpected status, open-cpu error ${wsResponse.status} : ${wsResponse.body}")
}
println(s"OK, received ${wsResponse.body}")
wsResponse.json.validate[QuantileResult]
match {
case JsSuccess(arrOut: QuantileResult, _) => QuantileResult //TODO how to perform mapping here to Map[String, Any]
case e: JsError => JsError.toFlatForm(e) // TODO empty array does not work here otherwise future[object] does not conform to expected return type of Map[String, Any]
}
}
}
how to perform mapping here to Map[String, Any]
empty map does not work here otherwise future[object] does not conform to expected return type of Map[String, Any]
What works is
wsClient.url("url").withHeaders("Content-Type" -> "application/json").post(payload).map { wsResponse => result = wsResponse.body }
val json = Json.parse(result)
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
val returnValue = mapper.readValue[Map[String, Any]](result)
But that seems to be really clunky and is not async.
The solution is to change the method signature to a proper future Future[Seq[Map[String, Any]]]and resolve it via Await.result(someMethod), 20.seconds)
The mapping is performed like
wsResponse.json.validate[Seq[OutlierResult]] match {
case JsSuccess(result, _) => result.map(outlierRes => Map("period" -> outlierRes.period, "amount" -> outlierRes.amount, "outlier" -> outlierRes.outlier))
case JsError(error) => throw new OutlierParseException(error.toString())
}
even though this could be improved to not throw an Exception

Try to update\fetch Postgres json column into JsValue using anorm [duplicate]

I am using anorm to query and save elements into my postgres database.
I have a json column which I want to read as class of my own.
So for example if I have the following class
case class Table(id: Long, name:String, myJsonColumn:Option[MyClass])
case class MyClass(site: Option[String], user:Option[String])
I am trying to write the following update:
DB.withConnection { implicit conn =>
val updated = SQL(
"""UPDATE employee
|SET name = {name}, my_json_column = {myClass}
|WHERE id = {id}
""".stripMargin)
.on(
'name -> name,
'myClass -> myClass,
'custom -> id
).executeUpdate()
}
}
I also defined a implicit convertor from json to my object
implicit def columnToSocialData: Column[MyClass] = anorm.Column.nonNull[MyClass] { (value, meta) =>
val MetaDataItem(qualified, nullable, clazz) = meta
value match {
case json: org.postgresql.util.PGobject => {
val result = Json.fromJson[MyClass](Json.parse(json.getValue))
result.fold(
errors => Left(TypeDoesNotMatch(s"Cannot convert $value: ${value.asInstanceOf[AnyRef].getClass} to Json for column $qualified")),
valid => Right(valid)
)
}
case _ => Left(TypeDoesNotMatch(s"Cannot convert $value: ${value.asInstanceOf[AnyRef].getClass} to Json for column $qualified"))
}
And the error I get is:
type mismatch;
found : (Symbol, Option[com.MyClass])
required: anorm.NamedParameter
'myClass -> myClass,
^
The solution is just to add the following:
implicit val socialDataToStatement = new ToStatement[MyClass] {
def set(s: PreparedStatement, i: Int, myClass: MyClass): Unit = {
val jsonObject = new org.postgresql.util.PGobject()
jsonObject.setType("json")
jsonObject.setValue(Json.stringify(Json.toJson(myClass)))
s.setObject(i, jsonObject)
}
}
and:
implicit object MyClassMetaData extends ParameterMetaData[MyClass] {
val sqlType = "OTHER"
val jdbcType = Types.OTHER
}

How to add an additional json item per object in scala

I'm writing a simple scala application that opens a flat file of json data, parses it and finally prints it out to the screen.
The next step will require that I stop at each object and add another item (string) to the front of it. My question is how can I add a new string per object in this list?
The following is my JSON implementation (credit goes to the init author here)
import scala.util.parsing.combinator._
class JSON extends JavaTokenParsers {
def obj: Parser[Map[String, Any]] =
"{"~> repsep(member, ",") <~"}" ^^ (Map() ++ _)
def arr: Parser[List[Any]] =
"["~> repsep(value, ",") <~"]"
def member: Parser[(String, Any)] =
stringLiteral~":"~value ^^
{ case name~":"~value => (name, value) }
def value: Parser[Any] = (
obj
| arr
| stringLiteral
| floatingPointNumber ^^ (_.toInt)
| "null" ^^ (x => null)
| "true" ^^ (x => true)
| "false" ^^ (x => false)
)
}
Next I call this w/ a flat file like so
import java.io.FileReader
import scala23.JSON
class JSONTest extends JSON {
def main(args: String) {
val reader = new FileReader(args)
println(parseAll(value, reader))
}
}
Then I get a valid println of the json contents. Instead I would like to pass this parse method a String and have it append it or create a new json object that has the string at the front of each object inside
Update
My current attempt looks something like the below
class JSONTest extends JSON {
def main(args: String) {
val reader = new FileReader(args)
val header = ("abc", "def")
// println(parseAll(value, reader).map(addHeader(_, header)))
println(parseAll(value, reader).map(addHeader(_.asInstanceOf[Map[String, Any]], header)))
}
def addHeader(xyz:Map[String, Any], header:(String, Any)):Map[String, Any] = {
xyz.map {
case (k, m:Map[String, Any]) => (k, addHeader(m))
case e => e
} + header
}
}
But I'm currently getting a few errors in Intellij
error: missing parameter type for expanded function ((x$1) => x$1.asInstanceOf[Map[String, Any]])
println(parseAll(value, reader).map(addHeader(_.asInstanceOf[Map[String, Any]], header)))
AND
error: not enough arguments for method addHeader: (xyz: Map[String,Any],header: (String, Any))Map[String,Any].
Unspecified value parameter header.
case (k, m:Map[String, Any]) => (k, addHeader(m))
Any help would be much appreciated (thank you in advance!)
Have you tried using map on the parser output instead.
Edit: this compiles on my machine
import java.io.FileReader
import scala23.JSON
class JSONTest extends JSON {
def main(args: String) {
val reader = new FileReader(args)
val header = ("abc", "def")
// println(parseAll(value, reader).map(addHeader(_, header)))
println(parseAll(value, reader).map(addHeader(_, header)))
}
def addHeader(xyz:Any, header:(String, Any)):Any = xyz match {
case obj:Map[String, Any] => obj.map {
case (k, m:Map[String, Any]) => (k, addHeader(m, header))
case e => e
} + header
case arr:List[Any] => arr.map(addHeader(_, header))
case e => e
}
}
It should be handling the varied output of the parse better.