So I have a list of maps with dynamic field and values.
E.g.
val sampleMap = List(
Map("field1" -> 1, "field2" -> "helloworld"),
Map("field3" -> "abcd", "field4" -> 123.34212543))
So basically, I have a variable that is List[Map[String, Object]].
How can I convert this whole thing into a JSON circe?
Nevermind, I found the answer.
Basically we need to match the datatype of the object and convert it to JSON.
def toJson(obj: Object): Json = {
obj match {
case null => Json.Null
case b: java.lang.Boolean => Json.fromBoolean(b)
case i: java.lang.Integer => Json.fromInt(i)
case d: java.lang.Double =>
Json.fromDouble(d).getOrElse(Json.fromDouble(0d).get)
case l: java.lang.Long => Json.fromLong(l)
case t: TemporalAccessor =>
Json.fromString(DtFormatter.format(t))
case u => Json.fromString(String.valueOf(u))
}
}
Related
I have some json when I parsed that it is returning stucture Some(List(Map...))
How to match with something and get the values
Below is the code I tried, I need to get all the map values
import scala.util.parsing.json._
val result = JSON.parseFull("[{\"start\":\"starting\",\"test\":123,\"test2\":324,\"end\":\"ending\"}]")
result match {
case Some(map: Map[String, Any]) => { println(map)
}
case None => println("Parsing failed")
case other => println("Unknown data structure: " + other)
}
but its printing non matching
Unknown data structure: Some(List(Map(start -> starting, test -> 123, test2 -> 324, end -> ending)))
Because of type erasure, you cannot pattern match on generics types. List, Map, Option are generic containers, and in runtime compiler will erase types of these generic containers. e.g. List[String], String will be erased and type will be List[_].
case Some(map: List[Map[String, Any]]) => println(map)
In your case above case, if result is val result: Option[Any] = Some(List(12)) i.e. 12, which type is Int and not Map[String, Any], compiler will still match the result with above case even it expects Map[String, Any]] as type of List.
So, Whats going on?
Its all because of type erasure. The compiler will erase all the types and it won't have any type information at runtime unless you use reflection. That means:
case Some(map: List[Map[String, Any]]) => println(map) is essentially case Some(map: List[_]) => println(map) and therefore, match will success for any type parameter of List e.g. List[Map[String, Any]], List[Map[String, Int]], List[String], List[Int] etc.
Therefore, In case you need to match on such generic container, you have to resolve each container and its nested subtypes explicitly.
def resolve(result: Any): Unit = result match {
case Some(res) if res.isInstanceOf[List[_]] && res.asInstanceOf[List[_]].isEmpty => println("Empty List.") //Some(List())
case Some(res) if res.isInstanceOf[List[_]] && !res.asInstanceOf[List[_]].exists(p => p.isInstanceOf[Map[_, _]] && p.asInstanceOf[Map[_, _]].nonEmpty) => println("List is not empty but each item of List is empty Map.") //Some(List(Map(), Map()))
case Some(res) if res.isInstanceOf[List[_]] && res.asInstanceOf[List[_]].filter(p => p.isInstanceOf[Map[_, _]] && p.asInstanceOf[Map[_, _]].nonEmpty).map(_.asInstanceOf[Map[_,_]]).exists(p => {p.head match {case e if e._1.isInstanceOf[String] && e._2.isInstanceOf[Any] => true; case _ => false}}) => println("Correct data.") // Some(List(Map("key1"-> 1), Map("key2" -> 2)))
case None => println("none")
case other => println("other")
}
val a: Option[Any] = Some(List())
val b: Option[Any] = Some(List(Map(), Map()))
val c: Option[Any] = Some(List(Map("key1"-> 1), Map("key2" -> 2)))
val d: Option[Any] = None
val e: Option[Any] = Some("apple")
resolve(a) // match first case
resolve(b) // match second case
resolve(c) // match third case
resolve(d) // match fourth case
resolve(e) // match fifth case
As has been pointed out the return type is actually Option[List[Map[String, Any]]] so you need to unpick this. However you cannot do this with a single match because of type erasure, so you need to do nested matches to ensure that you have the correct type. This is really tedious, so I thoroughly recommend using something like the Extraction.extract function in json4s that will attempt to match your JSON to a specific Scala type:
type ResultType = List[Map[String, Any]]
def extract(json: JValue)(implicit formats: Formats, mf: Manifest[ResultType]): ResultType =
Extraction.extract[ResultType](json)
If you must do it by hand, it looks something like this:
result match {
case Some(l: List[_]) =>
l.headOption match {
case Some(m) =>
m match {
case m: Map[_,_] =>
m.headOption match {
case Some(p) =>
p match {
case (_: String, _) =>
m.foreach(println(_))
case _ => println("Map key was not String")
}
case _ => println("Map was empty")
}
case _ => println("List did not contain a Map")
}
case _ => println("Result List was empty")
}
case _ => println("Parsing failed")
}
Your output is Option[List[Map[String, Any]]], not Option[Map[String, Any]]. match on the List and you'll be fine:
import scala.util.parsing.json._
val result = JSON.parseFull("[{\"start\":\"starting\",\"test\":123,\"test2\":324,\"end\":\"ending\"}]")
val l: List[Map[String, Any]] = result match {
case Some(list: List[Map[String, Any]]) => list
case _ => throw new Exception("I shouldn't be here") // whatever for a non-match
}
Then you can map (if you want a non-Unit return type)/ foreach (if you don't care about a Unit return type) on that List and do whatever you want it:
l.foreach(println)
l.map(_.toString) // or whatever you want ot do with the Map
I am new to scala and I am trying to parse a JSON shown below
val result = JSON.parseFull("""
{"name": "Naoki", "lang": ["Java", "Scala"] , "positionandvalue": ["5:21", "6:24", "7:6"]}
""")
result: Option[Any] = Some(Map(name -> Naoki, lang -> List(Java, Scala), positionandvalue -> List(5:21, 6:24, 7:6)))
And get the parsed values in a Map
val myMap = result match {
case Some(e) => e
case None => None
}
myMap: Any = Map(name -> Naoki, lang -> List(Java, Scala), positionandvalue -> List(5:21, 6:24, 7:6))
What I need is
1. To get the key as a new variable (to be used as metadata to validate the file) with its corresponding value assigned to it. Something like,
val name = "Naoki"
positionandvalue -> List(5:21, 6:24, 7:6). This variable indicates the List of(Position of string delimited a in file:length of string in position). How can I use this variable to satisfy the requirement.
you cannot dynamically create the variables name and positionandvalue from the Map key. However they can be statically created using the below approach.
val result: Option[Any] = Some(Map("name" -> "Naoki", "lang" -> List("Java", "Scala"), "positionandvalue" -> List("5:21", "6:24", "7:6")))
val myMap: Map[String, Any] = result match {
case Some(e: Map[String, Any] #unchecked) => e
case _ => Map()
}
val name = myMap.get("name") match {
case Some(x: String) => x
case _ => throw new RuntimeException("failure retrieving name key")
}
val positionandvalue = myMap.get("positionandvalue") match {
case Some(x: List[String] #unchecked) => x.map(y => (y.split(":") match {case Array(x1,x2) => x1 -> x2})).toMap
case _ => throw new RuntimeException("failure retrieving positionandvalue key")
}
positionandvalue: scala.collection.immutable.Map[String,String] = Map(5 -> 21, 6 -> 24, 7 -> 6)
I am building a web app using Scala / Play Framework and Reactive Mongo and I want the models to be defined in the database instead of having them hardcoded.
To do so, I am writing a class EntityInstance taking a Sequence of FieldInstance :
case class EntityInstance(fields: Seq[FieldInstance])
I am trying to accept fields from any types and to convert them to Json : example
new FieldInstance("name", "John") | json: { "name": "John" }
new FieldInstance("age", 18) | json: { "age": 18 }
At the moment I am trying to accept Strings, Booleans and Integers and if the type is not supported I write some error :
new FieldInstance("profilePicture", new Picture("john.jpg") | json: { "profilePicture": "Unsupported type
I wrote a FieldInstance class taking a fieldName as a String and a value as any type. As soon as that class is instantiated I cast the value to a known type or to the String describing the error.
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec match {
case v: Int => v
case v: String => v
case v: Boolean => v
case _ => "Unrecognized type"
}
}
object FieldInstance {
implicit val fieldInstanceWrites = new Writes[FieldInstance] {
def writes(fieldInstance: FieldInstance) = Json.obj(
fieldInstance.fieldName -> fieldInstance.value
)
}
}
I created a companion object with an implicit Write to json so I can call "Json.toJson()" on an instance of FieldInstance and get a json as described on my examples above.
I get an error : found: Any required: play.api.libs.json.Json.JsValueWrapper
I understand that it comes from the fact that my value is of type Any but I thought the cast would change that Any to String || Boolean || Int before hitting the Writer.
PS: Ignore the bad naming of the classes, I could not name EntityInstance and FieldInstance, Entity and Field because these as the classes I use to describe my models.
I found a fix to my problem :
The type matching that I was doing in the class should be done in the implicit Write !
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec
override def toString(): String = "(" + fieldName + "," + value + ")";
}
object FieldInstance {
implicit val fieldInstanceWrites = new Writes[FieldInstance] {
def writes(fieldInstance: FieldInstance) =
fieldInstance.value match {
case v: Int => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[Int])
case v: String => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[String])
case v: Boolean => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[Boolean])
case _ => Json.obj(fieldInstance.fieldName -> "Unsupported type")
}
}
}
This code now allows a user to create an EntityInstance with Fields of Any type :
val ei = new EntityInstance(Seq[FieldInstance](new FieldInstance("name", "George"), new FieldInstance("age", 25), new FieldInstance("married", true)))
println("-- TEST ENTITY INSTANCE TO JSON --")
println(Json.toJson(ei))
prints : {"entity":[{"name":"George"},{"age":25},{"married":true}]}
Here is my EntityInstance code if you are trying to test it :
case class EntityInstance(fields: Seq[FieldInstance])
object EntityInstance {
implicit val EntityInstanceWrites = new Writes[EntityInstance] {
def writes(entityInstance: EntityInstance) =
Json.obj("entity" -> entityInstance.fields)
}
}
It is returning a String, Int or Boolean but Json.obj is expecting the value parameter of type (String, JsValueWrapper)
def obj(fields: (String, JsValueWrapper)*): JsObject = JsObject(fields.map(f => (f._1, f._2.asInstanceOf[JsValueWrapperImpl].field)))
a quick fix could be to convert the matched value v with toJson provided the implicit Writes[T] for type T is available (which they are for String, Int and Boolean)
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec match {
case v: Int => Json.toJson(v)
case v: String => Json.toJson(v)
case v: Boolean => Json.toJson(v)
case _ => Json.toJson("Unrecognized type")
}
}
If you'd like to see which DefaultWrites are available you can browse them in the play.api.libs.json package in trait DefaultWrites
for example:
/**
* Serializer for Boolean types.
*/
implicit object BooleanWrites extends Writes[Boolean] {
def writes(o: Boolean) = JsBoolean(o)
}
I need to handle a case that hasn't been answered by this question: Convert any Scala object to JSON
Basically, I don't know how to handle List and Map types. I've tried the following which doesn't work:
implicit val anyValWriter = Writes[Any] (a =>
a match {
case v: Double => Json.toJson(v)
case v: Float => Json.toJson(v)
case v: Long => Json.toJson(v)
case v: Int => Json.toJson(v)
case v: String => Json.toJson(v)
case v: Iterable[Any] => Json.toJson(v.map(t => Json.toJson(t)).toList)
case v: Map[String, Any] => JsObject(v.map { case (k, v) => (k, Json.toJson(v)) }.toList)
// or, if you don't care about the value
case _ => throw new RuntimeException("Type not serializable.")
})
The resulting error is: No Json serializer found for type Any. Try to implement an implicit Writes or Format for this type.
Trying to add anyValWriter to Json.toJson results in: recursive value anyValWriter needs type
Any idea?
I finally sorted my problem out through a workaround. I've chosen not to handle Iterable & Map types in the Writes[Any] and I used a wrapper to handle both types as in my case Any won't be an occurrence of either List[...] or Map[String, Any].
Here's the code for those who are interested:
implicit val anyValWriter = Writes[Any] (a =>
a match {
case v: Double => Json.toJson(v)
case v: Float => Json.toJson(v)
case v: Long => Json.toJson(v)
case v: Int => Json.toJson(v)
case v: String => Json.toJson(v)
case v: JsArray => v
// or, if you don't care about the value
case _ => throw new RuntimeException("Type not serializable.")
})
And the wrapper:
def convertToJsonValue(v : Iterable[Map[String, Any]])(implicit write: Writes[Any]) = {
JsArray(v.map(t => JsObject(t.map {
case (k, value) => (k, Json.toJson(value)(write)) }.toList)).toList)
}
I'll find out later if I can sort out the ugly JsArray case.
PS: I know it's wrong but I'm using a SQL API which returns Map[String, Any] as I don't know the columns that will be returned (and I have good reasons for this and to not use the ORM feature).
I'm reading data from a KV store (Redis) in this case. The data returned is in the following format.
{ "key1":"value1", "key2":"value2", "key3":"value3" ...}
Key is String and value is Int. I want to convert it into a Map[String,Int]
I looked at the json4s JSON API and my current code looks like the following. Is there a better/easier/cleaner way of doing this?
//send a async query to Redis to
val queryFuture = redis.zrangebyscore[String](tablename, keymin, keymax )
queryFuture.onComplete {
case Success(datarows) =>
println(s"Got %d rows of type %s for the query successfully".format(datarows.length))
val jsonrows = for { x <- datarows.toList }
yield parse(x)
println("Got json rows %d".format(jsonrows.size))
val mapdata = jsonrows.map(x => x.extract[Map[String,String]]).map( x => x.mapValues(_.toInt))
//need to do something with this data now
case Failure(y) =>
println(s" there was some failure in getting the data from Redis")
}
This looks to me like the simplest way to do it:
val map = parse("""{"a":"1","b":"2","c":"3"}""")
.children
.collect { case JField(k, JString(v)) => (k, v.toInt) }
.toMap
Your Json4s solution looks fine. Alternatively you can use mapField to transform the fields of a JObject and after that extract value of type Map[String, Int].
val json1 = parse(
"""
|{
| "key1": "1024",
| "key2": "2048",
| "key3": "4096"
|}
""".stripMargin)
val json2 = json1.mapField {
case (key, JString(value)) => (key, JInt(value.toInt))
case x => x
}
val res = json2.extract[Map[String, Int]]
println(res)
// Map(key1 -> 1024, key2 -> 2048, key3 -> 4096)
Not knowing json4s, and unfortunately you ommited the types, but guessing that jsonrows is probably something like a List[(String, String)] you could do
List(("key1" -> "1"),("key2" -> "2")).map { case (k, v) => (k, v.toInt)}.toMap
BTW, if you say need to do something with this data now in your onComplete - that could only be a side effecting operation. Better map over the future until your processing is complete.