I need to handle a case that hasn't been answered by this question: Convert any Scala object to JSON
Basically, I don't know how to handle List and Map types. I've tried the following which doesn't work:
implicit val anyValWriter = Writes[Any] (a =>
a match {
case v: Double => Json.toJson(v)
case v: Float => Json.toJson(v)
case v: Long => Json.toJson(v)
case v: Int => Json.toJson(v)
case v: String => Json.toJson(v)
case v: Iterable[Any] => Json.toJson(v.map(t => Json.toJson(t)).toList)
case v: Map[String, Any] => JsObject(v.map { case (k, v) => (k, Json.toJson(v)) }.toList)
// or, if you don't care about the value
case _ => throw new RuntimeException("Type not serializable.")
})
The resulting error is: No Json serializer found for type Any. Try to implement an implicit Writes or Format for this type.
Trying to add anyValWriter to Json.toJson results in: recursive value anyValWriter needs type
Any idea?
I finally sorted my problem out through a workaround. I've chosen not to handle Iterable & Map types in the Writes[Any] and I used a wrapper to handle both types as in my case Any won't be an occurrence of either List[...] or Map[String, Any].
Here's the code for those who are interested:
implicit val anyValWriter = Writes[Any] (a =>
a match {
case v: Double => Json.toJson(v)
case v: Float => Json.toJson(v)
case v: Long => Json.toJson(v)
case v: Int => Json.toJson(v)
case v: String => Json.toJson(v)
case v: JsArray => v
// or, if you don't care about the value
case _ => throw new RuntimeException("Type not serializable.")
})
And the wrapper:
def convertToJsonValue(v : Iterable[Map[String, Any]])(implicit write: Writes[Any]) = {
JsArray(v.map(t => JsObject(t.map {
case (k, value) => (k, Json.toJson(value)(write)) }.toList)).toList)
}
I'll find out later if I can sort out the ugly JsArray case.
PS: I know it's wrong but I'm using a SQL API which returns Map[String, Any] as I don't know the columns that will be returned (and I have good reasons for this and to not use the ORM feature).
Related
So I have a list of maps with dynamic field and values.
E.g.
val sampleMap = List(
Map("field1" -> 1, "field2" -> "helloworld"),
Map("field3" -> "abcd", "field4" -> 123.34212543))
So basically, I have a variable that is List[Map[String, Object]].
How can I convert this whole thing into a JSON circe?
Nevermind, I found the answer.
Basically we need to match the datatype of the object and convert it to JSON.
def toJson(obj: Object): Json = {
obj match {
case null => Json.Null
case b: java.lang.Boolean => Json.fromBoolean(b)
case i: java.lang.Integer => Json.fromInt(i)
case d: java.lang.Double =>
Json.fromDouble(d).getOrElse(Json.fromDouble(0d).get)
case l: java.lang.Long => Json.fromLong(l)
case t: TemporalAccessor =>
Json.fromString(DtFormatter.format(t))
case u => Json.fromString(String.valueOf(u))
}
}
Hi guys I have some odd json and need to convert it to type that will allow spark parse it. So I decided to use jackson lib to load it as value. So the jackson creates for me something Like this:
Map(A-> Map(B-> Map(C-> C, CC-> CC, CCC-> CCC, CCCC-> , CCCC-> Map(D-> ....
Now what I need to do is convert whole structure that each map will be wrapped to List :
Map(A-> List(Map(B-> List(Map(C-> C, CC-> CC, CCC-> CCC, CCCC-> CCCC) List(Map(D-> ....
I have tried something like this but this only helps me only for first value. Also I need to check if this is a Map of cause sometimes it could be List which I don't want to wrap
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
val parsedJson = mapper.readValue[Map[String, Object]](j)
val b = parsedJson.map {
case (k,v) if v.isInstanceOf[Map[String, Object]] => (k,List(v))
}
Have you guys any idea how I can do this recursively that afterall my Json will be look like this:
{
"A":[{
"B":[{
"C":"C",
"CC":"CC",
"CCC":"CCC"
}]
}]
}
{ //and not like that
"A":{
"B":{
"C":"C",
"CC":"CC",
"CCC":"CCC"
}
}
}
ANSWER If we wan't to map List(Map(k -> Map(k -> v))) this is my solution with #Dima help
def wrapIt(m: Map[String, Any]): Map[String, Any] = m.map {
case (k, v: Map[String, Any]) => k -> List(wrapIt(v))
case (k, v: List[Map[String, Any]]) => k -> v.map(m => m.map {
case (l, p: Map[String, Any]) => l -> List(wrapIt(p))
case lp => lp
})
case kv => kv
}
You forgot to do the recursive part :)
def wrapIt(m: Map[String, Any]): Map[String, Any] = m.map {
case (k, v: Map[String, Any]) => k -> List(wrapIt(v))
case kv => kv // ^^^^^^^ - magic!
}
I have some json when I parsed that it is returning stucture Some(List(Map...))
How to match with something and get the values
Below is the code I tried, I need to get all the map values
import scala.util.parsing.json._
val result = JSON.parseFull("[{\"start\":\"starting\",\"test\":123,\"test2\":324,\"end\":\"ending\"}]")
result match {
case Some(map: Map[String, Any]) => { println(map)
}
case None => println("Parsing failed")
case other => println("Unknown data structure: " + other)
}
but its printing non matching
Unknown data structure: Some(List(Map(start -> starting, test -> 123, test2 -> 324, end -> ending)))
Because of type erasure, you cannot pattern match on generics types. List, Map, Option are generic containers, and in runtime compiler will erase types of these generic containers. e.g. List[String], String will be erased and type will be List[_].
case Some(map: List[Map[String, Any]]) => println(map)
In your case above case, if result is val result: Option[Any] = Some(List(12)) i.e. 12, which type is Int and not Map[String, Any], compiler will still match the result with above case even it expects Map[String, Any]] as type of List.
So, Whats going on?
Its all because of type erasure. The compiler will erase all the types and it won't have any type information at runtime unless you use reflection. That means:
case Some(map: List[Map[String, Any]]) => println(map) is essentially case Some(map: List[_]) => println(map) and therefore, match will success for any type parameter of List e.g. List[Map[String, Any]], List[Map[String, Int]], List[String], List[Int] etc.
Therefore, In case you need to match on such generic container, you have to resolve each container and its nested subtypes explicitly.
def resolve(result: Any): Unit = result match {
case Some(res) if res.isInstanceOf[List[_]] && res.asInstanceOf[List[_]].isEmpty => println("Empty List.") //Some(List())
case Some(res) if res.isInstanceOf[List[_]] && !res.asInstanceOf[List[_]].exists(p => p.isInstanceOf[Map[_, _]] && p.asInstanceOf[Map[_, _]].nonEmpty) => println("List is not empty but each item of List is empty Map.") //Some(List(Map(), Map()))
case Some(res) if res.isInstanceOf[List[_]] && res.asInstanceOf[List[_]].filter(p => p.isInstanceOf[Map[_, _]] && p.asInstanceOf[Map[_, _]].nonEmpty).map(_.asInstanceOf[Map[_,_]]).exists(p => {p.head match {case e if e._1.isInstanceOf[String] && e._2.isInstanceOf[Any] => true; case _ => false}}) => println("Correct data.") // Some(List(Map("key1"-> 1), Map("key2" -> 2)))
case None => println("none")
case other => println("other")
}
val a: Option[Any] = Some(List())
val b: Option[Any] = Some(List(Map(), Map()))
val c: Option[Any] = Some(List(Map("key1"-> 1), Map("key2" -> 2)))
val d: Option[Any] = None
val e: Option[Any] = Some("apple")
resolve(a) // match first case
resolve(b) // match second case
resolve(c) // match third case
resolve(d) // match fourth case
resolve(e) // match fifth case
As has been pointed out the return type is actually Option[List[Map[String, Any]]] so you need to unpick this. However you cannot do this with a single match because of type erasure, so you need to do nested matches to ensure that you have the correct type. This is really tedious, so I thoroughly recommend using something like the Extraction.extract function in json4s that will attempt to match your JSON to a specific Scala type:
type ResultType = List[Map[String, Any]]
def extract(json: JValue)(implicit formats: Formats, mf: Manifest[ResultType]): ResultType =
Extraction.extract[ResultType](json)
If you must do it by hand, it looks something like this:
result match {
case Some(l: List[_]) =>
l.headOption match {
case Some(m) =>
m match {
case m: Map[_,_] =>
m.headOption match {
case Some(p) =>
p match {
case (_: String, _) =>
m.foreach(println(_))
case _ => println("Map key was not String")
}
case _ => println("Map was empty")
}
case _ => println("List did not contain a Map")
}
case _ => println("Result List was empty")
}
case _ => println("Parsing failed")
}
Your output is Option[List[Map[String, Any]]], not Option[Map[String, Any]]. match on the List and you'll be fine:
import scala.util.parsing.json._
val result = JSON.parseFull("[{\"start\":\"starting\",\"test\":123,\"test2\":324,\"end\":\"ending\"}]")
val l: List[Map[String, Any]] = result match {
case Some(list: List[Map[String, Any]]) => list
case _ => throw new Exception("I shouldn't be here") // whatever for a non-match
}
Then you can map (if you want a non-Unit return type)/ foreach (if you don't care about a Unit return type) on that List and do whatever you want it:
l.foreach(println)
l.map(_.toString) // or whatever you want ot do with the Map
I am building a web app using Scala / Play Framework and Reactive Mongo and I want the models to be defined in the database instead of having them hardcoded.
To do so, I am writing a class EntityInstance taking a Sequence of FieldInstance :
case class EntityInstance(fields: Seq[FieldInstance])
I am trying to accept fields from any types and to convert them to Json : example
new FieldInstance("name", "John") | json: { "name": "John" }
new FieldInstance("age", 18) | json: { "age": 18 }
At the moment I am trying to accept Strings, Booleans and Integers and if the type is not supported I write some error :
new FieldInstance("profilePicture", new Picture("john.jpg") | json: { "profilePicture": "Unsupported type
I wrote a FieldInstance class taking a fieldName as a String and a value as any type. As soon as that class is instantiated I cast the value to a known type or to the String describing the error.
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec match {
case v: Int => v
case v: String => v
case v: Boolean => v
case _ => "Unrecognized type"
}
}
object FieldInstance {
implicit val fieldInstanceWrites = new Writes[FieldInstance] {
def writes(fieldInstance: FieldInstance) = Json.obj(
fieldInstance.fieldName -> fieldInstance.value
)
}
}
I created a companion object with an implicit Write to json so I can call "Json.toJson()" on an instance of FieldInstance and get a json as described on my examples above.
I get an error : found: Any required: play.api.libs.json.Json.JsValueWrapper
I understand that it comes from the fact that my value is of type Any but I thought the cast would change that Any to String || Boolean || Int before hitting the Writer.
PS: Ignore the bad naming of the classes, I could not name EntityInstance and FieldInstance, Entity and Field because these as the classes I use to describe my models.
I found a fix to my problem :
The type matching that I was doing in the class should be done in the implicit Write !
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec
override def toString(): String = "(" + fieldName + "," + value + ")";
}
object FieldInstance {
implicit val fieldInstanceWrites = new Writes[FieldInstance] {
def writes(fieldInstance: FieldInstance) =
fieldInstance.value match {
case v: Int => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[Int])
case v: String => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[String])
case v: Boolean => Json.obj(fieldInstance.fieldName -> v.asInstanceOf[Boolean])
case _ => Json.obj(fieldInstance.fieldName -> "Unsupported type")
}
}
}
This code now allows a user to create an EntityInstance with Fields of Any type :
val ei = new EntityInstance(Seq[FieldInstance](new FieldInstance("name", "George"), new FieldInstance("age", 25), new FieldInstance("married", true)))
println("-- TEST ENTITY INSTANCE TO JSON --")
println(Json.toJson(ei))
prints : {"entity":[{"name":"George"},{"age":25},{"married":true}]}
Here is my EntityInstance code if you are trying to test it :
case class EntityInstance(fields: Seq[FieldInstance])
object EntityInstance {
implicit val EntityInstanceWrites = new Writes[EntityInstance] {
def writes(entityInstance: EntityInstance) =
Json.obj("entity" -> entityInstance.fields)
}
}
It is returning a String, Int or Boolean but Json.obj is expecting the value parameter of type (String, JsValueWrapper)
def obj(fields: (String, JsValueWrapper)*): JsObject = JsObject(fields.map(f => (f._1, f._2.asInstanceOf[JsValueWrapperImpl].field)))
a quick fix could be to convert the matched value v with toJson provided the implicit Writes[T] for type T is available (which they are for String, Int and Boolean)
class FieldInstance(fieldNamec: String, valuec: Any) {
val fieldName = fieldNamec
val value = valuec match {
case v: Int => Json.toJson(v)
case v: String => Json.toJson(v)
case v: Boolean => Json.toJson(v)
case _ => Json.toJson("Unrecognized type")
}
}
If you'd like to see which DefaultWrites are available you can browse them in the play.api.libs.json package in trait DefaultWrites
for example:
/**
* Serializer for Boolean types.
*/
implicit object BooleanWrites extends Writes[Boolean] {
def writes(o: Boolean) = JsBoolean(o)
}
I am trying to pass a function as a parameter, but that function has multiple arguments (one of which is a function).
Here is what I am trying to do in a basic Python example:
def first(string1, string2, func):
func(string1, string2, third)
def second(string1, string2, func):
func(string1, string2)
def third(string1, string):
# operations go here
first("one", "two", second)
My attempt at this in Scala was the following:
def first(string1: String, string2: String, func: (Any, Any, Any) => Unit) = {
func(string1, string2, func)
}
def second(string1: String, string2: String, func: (Any, Any) => Unit) = {
func(string1, string2)
}
def third(string1: String, string2: String) = {
// operations
}
def main(args: Array[String]): Unit = {
first("one", "two", second)
}
I get an error for trying to pass second into first with an insufficient amount of arguments. Is it possible to achieve this functionality in the same style as the Python example?
EDIT:
I tried replacing the body of my main method with first("one", "two", second _) and it gives me a type mismatch error
type mismatch; found : (String, String, (Any, Any, Any) => Unit) => Unit required: (Any, Any, Any) =>
Unit
Any idea what's going on here?
What you are trying to do is not type-safe. You cannot assign (String, String, (Any, Any) => Unit) => Unit to (Any, Any, Any) => Unit. If you could, then you could do the following:
val f = second _
val af: (Any, Any, Any) => Unit = f
af(1, "abc", 5)
You can do it if you specify the types more precisely:
def second(string1: String, string2: String, func: (String, String) => Unit) = {
func(string1, string2)
}
def third(string1: String, string2: String) = {
// operations
}
def first(string1: String, string2: String, func: (String, String, (String, String) => Unit) => Unit) = {
func(string1, string2, third)
}