I have a simple action which outputs a json object string, like this:
Ok(toJson(Map(
"results" -> result_lists
)))
This works all right. But if I do:
Ok(toJson(Map(
"action" -> action_string, // a Scala String
"results" -> result_lists // a Scala List
)))
I got
No Json serializer found for type scala.collection.immutable.Map[String,java.io.Serializable]
compilation error...what's the problem?
As others have posted in the comments before, the type of the Map is not something which can be deserialized into Json by the framework, but you can easily get rid of the Map:
scala> val s = "hello"
s: String = hello
scala> val list = List(1,2,3)
list: List[Int] = List(1, 2, 3)
scala> Json.obj("somestring" -> s, "somemap" -> list)
res0: play.api.libs.json.JsObject = {"somestring":"hello","somemap":[1,2,3]}
The resulting object can then be returned by the action as desired.
Related
I want to read a generic CSV file with headers but unknown column number into a typed structure. My question is kind of the same as Strongly typed access to csv in scala? but with the fact I would have no schema to pass to the parser...
Until now, I was using Jackson CSV mapper to read each row as a Map[String,String], and it was working well.
import com.fasterxml.jackson.module.scala.DefaultScalaModule
def genericStringIterator(input: InputStream): Iterator[Map[String, String]] = {
val mapper = new CsvMapper()
mapper.registerModule(DefaultScalaModule)
val schema = CsvSchema.emptySchema.withHeader
val iterator = mapper
.readerFor(classOf[Map[String, String]])
.`with`(schema)
.readValues[Map[String, String]](input)
iterator.asScala
}
Now, we need the field to be typed, so 4.2 would be a Double but "4.2" would still be a String.
We are using play-json everywhere in our project, and so I know JsValue already has a great type inference for generic stuff like that.
As paly-json it is based on Jackson too, I thought it would be great to have something like
import play.api.libs.json.jackson.PlayJsonModule
def genericStringIterator(input: InputStream): Iterator[JsValue] = {
val mapper = new CsvMapper()
mapper.registerModule(PlayJsonModule)
val schema = CsvSchema.emptySchema.withHeader
val iterator = mapper
.readerFor(classOf[JsValue])
.`with`(schema)
.readValues[JsValue](input)
iterator.asScala
}
But when I try the former code, I get an exception :
val iterator = CSV.genericAnyIterator(input(
"""foo,bar,baz
|"toto",42,43
|"tata",,45
| titi,87,88
|"tutu",,
|""".stripMargin))
iterator
.foreach { a =>
println(a)
}
java.lang.RuntimeException: We should be reading map, something got wrong
at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:165)
at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:128)
at play.api.libs.json.jackson.JsValueDeserializer.deserialize(JacksonJson.scala:123)
at com.fasterxml.jackson.databind.MappingIterator.nextValue(MappingIterator.java:277)
at com.fasterxml.jackson.databind.MappingIterator.next(MappingIterator.java:192)
at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:40)
at scala.collection.Iterator.foreach(Iterator.scala:929)
at scala.collection.Iterator.foreach$(Iterator.scala:929)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
at my.company.csv.CSVSpec$$anon$4.<init>(CSVSpec.scala:240)
Is there something I'm doing wrong ?
I don't care particulary to have a play-json JsValue in the end, any Json structure with generic typed field would be ok. Is there another lib I could use for that ? For what I found, all other libs are based on a mapping given to the CSV Reader in advance, and what is important for me is to be able to infer the type from the CSV.
Ok, I was lazy to want to find something working out of the box :)
In fact it was easy to implement it myself.
I went looking to lib in other languages that does this infering (PapaParse in JS, Pandas in python), and discovered that they were doing a test-and-retry with priority to guess the types.
So I implemented it myself, and it works fine !
Here it is:
def genericAnyIterator(input: InputStream): Iterator[JsValue] = {
// the result of former code, mapping to Map[String,String]
val strings = genericStringIterator(input)
val decimalRegExp: Regex = "(\\d*){1}(\\.\\d*){0,1}".r
val jsValues: Iterator[Map[String, JsValue]] = strings.map { m =>
m.mapValues {
case "" => None
case "false" | "FALSE" => Some(JsFalse)
case "true" | "TRUE" => Some(JsTrue)
case value#decimalRegExp(g1, g2) if !value.isEmpty => Some(JsNumber(value.toDouble))
case "null" | "NULL" => Some(JsNull)
case value#_ => Some(JsString(value))
}
.filter(_._2.isDefined)
.mapValues(_.get)
}
jsValues.map(map => JsObject(map.toSeq))
}
which does in a test
it should "read any csv in JsObject" in new WithInputStream {
val iterator = CSV.genericAnyIterator(input(
"""foo,bar,baz
|"toto",NaN,false
|"tata",,TRUE
|titi,87.79,88
|"tutu",,null
|"tete",5.,.5
|""".stripMargin))
val result: Seq[JsValue] = iterator.toSeq
result should be(Stream(
Json.obj("foo" -> "toto", "bar" -> "NaN", "baz" -> false)
, Json.obj("foo" -> "tata", "baz" -> true)
, Json.obj("foo" -> "titi", "bar" -> 87.79, "baz" -> 88)
, Json.obj("foo" -> "tutu", "baz" -> JsNull)
, Json.obj("foo" -> "tete", "bar" -> 5, "baz" -> 0.5)
) )
}
I use json4s native,with a json string like this
val myjson = """
{
"normative":"C",
"prefixType":{
"cod":["smallint", "int", "varchar(5)"],
"des":["varchar", "string"],
"fec":["timestamp"],
"hms":["timestamp"],
"tim":["timestamp"],
"imp":["decimal","Float", "Double"]
},
"fixcolname":{
"aud_usuario":"varchar(8)",
"aud_fec":"timestamp",
"aud_tim":"timestamp"
},
"symSep":"_",
"maxLength":26
}"""
And a case class
case class colVerify(prefixType: Map[String, Array[String]], fixcolname: Map[String, String], symSep: String, maxLength: Int)
and I want to extract it from the json String
val t = parse(myjson)
implicit val formats = DefaultFormats
val myvfy = t.extract[colVerify]
then got an error like this
Exception in thread "main" org.json4s.package$MappingException: Parsed JSON values do not match with class constructor
args=Map(des -> [Ljava.lang.String;#d7b1517, fec -> [Ljava.lang.String;#16c0663d, tim -> [Ljava.lang.String;#23223dd8, hms -> [Ljava.lang.String;#4ec6a292, imp -> [Ljava.lang.String;#1b40d5f0, cod -> [Ljava.lang.String;#ea4a92b),Map(aud_usuario -> varchar(8), aud_fec -> timestamp, aud_tim -> timestamp),_,26
arg types=scala.collection.immutable.HashMap$HashTrieMap,scala.collection.immutable.Map$Map3,java.lang.String,java.lang.Integer
constructor=public colVerify(scala.collection.mutable.Map,scala.collection.mutable.Map,java.lang.String,int)
Seems like it has problem with the type of Map, but how can I convert it implicitly?
The problem is that the maps in your case class are mutable maps, is this intentional or did you accidently import collection.mutable.Map?
If you really want the mutable maps, you could implement a custom Serializer as described here: https://github.com/json4s/json4s#serializing-non-supported-types
My first idea to add another constructor with immutable maps in case class doesn't seem to work reliably.
How to write a Json Transformer that converts
{"id" : [1,2], "name":["a","b"]} to idname:[ {id:1, name:"a"}, {id:2 , name:"b"}]
I am following this guide for implementing transfomration https://www.playframework.com/documentation/2.4.x/ScalaJsonTransformers
Writing the general case transformer can be difficult. Instead you can try reasoning through the standard Scala objects of Map and List. Here's a way to do it using map twice:
import play.api.libs.json._
val a = Json.parse("""{"id" : [1,2], "name":["a","b"]}""").as[Map[String, List[JsValue]]]
val b = a.map {case (k, v) => v.map(k -> _)}
.transpose
.map(_.toMap)
Json.toJson(b)
// play.api.libs.json.JsValue = [{"id":1,"name":"a"},{"id":2,"name":"b"}]
Would anyone please explain why the following happens?
scala> import play.api.libs.json._
scala> Json.toJson("""{"basic":"test"}""") // WORKS CORRECTLY
res134: play.api.libs.json.JsValue = "{\"basic\":\"test\"}"
scala> Json.toJson(""" {"basic":"test"} """) \ "basic" // ??? HOW COME?
res131: play.api.libs.json.JsValue = JsUndefined('basic' is undefined on object: " {\"basic\":\"test\"} ")
Many thanks
Json.toJson renders its argument as a JSON value using an implicitly provided Writes instance. If you give it a string, you'll get a JsString (typed as a JsValue). You want Json.parse, which parses its argument:
scala> Json.parse("""{"basic":"test"}""") \ "basic"
res0: play.api.libs.json.JsValue = "test"
As expected.
And to address your answer (which should be a comment or a new question, by the way), if you give toJson a value of some type A, it will convert it into a JSON value, assuming that there's an instance of the Writes type class in scope for that A. For example, the library provides Writes[String], Writes[Int], etc., so you can do the following:
scala> Json.prettyPrint(Json.toJson(1))
res11: String = 1
scala> Json.prettyPrint(Json.toJson("a"))
res12: String = "a"
scala> Json.prettyPrint(Json.toJson(List("a", "b")))
res13: String = [ "a", "b" ]
You can also create Writes instances for your own types (here I'm using Play's "JSON inception"):
case class Foo(i: Int, s: String)
implicit val fooWrites: Writes[Foo] = Json.writes[Foo]
And then:
scala> Json.prettyPrint(Json.toJson(Foo(123, "foo")))
res14: String =
{
"i" : 123,
"s" : "foo"
}
Using type classes to manage encoding and decoding is an alternative to reflection-based approaches, and it has a lot of advantages (but that's out of the scope of this question).
Turning my comment into an answer:
Json.toJson() does not create an object. It turns an object into a JSON string. What I think you're wanting is Json.parse(). Once you've parsed a JSON string, it's an object, and you can get to the properties.
Thanks a lot to both of you. So the following works as expected.
scala> Json.parse("""{"basic":"test"}""") \ "basic"
res137: play.api.libs.json.JsValue = "test"
I'd still like to understand what Json.toJson does. The docs state "Transform a stream of A to a stream of JsValue". Can anyone point out in what context this can be used?
I have the following piece of code:
(json \ field.name).as[Int]
The problem is, the code seems to look only into the 1st "layer" of a json document, giving me an error when the JsObject is wrapped into an Array.
To better illustrate the point:
This json works:
{
fieldName: 123
}
This doesnt:
[
{
fieldName: 123
}
]
So, how do I look up the value of the fieldName in the 2nd json?
As you said, this json is an array where the first value is your object.
You can use ordinal traversing to obtain the first object, and parse it.
scala> val json = Json.arr(Json.obj("value" -> 10))
scala> json(0)
res0: play.api.libs.json.JsValue = {"value":10}
scala> (json(0) \ "value").as[Int]
res1: Int = 10