I am using json-spray. It seems that when I attempt to print a parsed JsString value, it includes book-ended quotes on the string.
val x1 = """ {"key1": "value1", "key2": 4} """
println(x1.asJson)
println(x1.asJson.convertTo[Map[String, JsValue]])
Which outputs:
{"key1":"value1","key2":4}
Map(key1 -> "value1", key2 -> 4)
But that means that the string value of key1 is actually quoted since scala displays strings without their quotes. i.e. val k = "value1" outputs: value1 not "value1". Perhaps I am doing something wrong, but the best I could come up with to alleviate this was the following:
val m = x1.asJson.convertTo[Map[String, JsValue]]
val z = m.map({
case(x,y) => {
val ny = y.toString( x => x match {
case v: JsString =>
v.toString().tail.init
case v =>
v.toString()
} )
(x,ny)
}})
println(z)
Which outputs a correctly displayed string:
Map(key1 -> value1, key2 -> 4)
But this solution won't work for recursively nested JSON. Is there a better workaround?
Try this:
import spray.json._
import DefaultJsonProtocol._
val jsString = new JsString("hello")
val string = jsString.convertTo[String]
In the new version, there is a little difference:
libraryDependencies ++= "io.spray" % "spray-json_2.12" % "1.3.3"
import spray.json.DefaultJsonProtocol._
import spray.json._
object SprayJson extends ExampleBase {
private def jsonValue(): Map[String, String] = {
val x1 = """ {"key1": "value1", "key2": 4} """
val json = x1.parseJson
println(json.prettyPrint)
json.convertTo[Map[String, JsValue]].map(v =>
(v._1, v._2 match {
case s: JsString => s.value
case o => o.toString()
}))
}
override def runAll(): Unit = {
println(jsonValue())
}
}
The output:
{
"key1": "value1",
"key2": 4
}
Map(key1 -> value1, key2 -> 4)
I ran into exact same problem. Digging through the source code, they are using compactPrint which seems fine. I don't at what point it is bring wrapped with quotes
Related
I have a list of JsObject like:
[{
"a":"test1",
"b": 2,
"c": 5,
"errors": "Error example"
}]
I would like to get something like this a:List[Option[String]], b: List[Option[Int]] and so on. I need an option since not all the fields are alway present.
My code is:
jsObjList.map(js => {
val a = (js \ "a").asOpt[String]
val b = (js \ "b").asOpt[Int]
val c = (js \ "c").asOpt[Int]
val er= (js \ "errors").asOpt[String]
(a, b, er)
})
I read about unzip and unzip3 but I haven't found a generic function.
P.S. I am using Scala Play for the json parsing
Thanks for your help!
Class to extract values from raw JSON.
case class Foo(a: Option[String], b: Option[Int], c: Option[Int],errors: Option[String])
object Foo {
// Automatically generate json reader and writer for the class Foo
implicit val format = Json.format[Foo]
}
Keeping the implicit value in companion object of Foo will make the Scala to pick up the implicit when required automatically.
Code to parse JSON into list of case class instances
payload.validate[List[Foo]]
Use validateOpt in case you expect any parse error
payload.validateOpt[List[Foo]]
Scala REPL
scala> :paste
// Entering paste mode (ctrl-D to finish)
val str = """
[{
"a":"test1",
"b": 2,
"c": 5,
"errors": "Error example"
}]
"""
// Exiting paste mode, now interpreting.
str: String =
"
[{
"a":"test1",
"b": 2,
"c": 5,
"errors": "Error example"
}]
"
scala> val payload = Json.parse(str)
payload: play.api.libs.json.JsValue = [{"a":"test1","b":2,"c":5,"errors":"Error example"}]
scala> case class Foo(a: Option[String], b: Option[Int], c: Option[Int],errors: Option[String])
defined class Foo
scala> implicit val format = Json.format[Foo]
format: play.api.libs.json.OFormat[Foo] = play.api.libs.json.OFormat$$anon$1#53a0b0a3
scala> payload.validate[List[Foo]]
res5: play.api.libs.json.JsResult[List[Foo]] = JsSuccess(List(Foo(Some(test1),Some(2),Some(5),Some(Error example))),)
You can parse JSON as a Scala case class with a companion object containing a special val called implicit val format = Json.format[*your class*].
Here's an example similar to yours:
import play.api.libs.json.Json
val body =
"""{
| "a":"my string",
| "b": 1,
| "c": 2
|}
""".stripMargin
val body2 =
"""{
| "a":"my string",
| "c": 5
|}
""".stripMargin
case class MyClass(a: Option[String], b: Option[Int], c: Option[Int])
object MyClass {
implicit val format = Json.format[MyClass]
}
Using this, calling Json.parse(body).as[MyClass] gives:
res0: MyClass = MyClass(Some(my string),Some(2),Some(5))
Calling this Json.parse function with missing fields (assuming they are optional), such as Json.parse(body2).as[MyClass] gives:
res1: MyClass = MyClass(Some(my string),None,Some(5))
If one of the missing fields is not Optional, this parse will not work.
Origin
{
"first_name" : "foo",
"last_name" : "bar",
"parent" : {
"first_name" : "baz",
"last_name" : "bazz",
}
}
Expected
{
"firstName" : "foo",
"lastName" : "bar",
"parent" : {
"firstName" : "baz",
"lastName" : "bazz",
}
}
How can I transform all keys of json objects ?
Here's how I'd write this. It's not as concise as I'd like, but it's not terrible:
import cats.free.Trampoline
import cats.std.list._
import cats.syntax.traverse._
import io.circe.{ Json, JsonObject }
/**
* Helper method that transforms a single layer.
*/
def transformObjectKeys(obj: JsonObject, f: String => String): JsonObject =
JsonObject.fromIterable(
obj.toList.map {
case (k, v) => f(k) -> v
}
)
def transformKeys(json: Json, f: String => String): Trampoline[Json] =
json.arrayOrObject(
Trampoline.done(json),
_.traverse(j => Trampoline.suspend(transformKeys(j, f))).map(Json.fromValues),
transformObjectKeys(_, f).traverse(obj => Trampoline.suspend(transformKeys(obj, f))).map(Json.fromJsonObject)
)
And then:
import io.circe.literal._
val doc = json"""
{
"first_name" : "foo",
"last_name" : "bar",
"parent" : {
"first_name" : "baz",
"last_name" : "bazz"
}
}
"""
def sc2cc(in: String) = "_([a-z\\d])".r.replaceAllIn(in, _.group(1).toUpperCase)
And finally:
scala> import cats.std.function._
import cats.std.function._
scala> transformKeys(doc, sc2cc).run
res0: io.circe.Json =
{
"firstName" : "foo",
"lastName" : "bar",
"parent" : {
"firstName" : "baz",
"lastName" : "bazz"
}
}
We probably should have some way of recursively applying a Json => F[Json] transformation like this more conveniently.
Depending on your full use-case, with the latest Circe you might prefer just leveraging the existing decoder/encoder for converting between camel/snake according to these references:
https://dzone.com/articles/5-useful-circe-feature-you-may-have-overlooked
https://github.com/circe/circe/issues/663
For instance, in my particular use-case this makes sense because I'm doing other operations that benefit from the type-safety of first deserializing into case classes. So if you're willing to decode the JSON into a case class, and then encode it back into JSON, all you would need is for your (de)serializing code to extend a trait that configures this, like:
import io.circe.derivation._
import io.circe.{Decoder, Encoder, ObjectEncoder, derivation}
import io.circe.generic.auto._
import io.circe.parser.decode
import io.circe.syntax._
trait JsonSnakeParsing {
implicit val myCustomDecoder: Decoder[MyCaseClass] = deriveDecoder[MyCaseClass](io.circe.derivation.renaming.snakeCase)
// only needed if you want to serialize back to snake case json:
// implicit val myCustomEncoder: ObjectEncoder[MyCaseClass] = deriveEncoder[MyCaseClass](io.circe.derivation.renaming.snakeCase)
}
For example, I then extend that when I actually parse or output the JSON:
trait Parsing extends JsonSnakeParsing {
val result: MyCaseClass = decode[MyCaseClass](scala.io.Source.fromResource("my.json").mkString) match {
case Left(jsonError) => throw new Exception(jsonError)
case Right(source) => source
}
val theJson = result.asJson
}
For this example, your case class might look like:
case class MyCaseClass(firstName: String, lastName: String, parent: MyCaseClass)
Here's my full list of circe dependencies for this example:
val circeVersion = "0.10.0-M1"
"io.circe" %% "circe-generic" % circeVersion,
"io.circe" %% "circe-parser" % circeVersion,
"io.circe" %% "circe-generic-extras" % circeVersion,
"io.circe" %% "circe-derivation" % "0.9.0-M5",
def transformKeys(json: Json, f: String => String): TailRec[Json] = {
if(json.isObject) {
val obj = json.asObject.get
val fields = obj.toList.foldLeft(done(List.empty[(String, Json)])) { (r, kv) =>
val (k, v) = kv
for {
fs <- r
fv <- tailcall(transformKeys(v, f))
} yield fs :+ (f(k) -> fv)
}
fields.map(fs => Json.obj(fs: _*))
} else if(json.isArray) {
val arr = json.asArray.get
val vsRec = arr.foldLeft(done(List.empty[Json])) { (vs, v) =>
for {
s <- vs
e <- tailcall(transformKeys(v, f))
} yield s :+ e
}
vsRec.map(vs => Json.arr(vs: _*))
} else {
done(json)
}
}
Currently I do transform like this, but is rather complicated, hope there is a simple way.
I took #Travis answer and modernized it a bit, I took his code and I had several error and warnings, so the updated version for Scala 2.12 with Cats 1.0.0-MF:
import io.circe.literal._
import cats.free.Trampoline, cats.instances.list._, cats.instances.function._, cats.syntax.traverse._, cats.instances.option._
def transformKeys(json: Json, f: String => String): Trampoline[Json] = {
def transformObjectKeys(obj: JsonObject, f: String => String): JsonObject =
JsonObject.fromIterable(
obj.toList.map {
case (k, v) => f(k) -> v
}
)
json.arrayOrObject(
Trampoline.done(json),
_.toList.traverse(j => Trampoline.defer(transformKeys(j, f))).map(Json.fromValues(_)),
transformObjectKeys(_, f).traverse(obj => Trampoline.defer(transformKeys(obj, f))).map(Json.fromJsonObject)
)
}
def sc2cc(in: String) = "_([a-z\\d])".r.replaceAllIn(in, _.group(1).toUpperCase)
def camelizeKeys(json: io.circe.Json) = transformKeys(json, sc2cc).run
Given the following JSON:
{
"field1": "value1",
"field2": "",
"field3": "value3",
"field4": ""
}
How do I get two distinct JSONs, one containing the fields with value and another one containing the fields without value? Here below is how the final result should look like:
{
"field1": "value1",
"field3": "value3"
}
{
"field2": "",
"field4": ""
}
You have access to the JSON object's fields as a sequence of (String, JsValue) pairs and you can filter through them. You can filter out the ones with and without value and use the filtered sequences to construct new JsObject objects.
import play.api.libs.json._
val ls =
("field1", JsString("value1")) ::
("field2", JsString("")) ::
("field3", JsString("value3")) ::
("field4", JsString("")) ::
Nil
val js0 = new JsObject(ls)
def withoutValue(v: JsValue) = v match {
case JsNull => true
case JsString("") => true
case _ => false
}
val js1 = JsObject(js0.fields.filterNot(t => withoutValue(t._2)))
val js2 = JsObject(js0.fields.filter(t => withoutValue(t._2)))
You can improve #nietaki solution by using partition function.
import play.api.libs.json._
val ls =
("field1", JsString("value1")) ::
("field2", JsString("")) ::
("field3", JsString("value3")) ::
("field4", JsString("")) ::
Nil
val js0 = new JsObject(ls)
def withoutValue(v: JsValue) = v match {
case JsNull => true
case JsString("") => true
case _ => false
}
val (js1, js2) = js0.fields.partition(t => withoutValue(t._2))
JsObject(js1)
JsObject(js2)
You can also do it in a one liner:
val (js1, js2) = j.fields.partition(_._2 != JsString(""))
println(JsObject(js1))
println(JsObject(js2))
Code run at Scastie
I'm reading data from a KV store (Redis) in this case. The data returned is in the following format.
{ "key1":"value1", "key2":"value2", "key3":"value3" ...}
Key is String and value is Int. I want to convert it into a Map[String,Int]
I looked at the json4s JSON API and my current code looks like the following. Is there a better/easier/cleaner way of doing this?
//send a async query to Redis to
val queryFuture = redis.zrangebyscore[String](tablename, keymin, keymax )
queryFuture.onComplete {
case Success(datarows) =>
println(s"Got %d rows of type %s for the query successfully".format(datarows.length))
val jsonrows = for { x <- datarows.toList }
yield parse(x)
println("Got json rows %d".format(jsonrows.size))
val mapdata = jsonrows.map(x => x.extract[Map[String,String]]).map( x => x.mapValues(_.toInt))
//need to do something with this data now
case Failure(y) =>
println(s" there was some failure in getting the data from Redis")
}
This looks to me like the simplest way to do it:
val map = parse("""{"a":"1","b":"2","c":"3"}""")
.children
.collect { case JField(k, JString(v)) => (k, v.toInt) }
.toMap
Your Json4s solution looks fine. Alternatively you can use mapField to transform the fields of a JObject and after that extract value of type Map[String, Int].
val json1 = parse(
"""
|{
| "key1": "1024",
| "key2": "2048",
| "key3": "4096"
|}
""".stripMargin)
val json2 = json1.mapField {
case (key, JString(value)) => (key, JInt(value.toInt))
case x => x
}
val res = json2.extract[Map[String, Int]]
println(res)
// Map(key1 -> 1024, key2 -> 2048, key3 -> 4096)
Not knowing json4s, and unfortunately you ommited the types, but guessing that jsonrows is probably something like a List[(String, String)] you could do
List(("key1" -> "1"),("key2" -> "2")).map { case (k, v) => (k, v.toInt)}.toMap
BTW, if you say need to do something with this data now in your onComplete - that could only be a side effecting operation. Better map over the future until your processing is complete.
I'm trying to convert Scala to JSON in the 2.1RC Play Framework.
I can do the following and get JSON:
import play.api.libs.json._
val a1=Map("val1"->"a", "val2"->"b")
Json.toJSon(a1)
Because a1 is just Map[String,String] that works OK.
But if I have something more complex like where I have Map[String,Object], that doesn't work:
val a = Map("val1" -> "xxx", "val2"-> List("a", "b", "c"))
Json.toJSon(a1)
>>> error: No Json deserializer found for type scala.collection.immutable.Map[String,Object]
I found that I can do something like the following:
val a2 = Map("val1" -> Json.toJson("a"), "val2" -> Json.toJson(List("a", "b", "c")))
Json.toJson(a2)
And that works.
But how can I do that in a general way? I thought that I could do something like the following:
a.map{ case(k,v)=> (k, Json.toJson(v) )}
>>> error: No Json deserializer found for type Object
But I still get an error that it can't be deserialized
Additional Information:
Json.toJson can convert a Map[String, String] to a JsValue:
scala> val b = Map( "1" -> "A", "2" -> "B", "3" -> "C", "4" -> "D" )
b: scala.collection.immutable.Map[String,String] = Map(1 -> A, 2 -> B, 3 -> C, 4 -> D)
scala> Json.toJson(b)
res31: play.api.libs.json.JsValue = {"1":"A","2":"B","3":"C","4":"D"}
But, it fails in trying to convert a Map[String, Object]:
scala> a
res34: scala.collection.immutable.Map[String,Object] = Map(val1 -> xxx, val2 -> List(a, b, c))
scala> Json.toJson(a)
<console>:12: error: No Json deserializer found for type scala.collection.immutable.Map[String,Object]. Try to implement an implicit Writes or Format for this type.
Json.toJson(a)
Using the 'hint' from this Play Framework page on converting Scala to Json, I found the following (http://www.playframework.org/documentation/2.0.1/ScalaJson):
If instead of Map[String, Object], there is a Map[String, JsValue], then Json.toJson() will work:
scala> val c = Map("aa" -> Json.toJson("xxxx"), "bb" -> Json.toJson( List("11", "22", "33") ) )
c: scala.collection.immutable.Map[String,play.api.libs.json.JsValue] = Map(aa -> "xxxx", bb -> ["11","22","33"])
scala> Json.toJson(c)
res36: play.api.libs.json.JsValue = {"aa":"xxxx","bb":["11","22","33"]}
So, what I would like, is that given a Map[String, Object], where I know that the Object values were all originally of type String or List[String], how to apply the function Json.toJson() to the all the values in the map and get a Map[String, JsValue].
I also found that I can filter out those values that are purely string and those that are (were) of type List[String]:
scala> val a1 = a.filter({case(k,v) => v.isInstanceOf[String]})
a1: scala.collection.immutable.Map[String,Object] = Map(val1 -> xxx)
scala> val a2 = a.filter({case(k,v) => v.isInstanceOf[List[String]]})
<console>:11: warning: non-variable type argument String in type List[String] is unchecked since it is eliminated by erasure
val a2 = a.filter({case(k,v) => v.isInstanceOf[List[String]]})
^
a2: scala.collection.immutable.Map[String,Object] = Map(val2 -> List(a, b, c))
The List[String] filtering gives a warning, but seems to give the answer I want. If the two filters could be applied and then Json.toJson() used on the values of the result, and the results combined, maybe that would work?
But the filtered results are still of type Map[String, Object] which causes a problem:
scala> Json.toJson(a1)
<console>:13: error: No Json deserializer found for type scala.collection.immutable.Map[String,Object]. Try to implement an implicit Writes or Format for this type.
Json.toJson(a1)
Play 2.1 JSON API does not provide a serializer for the Type Map[String, Ojbect].
Define case class and Format for the specific type instead of Map[String, Object]:
// { "val1" : "xxx", "val2" : ["a", "b", "c"] }
case class Hoge(val1: String, val2: List[String])
implicit val hogeFormat = Json.format[Hoge]
If you don't want to create case class.
The following code provides JSON serializer/deserializer for Map[String, Object]:
implicit val objectMapFormat = new Format[Map[String, Object]] {
def writes(map: Map[String, Object]): JsValue =
Json.obj(
"val1" -> map("val1").asInstanceOf[String],
"val2" -> map("val2").asInstanceOf[List[String]]
)
def reads(jv: JsValue): JsResult[Map[String, Object]] =
JsSuccess(Map("val1" -> (jv \ "val1").as[String], "val2" -> (jv \ "val2").as[List[String]]))
}
More dynamically
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.json.Json.JsValueWrapper
implicit val objectMapFormat = new Format[Map[String, Object]] {
def writes(map: Map[String, Object]): JsValue =
Json.obj(map.map{case (s, o) =>
val ret:(String, JsValueWrapper) = o match {
case _:String => s -> JsString(o.asInstanceOf[String])
case _ => s -> JsArray(o.asInstanceOf[List[String]].map(JsString(_)))
}
ret
}.toSeq:_*)
def reads(jv: JsValue): JsResult[Map[String, Object]] =
JsSuccess(jv.as[Map[String, JsValue]].map{case (k, v) =>
k -> (v match {
case s:JsString => s.as[String]
case l => l.as[List[String]]
})
})
}
Sample code:
val jv = Json.toJson(Map("val1" -> "xxx", "val2" -> List("a", "b", "c"), "val3" -> "sss", "val4" -> List("d", "e", "f")))
println(jv)
val jr = Json.fromJson[Map[String, Object]](jv)
println(jr.get)
The output:
> {"val1":"xxx","val2":["a","b","c"],"val3":"sss","val4":["d","e","f"]}
> Map(val1 -> xxx, val2 -> List(a, b, c), val3 -> sss, val4 -> List(d, e, f))