I am trying to understand Spray Json and very new to Scala. I have Seq(Seq("abc", 123, false, null), Seq("def", 45, "1234", 'C')) so a Seq[Seq[Any]]. I am not sure how to go about this and I cannot find any examples online.
case class SeqFormat(Seq[Seq[Any]]) => {
// something that would convert to Seq[Seq[String]]
//which will would return a Json like this
//[["abc", "123", "false", "null"],["def", "45", "1234", "C"]]
}
I tried
val someSeq = [["abc", "123", "false", "null"],["def", "45", "1234", "C"]]
val myObj = someSeq.toJson
// This gives me an error saying Any is not valid
I would appreciate any hints or snippets to help me understand this.
You could use an encoder such as:
import spray.json._
import DefaultJsonProtocol._
implicit object AnyJsonFormat extends JsonFormat[Any] {
def write(x: Any) =
try {
x match {
case n: Int => JsNumber(n)
case s: String => JsString(s)
case c: Char => JsString(c.toString)
case _ => JsString("null")
}
} catch {
case _: NullPointerException => JsString("null")
}
def read(value: JsValue) = ???
}
Which you can use as follow:
val input = Seq(Seq("abc", 123, false, null), Seq("def", 45, "1234", 'C'))
println(input.toJson)
In order to get:
[["abc",123,"null","null"],["def",45,"1234","C"]]
This is an adapted version of this post: Serialize Map[String, Any] with spray json
Notice the NullPointerException handling for the null case.
Related
The following data can be seen with different value types. How can I get the desired output?
package ceshi
import scala.util.parsing.json.JSON
object ceshi1212 {
def main(args: Array[String]): Unit = {
class CC[T] {
def unapply(a: Any): Option[T] = Some(a.asInstanceOf[T])
}
object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
val jsonString =
"""
{
"languages": [
{
"name": "English",
"is_active": "true",
"completeness": "asdf"
},
{
"name": "Latin",
"is_active": "asdf",
"completeness": "232"
}
,{
"name": "Latin",
"is_active": "0009"
}
,
"error"
]
}
""".stripMargin
// 不规则json error和并列的数组类型不同 怎么解析自动跳过?
val result = for {
Some(M(map)) <- List(JSON.parseFull(jsonString))
L(languages) = map("languages")
M(language) <- languages
S(name) = language("name")
S(active) = language("is_active")
S(completeness) = language.getOrElse("completeness","--")
} yield {
(name, active, completeness)
}
println(result)
//i want result is: List((English,true,asdf), (Latin,asdf,232),(Latain,0009,""))
}
}
i want get result is List((English,true,asdf), (Latin,asdf,232),(Latain,0009,""))
note: 1 The string is not always at the end of the array, and the position is indeterminate
2 The three keys I need may not be complete
As said in the comments there are other libraries to be recommended for working with json have a look at this post to get an overview: What JSON library to use in Scala?
Answer to your question with specific framework (play-json)
Personally I can recommend to use the play json framework.
To obtain the result you have described with play json, your code might look like this:
import play.api.libs.json._
val json: JsValue = Json.parse(jsonString)
val list = (json \ "languages").as[Seq[JsValue]]
val names = list.map(x => ((x\"name").validate[String] match {
case JsSuccess(v, p ) => v
case _ => ""
}
))
val isActives = list.map(x => ((x\"is_active").validate[String] match {
case JsSuccess(v, p ) => v
case _ => ""
}
))
val completeness = list.map(x => ((x\"completeness").validate[String] match {
case JsSuccess(v, p ) => v
case _ => ""
}
))
// need to know in advance what is your max length of your tuple (tmax)
// since 3rd value "completeness" can be missing, so we just take "" instead
val tmax = 3
val res = for(idx <-0 to tmax-1) yield (names(idx),isActives(idx),completeness(idx))
res.toList
// List[(String, String, String)] = List((English,true,asdf), (Latin,asdf,232), (Latin,0009,""))
There's also a very good documentation for the play json framework, just check it out yourself: https://www.playframework.com/documentation/2.8.x/ScalaJson
If you can switch parser library to circe, you can deal with this types of bad data.
Given you have data model
import io.circe.generic.semiauto._
import io.circe.parser.decode
import io.circe.{Decoder, Json}
case class Languages(languages: Seq[Language])
case class Language(name: String, is_active: String, completeness: Option[String])
You can define a fault-tolerant seq decoder that would skip bad data rather than crash whole parse
def tolerantSeqDecoder[A: Decoder]: Decoder[Seq[A]] = Decoder.decodeSeq(Decoder[A]
.either(Decoder[Json])).map(_.flatMap(_.left.toOption))
and the rest...
val jsonString = """
{
"languages": [
{
"name": "English",
"is_active": "true",
"completeness": "asdf"
},
{
"name": "Latin",
"is_active": "asdf",
"completeness": "232"
},
{
"name": "Latin",
"is_active": "0009"
},
"error"
]
}
"""
val languageDecoder = deriveDecoder[Language]
implicit val tolerantDecoder = tolerantSeqDecoder[Language](languageDecoder)
implicit val languagesDecoder = deriveDecoder[Languages]
val parsed = decode[Languages](jsonString)
println(parsed)
out:
Right(Languages(List(Language(English,true,Some(asdf)), Language(Latin,asdf,Some(232)), Language(Latin,0009,None))))
This approach was suggested by one of circe developers: How do I ignore decoding failures in a JSON array?
I think there is not a default format for Map[IndexedSeq[String], Int] in scala (right?) So I've written my own format as follows, however it's very slow. Is there a better way to do this?
class IndexedSeqToIntMapFormat() extends Format[Map[IndexedSeq[String], Int]] {
def writes(o: Map[IndexedSeq[String], Int]): JsValue = {
val mapItems: Seq[String] = o.toSeq.map{case (rowKey, index) => (index.toString +: rowKey).mkString(",")}
Json.obj("items" -> Json.toJson(mapItems))
}
def reads(json: JsValue): JsResult[Map[IndexedSeq[String], Int]] = {
val mapItemsAsString: IndexedSeq[String] = (json \ "items").as[IndexedSeq[String]]
val map: Map[IndexedSeq[String], Int] = mapItemsAsString.map(itemAsString => {
val item: IndexedSeq[String] = itemAsString.split(",").toIndexedSeq
val rowKey: IndexedSeq[String] = item.tail
val rowIndex: Int = item.head.toInt
(rowKey, rowIndex)
}).toMap
JsSuccess(map)
}
}
Thanks!
Can't say for sure whether the following approach is significantly faster than yours, but to my understanding, it does much more conform to the "spirit" of JSON. In a JSON serialization, each object and all its sub-objects and attributes should be named, by what they are. A list of custom string serializations of complex objects is not an actual JSON representation in my opinion.
Doing it in a "proper" JSON way should at least save some time in the parsing, as it does not require additional parsing work on strings but already provides all data in the needed places. And the code looks much more readable :-)
The resulting JSON should look like this:
"items": [
{ "keySeq": [ "key1", "key2", "key3" ], "value": 42 },
{ "keySeq": [ "key4", "key5" ], "value": 123 },
{ "keySeq": [ "key6", "key7", "key7" ], "value": 650 }
]
The formatter could be something like this:
class IndexedSeqToIntMapFormat() extends Format[Map[IndexedSeq[String], Int]] {
def writes(m: Map[IndexedSeq[String], Int]): JsValue = {
val objs = m.toSeq.map { case (keySeq, value) =>
Json.obj("keySeq" -> Json.toJson(keySeq), "value" -> JsNumber(value))
}
Json.obj("items" -> JsArray(objs))
}
def reads(json: JsValue): JsResult[Map[IndexedSeq[String], Int]] = {
val seq = (json \ "items").as[Seq[JsValue]] map { obj =>
( (obj \ "keySeq").as[IndexedSeq[String]], (obj \ "value").as[Int] )
}
JsSuccess(seq.toMap)
}
}
By the way, just out of curiosity - can you tell me in what context you need a such map?
I have to deconstruct the following JSON into a list of case classes:
{
"data": [
[49, true, 14, null, null],
[52, false, null, null, null],
[72, true, 4, 2, 1]
]
}
case class:
case class Data(i1: Int, b: Bool, i2: Option[Int], i3: Option[Int], i4: Option[Int])
I started with a for comprehension, but was not able to finish it:
for {
JArray(data) <- json \ "data"
JArray(d) <- data
JInt(line) <- d.head // ???
} yield Data()
Any help is much appreciated.
Thanks,
Michael
If you can permit including the Rapture JSON library, this can be done as follows, still using the JSON4S backend. This requires the following imports:
import rapture.json._, jsonBackends.json4s._
If you already have the JSON as a JValue, you can convert it to Rapture's Json type as follows:
val json = Json(jValue)
Given your case class definition, you need to redefine the JSON extractor for Data types (there is already a default extractor which expects a JSON object), like this:
implicit val dataExtractor = Json.extractor[Json].map { j =>
Data(j(0).as[Int], j(1).as[Boolean], j(2).as[Option[Int]],
j(3).as[Option[Int]], j(4).as[Option[Int]])
}
and you can then extract it with:
val list = json.as[List[Data]]
You can write a CustomSerializer for Data.
I introduced a JOptionInt extractor to turn a JInt or a JNull into a Option[Int], it is possible it can be done in json4s directly.
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.json4s.JsonDSL._
case class Data(i1: Int, b: Boolean, i2: Option[Int], i3: Option[Int], i4: Option[Int])
object DataSerializer extends CustomSerializer[Data]( format => (
{
case JArray(List(JInt(i1), JBool(b), JOptionInt(i2), JOptionInt(i3), JOptionInt(i4))) =>
Data(i1.toInt, b, i2, i3 , i4)
}, {
case d: Data => JArray(List(d.i1, d.b, d.i2, d.i3, d.i4))
}
))
object JOptionInt {
def unapply(x: JValue) : Option[Option[Int]] = x match {
case JInt(i) => Option(Option(i.toInt))
case JNull => Option(None)
case _ => None
}
}
Which can be used as:
implicit val formats = DataSerializer
val json = parse("""
{
"data": [
[49, true, 14, null, null],
[52, false, null, null, null],
[72, true, 4, 2, 1]
]
}
""")
val result = (json \ "data").extract[Array[Data]]
// Array(Data(49,true,Some(14),None,None), Data(52,false,None,None,None), Data(72,true,Some(4),Some(2),Some(1)))
For example, I have this Map value in Scala:
val m = Map(
"name" -> "john doe",
"age" -> 18,
"hasChild" -> true,
"childs" -> List(
Map("name" -> "dorothy", "age" -> 5, "hasChild" -> false),
Map("name" -> "bill", "age" -> 8, "hasChild" -> false)
)
)
I want to convert it to its JSON string representation:
{
"name": "john doe",
"age": 18,
"hasChild": true,
"childs": [
{
"name": "dorothy",
"age": 5,
"hasChild": false
},
{
"name": "bill",
"age": 8,
"hasChild": false
}
]
}
I'm currenly working on Play framework v2.3, but the solution doesn't need to use Play JSON library, although it will be nice if someone can provide both Play and non-Play solution.
This is what I have done so far without success:
// using jackson library
val mapper = new ObjectMapper()
val res = mapper.writeValueAsString(m)
println(res)
Result:
{"empty":false,"traversableAgain":true}
I don't understand why I got that result.
As a non play solution, you can consider using json4s which provides a wrapper around jackson and its easy to use.
If you are using json4s then you can convert map to json just by using:
write(m)
//> res0: String = {"name":"john doe","age":18,"hasChild":true,"childs":[{"name":"dorothy","age":5,"hasChild":false},{"name":"bill","age":8,"hasChild":false}]}
--Updating to include the full example--
import org.json4s._
import org.json4s.native.Serialization._
import org.json4s.native.Serialization
implicit val formats = Serialization.formats(NoTypeHints)
val m = Map(
"name" -> "john doe",
"age" -> 18,
"hasChild" -> true,
"childs" -> List(
Map("name" -> "dorothy", "age" -> 5, "hasChild" -> false),
Map("name" -> "bill", "age" -> 8, "hasChild" -> false)))
write(m)
Output:
res0: String = {"name":"john doe","age":18,"hasChild":true,"childs":[{"name"
:"dorothy","age":5,"hasChild":false},{"name":"bill","age":8,"hasChild":false }]}
Alternative way:
import org.json4s.native.Json
import org.json4s.DefaultFormats
Json(DefaultFormats).write(m)
val mapper = new ObjectMapper()
mapper.writeValueAsString(Map("a" -> 1))
result> {"empty":false,"traversableAgain":true}
==============================
import com.fasterxml.jackson.module.scala.DefaultScalaModule
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
mapper.writeValueAsString(Map("a" -> 1))
result> {"a":1}
You need to tell jackson how to deal with scala objects: mapper.registerModule(DefaultScalaModule)
val mymap = array.map {
case 1 => ("A", 1)
case 2 => ("B", 2)
case 3 => ("C", 3)
}
.toMap
Using scala.util.parsing.json.JSONObject, you only need 1 line:
import scala.util.parsing.json.JSONObject
JSONObject(mymap).toString()
If you're working with a well-defined data model, why not define case classes and use Play JSON macros to handle conversion? i.e.
case class Person(name: String, age: Int, hasChild: Boolean, childs: List[Person])
implicit val fmt = Json.format[Person]
val person = Person(...)
val jsonStr = Json.toJson(person)
One thing you can do using the Jackson library is to use a java HashMap object, instead of a Scala one. Then you can basically use the same "without success" code you already wrote.
import org.codehaus.jackson.map.ObjectMapper
val mapper = new ObjectMapper()
val jmap = new java.util.HashMap[String, Int]()
jmap.put("dog", 4)
jmap.put("cat", 1)
// convert to json formatted string
val jstring = mapper.writeValueAsString(jmap)
println(jstring)
returns
jstring: String = {"dog":4,"cat":1}
In case somebody is looking for a solution using standard libraries.
def toJson(query: Any): String = query match {
case m: Map[String, Any] => s"{${m.map(toJson(_)).mkString(",")}}"
case t: (String, Any) => s""""${t._1}":${toJson(t._2)}"""
case ss: Seq[Any] => s"""[${ss.map(toJson(_)).mkString(",")}]"""
case s: String => s""""$s""""
case null => "null"
case _ => query.toString
}
Let's assume either of the following two JSON snippets:
{ "include": ["field1", "field2", "fieldN"] }
{ "exclude": ["field1", "field2", "fieldN"] }
I need to transform the include array like this...
{ "field1": 1, "field2": 1, "fieldN": 1 }
... and the exclude array like this:
{ "field1": 0, "field2": 0, "fieldN": 0 }
[Just for your info: I need to transform input JSON into Mongo's projections.]
Here below is my current solution – I've implemented it as a JsValue extension:
object testTypeExtensions {
implicit class TestJsExtensions(val json: JsValue) extends AnyVal {
def toProjection: JsValue = {
if (json \\ "include" nonEmpty)
JsObject(for (field <- (json \ "include").as[List[JsString]])
yield (field.value, JsNumber(1)))
else if (json \\ "exclude" nonEmpty)
JsObject(for (field <- (json \ "exclude").as[List[JsString]])
yield (field.value, JsNumber(0)))
else Json.obj()
}
}
}
The code above works...
scala> val p = Json.obj("exclude" -> Json.arr("field1", "field2"))
p: play.api.libs.json.JsObject = {"exclude":["field1","field2"]}
scala> p.toProjection
res12: play.api.libs.json.JsObject = {"field1":0,"field2":0}
... but I'm sure it could be written much better with JsZipper.
Furthermore it is not very flexible since it only manages the include and exclude keys, whereas I want to also manage other similar cases like sorting objects:
{ "asc": ["field1", "field2"] }
{ "desc": ["field1", "field2"] }
... transformed into...
{ "field1": 1, "field2": 1 }
... and
{ "field1": -1, "field2": -1 }
That said, what I've in mind is a generic method that manages any kind of named JSON array like:
object testypeExtensions {
implicit class TempJsExtensions(val json: JsValue) extends AnyVal {
def namedArrayToObject(keys: String*): JsValue = {
// how to implement it, possibly with JsZipper
}
}
}
The namedArrayToObject method should search for the specified keys in the current JSON and generate an object for the first match like the ones I described at the beginning of this post, possibly with JsZipper. Here is a simulation of the expected results.
Search for exclude and include and return the first match as a JsObject:
scala> val p = Json.obj("exclude" -> Json.arr("field1", "field2"))
scala> p.namedArrayToObject("exclude", "include")
res12: play.api.libs.json.JsObject = {"field1":0,"field2":0}
Same as before... but now input JSON contains include instead of exclude:
scala> val p = Json.obj("include" -> Json.arr("field1", "field2"))
scala> p.namedArrayToObject("exclude", "include")
res12: play.api.libs.json.JsObject = {"field1":1,"field2":1}
Search for asc and desc and return the first match as a JsObject:
scala> val p = Json.obj("desc" -> Json.arr("field1", "field2"))
scala> p.namedArrayToObject("asc", "desc")
res12: play.api.libs.json.JsObject = {"field1":-1,"field2":-1}
... and so on.
If there is no match, namedArrayToObject should return an empty JsObject. Any suggestion on how to implement this in the right way would be very appreciated.
You can do this pretty straightforwardly with JSON transformations:
import play.api.libs.json._
def toObj(value: Int) = Reads.of[List[String]].map(
keys => Json.toJson(keys.map(_ -> value).toMap)
)
val transformation =
(__ \ 'include).json.update(toObj(1)) andThen
(__ \ 'exclude).json.update(toObj(0))
We can define an example object and apply our transformation:
val example = Json.parse("""{
"include": ["field1", "field2", "field3"],
"exclude": ["field4", "field5", "field6"]
}""")
val transformed = example.transform(transformation)
And then:
scala> transformed.foreach(Json.prettyPrint _ andThen println)
{
"include" : {
"field1" : 1,
"field2" : 1,
"field3" : 1
},
"exclude" : {
"field4" : 0,
"field5" : 0,
"field6" : 0
}
}
This doesn't exactly match your desired API, but it should be easily adaptable, and I'd suggest staying away from the implicit class business, anyway—it's much less composable and makes handling invalid input less elegant.
Travis helped me to find the way... and here below eventually is my solution that does exactly what I was looking for:
object testExtensions {
implicit class testJsExtensions(val json: JsValue) extends AnyVal {
def toParams(pairs: (String, Int)*): JsValue = {
for (pair <- pairs) { json.getOpt(__ \ pair._1).map { values =>
JsObject(for (field <- values.as[List[JsString]])
yield (field.value, JsNumber(pair._2)))
} match {
case Some(params) => return params
case _ =>
}}
Json.obj()
}
}
}
scala> example.toParams(("include", 1), ("exclude", 0))
res63: play.api.libs.json.JsValue = {"field1":1,"field2":1,"field3":1}
scala> example.toParams(("exclude", 0))
res64: play.api.libs.json.JsValue = {"field4":0,"field5":0,"field6":0}
Again a bit thanks to Travis :-)