I need to convert values of type Distribution.PackageDescription.FlagName to a JSON object with the Text.JSON of the json package.
I ended up with the following approach:
instance JSON FlagName where
showJSON (FlagName n) = makeObj [ ("FlagName", showJSON n) ]
readJSON object = do
obj <- readJSON object
flag <- valFromObj "FlagName" obj
return flag
When I try to encode a value and decode it again the following happens:
> showJSON (FlagName "foo")
JSObject (JSONObject {fromJSObject = [("FlagName",JSString (JSONString {fromJSString = "foo"}))]})
> readJSON (showJSON (FlagName "foo")) :: Result FlagName
Error "Unable to read JSObject"
I guess the error is in that line: obj <- readJSON object
How do I force Haskell to use the readJSON function from the JSON String instance ?
Update: I found now a rather hackish solution:
instance JSON FlagName where
showJSON (FlagName n) = makeObj [ ("FlagName", showJSON n) ]
readJSON object = do
obj <- readJSON (showJSON (FlagName "foo")) :: Result (JSObject JSValue)
let maybeFlagName = lookup "FlagName" $ fromJSObject obj
maybe (fail "Not a FlagName object") (\jsn -> liftM FlagName $ (readJSON jsn :: Result String)) maybeFlagName
I'd appreciate it if someone comes up with a more elegant solution ...
Ok, I found the answer myself:
valFromObj returns the name of the FlagName (i.e. a String) rather than the FlagName itself.
instance JSON FlagName where
showJSON (FlagName n) = makeObj [ ("FlagName", showJSON n) ]
readJSON object = do
obj <- readJSON object
n <- valFromObj "FlagName" obj
return $ FlagName n
Better to pattern match so you can handle the case when object is no of JSObject. Instead of fail you can do some other thing. You will need to import Control.Applicative to use <$>. I just like the applicative syntax better for such things.
instance JSON FlagName where
showJSON (FlagName n) = makeObj [ ("FlagName", showJSON n) ]
readJSON (JSObject obj) = FlagName <$> valFromObj "FlagName" obj
readJSON _ = fail "unknown object"
Related
i am trying to write code to mask nested json fields..
def maskRecursively(map :mutable.Map[String,Object]):mutable.Map[String,Object] ={
val maskColumns = PII_Data.getPIIData()
for((k,v) <- map){
if(v.isInstanceOf[Map[String,Object]]){
maskRecursively(map.get(k).asInstanceOf[mutable.Map[String,Object]])
}else if(v.isInstanceOf[List[Object]]) {
} else {
if(maskColumns.contains(k)){map+=(k->"*****")}
}
}
map }
calling this method from ..
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
val result = mapper.readValue(jsonStr, classOf[ java.util.Map[String,Object] ])
import scala.collection.JavaConverters._
val myScalaMap = result.asScala
maskRecursively(result.asScala)
i am getting error while trying to iterate a nested json object ..
Cannot cast value of type 'scala.Some' to type 'scala.collection.mutable.Map'
how do i recurse a complex nested json object this way ?
Your mistake was
if(v.isInstanceOf[Map[String,Object]]){
maskRecursively(map.get(k).asInstanceOf[mutable.Map[String,Object]])
There are a few issues:
You check if v is an instance of Map, but then attempt to cast it to mutable.Map. They are technically different types (mutable vs immutable).
You check the type of v, but then apply the cast to map.get(k), which is going to be a different value and type from v. A map's get method returns an Option, hence the error message.
Thanks to type erasure on the JVM, the runtime won't be able to tell the difference between e.g. a Map[String, Object] and a Map[SomethingElse, Whatever] - both will just look like Map at runtime. The compiler should have given you a warning about the isInstanceOf call for this reason.
If you do an isInstanceOf / asInstanceOf combo, make sure the operand is the same each time. You already have v, so you don't need to look it up a second time from the map. And make sure you use the same type on both instanceOf calls.
Fix this by changing it to
if(v.isInstanceOf[mutable.Map[_, _]]){
maskRecursively(v.asInstanceOf[mutable.Map[String,Object]])
After some digging , i was able to solve this..
def maskJson(jsonStr: String): String = {
implicit val formats = org.json4s.DefaultFormats
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
val result = mapper.readValue(jsonStr, classOf[Map[String, Object]])
val maskedJson = maskRecursively(result)
mapper.writeValueAsString(maskedJson)
}
def maskRecursively(map: Map[String, Object]): collection.mutable.Map[String, Object] = {
val mutable = collection.mutable.Map[String, Object]()
val maskColumns = PII_Data.getJsonPIIFields()
for ((k, v) <- map) {
if (v.isInstanceOf[Map[String, Object]]) {
mutable += k -> maskRecursively(v.asInstanceOf[Map[String, Object]])
} else if (v.isInstanceOf[List[Object]]) {
val list = v.asInstanceOf[List[Map[String, Object]]].map(i => maskRecursively(i)).toList
mutable += k -> list
} else {
if (maskColumns.contains(k)) {
mutable += (k -> "*****")
}
else {
mutable += k -> v
}
}
}
mutable
}
I have a JSON structure like this
{
"tag1": 1,
"tag2": 7,
...
}
And I have a type like this
data TagResult { name :: String, numberOfDevicesTagged :: Int } deriving (Show, Eq)
newtype TagResultList = TagResultList { tags :: [TagResult] }
The tag names are of course fully dynamic and I don't know them at compile time.
I'd like to create an instance FromJSON to parse the JSON data but I just cannot make it compile.
How can I define parseJSON to make this happen?
You can use the fact that Object is an HasMap and extract the key at runtime. You can then write the FromJSON instance as follows -
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Data.Aeson
import qualified Data.Text as T
import qualified Data.HashMap.Lazy as HashMap
data TagResult = TagResult { name :: String
, numberOfDevicesTagged :: Int
} deriving (Show, Eq)
newtype TagResultList = TagResultList { tags :: [TagResult] } deriving Show
instance ToJSON TagResult where
toJSON (TagResult tag ntag) =
object [ T.pack tag .= ntag ]
instance ToJSON TagResultList where
toJSON (TagResultList tags) =
object [ "tagresults" .= toJSON tags ]
instance FromJSON TagResult where
parseJSON (Object v) =
let (k, _) = head (HashMap.toList v)
in TagResult (T.unpack k) <$> v .: k
parseJSON _ = fail "Invalid JSON type"
instance FromJSON TagResultList where
parseJSON (Object v) =
TagResultList <$> v .: "tagresults"
main :: IO ()
main = do
let tag1 = TagResult "tag1" 1
tag2 = TagResult "tag2" 7
taglist = TagResultList [tag1, tag2]
let encoded = encode taglist
decoded = decode encoded :: Maybe TagResultList
print decoded
The above program should print the tag result list.
Just (TagResultList {tags = [TagResult {name = "tag1", numberOfDevicesTagged = 1},TagResult {name = "tag2", numberOfDevicesTagged = 7}]})
Suppose I am using json4s to parse JSON:
val str = """{"a":"aaaa", "x": 0}"""
val json = JsonMethods.parse(str)
val a = for(JObject(fields) <- json; JField("a", JString(a)) <- fields) yield a
The type of a is List[String] but I need Option[String], so I am calling headOption:
val a = (
for(JObject(fields) <- json; JField("a", JString(a)) <- fields) yield a
).headOption
Since I found myself calling headOption again and again I tried an implicit conversion:
object X { implicit def foo[A](as: List[A]): Option[A] = as.headOption }
import X.foo
val a: Option[String] =
for(JObject(fields) <- json; JField("a", JString(a)) <- fields) yield a
The implicit conversion is working but I don't like it. What would you suggest ?
One approach is to use json4's typeclasses, e.g. json4s-scalaz has those:
trait JSONR[A] {
def read(json: JValue): Result[A]
}
trait JSONW[A] {
def write(value: A): JValue
}
source
For syntactic simplicity one can define extensions methods for JValue:
implicit class JValueOps(value: JValue) {
def validate[A: JSONR]: ValidationNel[Error, A] = implicitly[JSONR[A]].read(value)
def read[A: JSONR]: Error \/ A = implicitly[JSONR[A]].read(value).disjunction.leftMap(_.head)
}
And then do a traversal and also parse the resulting JValue of the traversal like this:
val str =
"""
|{
| "a": "aaaa",
| "x": 0
|}""".stripMargin
val json = parseJson(str)
(json \ "a").read[Option[String]]
// \/-(Some(aaaa))
(json \ "b").read[Option[String]]
// \/-(None)
(json \ "a").validate[Option[String]]
// Success(Some(aaaa))
(json \ "b").validate[Option[String]]
// Success(None)
Defining your own JSONR[A]/JSONW[A] instances (and putting them in implicit scope) is possible like this:
case class MyA(a: Option[String], x: Int)
implicit val myARead: JSONR[MyA] = JSON.readE[MyA] { json =>
for {
a <- (json \ "a").read[Option[String]]
x <- (json \ "x").read[Int]
} yield MyA(a, x)
}
implicit val myAWrite: JSONW[MyA] = JSON.write[MyA] { myA =>
("a" -> myA.a) ~
("x" -> myA.x)
}
json.read[MyA]
// \/-(MyA(Some(aaaa),0))
json.validate[MyA]
// Success(MyA(Some(aaaa),0))
MyA(Some("aaaa"), 0).toJson
// JObject(List((a,JString(aaaa)), (x,JInt(0))))
Note that the read[A] and write[a] methods are glue code which is not in json4s-scalaz available yet, you can find the source here. There are also more examples.
Then json.read[A] returns a Error \/ A and json.validate[A] yields a Validation types from scalaz. There are similar types in cats.
myARead is an example of monadic parsing style (compose via flatMap). An alternative uses applicative parsing. This has the benefit that all validation errors are accumulated:
val myARead2: JSONR[MyA] = JSON.read[MyA] { json =>
(
(json \ "a").validate[Option[String]] |#|
(json \ "x").validate[Int]
).tupled.map(MyA.tupled)
}
val myARead3: JSONR[MyA] = JSON.read[MyA] {
for {
a <- field[Option[String]]("a") _
x <- field[Int]("x") _
} yield (a |#| x).tupled.map(MyA.tupled)
}
There is also https://github.com/json4s/json4s/blob/3.4/core/src/main/scala/org/json4s/JsonFormat.scala
I'm reading data from a KV store (Redis) in this case. The data returned is in the following format.
{ "key1":"value1", "key2":"value2", "key3":"value3" ...}
Key is String and value is Int. I want to convert it into a Map[String,Int]
I looked at the json4s JSON API and my current code looks like the following. Is there a better/easier/cleaner way of doing this?
//send a async query to Redis to
val queryFuture = redis.zrangebyscore[String](tablename, keymin, keymax )
queryFuture.onComplete {
case Success(datarows) =>
println(s"Got %d rows of type %s for the query successfully".format(datarows.length))
val jsonrows = for { x <- datarows.toList }
yield parse(x)
println("Got json rows %d".format(jsonrows.size))
val mapdata = jsonrows.map(x => x.extract[Map[String,String]]).map( x => x.mapValues(_.toInt))
//need to do something with this data now
case Failure(y) =>
println(s" there was some failure in getting the data from Redis")
}
This looks to me like the simplest way to do it:
val map = parse("""{"a":"1","b":"2","c":"3"}""")
.children
.collect { case JField(k, JString(v)) => (k, v.toInt) }
.toMap
Your Json4s solution looks fine. Alternatively you can use mapField to transform the fields of a JObject and after that extract value of type Map[String, Int].
val json1 = parse(
"""
|{
| "key1": "1024",
| "key2": "2048",
| "key3": "4096"
|}
""".stripMargin)
val json2 = json1.mapField {
case (key, JString(value)) => (key, JInt(value.toInt))
case x => x
}
val res = json2.extract[Map[String, Int]]
println(res)
// Map(key1 -> 1024, key2 -> 2048, key3 -> 4096)
Not knowing json4s, and unfortunately you ommited the types, but guessing that jsonrows is probably something like a List[(String, String)] you could do
List(("key1" -> "1"),("key2" -> "2")).map { case (k, v) => (k, v.toInt)}.toMap
BTW, if you say need to do something with this data now in your onComplete - that could only be a side effecting operation. Better map over the future until your processing is complete.
I'm trying to convert Scala to JSON in the 2.1RC Play Framework.
I can do the following and get JSON:
import play.api.libs.json._
val a1=Map("val1"->"a", "val2"->"b")
Json.toJSon(a1)
Because a1 is just Map[String,String] that works OK.
But if I have something more complex like where I have Map[String,Object], that doesn't work:
val a = Map("val1" -> "xxx", "val2"-> List("a", "b", "c"))
Json.toJSon(a1)
>>> error: No Json deserializer found for type scala.collection.immutable.Map[String,Object]
I found that I can do something like the following:
val a2 = Map("val1" -> Json.toJson("a"), "val2" -> Json.toJson(List("a", "b", "c")))
Json.toJson(a2)
And that works.
But how can I do that in a general way? I thought that I could do something like the following:
a.map{ case(k,v)=> (k, Json.toJson(v) )}
>>> error: No Json deserializer found for type Object
But I still get an error that it can't be deserialized
Additional Information:
Json.toJson can convert a Map[String, String] to a JsValue:
scala> val b = Map( "1" -> "A", "2" -> "B", "3" -> "C", "4" -> "D" )
b: scala.collection.immutable.Map[String,String] = Map(1 -> A, 2 -> B, 3 -> C, 4 -> D)
scala> Json.toJson(b)
res31: play.api.libs.json.JsValue = {"1":"A","2":"B","3":"C","4":"D"}
But, it fails in trying to convert a Map[String, Object]:
scala> a
res34: scala.collection.immutable.Map[String,Object] = Map(val1 -> xxx, val2 -> List(a, b, c))
scala> Json.toJson(a)
<console>:12: error: No Json deserializer found for type scala.collection.immutable.Map[String,Object]. Try to implement an implicit Writes or Format for this type.
Json.toJson(a)
Using the 'hint' from this Play Framework page on converting Scala to Json, I found the following (http://www.playframework.org/documentation/2.0.1/ScalaJson):
If instead of Map[String, Object], there is a Map[String, JsValue], then Json.toJson() will work:
scala> val c = Map("aa" -> Json.toJson("xxxx"), "bb" -> Json.toJson( List("11", "22", "33") ) )
c: scala.collection.immutable.Map[String,play.api.libs.json.JsValue] = Map(aa -> "xxxx", bb -> ["11","22","33"])
scala> Json.toJson(c)
res36: play.api.libs.json.JsValue = {"aa":"xxxx","bb":["11","22","33"]}
So, what I would like, is that given a Map[String, Object], where I know that the Object values were all originally of type String or List[String], how to apply the function Json.toJson() to the all the values in the map and get a Map[String, JsValue].
I also found that I can filter out those values that are purely string and those that are (were) of type List[String]:
scala> val a1 = a.filter({case(k,v) => v.isInstanceOf[String]})
a1: scala.collection.immutable.Map[String,Object] = Map(val1 -> xxx)
scala> val a2 = a.filter({case(k,v) => v.isInstanceOf[List[String]]})
<console>:11: warning: non-variable type argument String in type List[String] is unchecked since it is eliminated by erasure
val a2 = a.filter({case(k,v) => v.isInstanceOf[List[String]]})
^
a2: scala.collection.immutable.Map[String,Object] = Map(val2 -> List(a, b, c))
The List[String] filtering gives a warning, but seems to give the answer I want. If the two filters could be applied and then Json.toJson() used on the values of the result, and the results combined, maybe that would work?
But the filtered results are still of type Map[String, Object] which causes a problem:
scala> Json.toJson(a1)
<console>:13: error: No Json deserializer found for type scala.collection.immutable.Map[String,Object]. Try to implement an implicit Writes or Format for this type.
Json.toJson(a1)
Play 2.1 JSON API does not provide a serializer for the Type Map[String, Ojbect].
Define case class and Format for the specific type instead of Map[String, Object]:
// { "val1" : "xxx", "val2" : ["a", "b", "c"] }
case class Hoge(val1: String, val2: List[String])
implicit val hogeFormat = Json.format[Hoge]
If you don't want to create case class.
The following code provides JSON serializer/deserializer for Map[String, Object]:
implicit val objectMapFormat = new Format[Map[String, Object]] {
def writes(map: Map[String, Object]): JsValue =
Json.obj(
"val1" -> map("val1").asInstanceOf[String],
"val2" -> map("val2").asInstanceOf[List[String]]
)
def reads(jv: JsValue): JsResult[Map[String, Object]] =
JsSuccess(Map("val1" -> (jv \ "val1").as[String], "val2" -> (jv \ "val2").as[List[String]]))
}
More dynamically
import play.api.libs.json._
import play.api.libs.json.Reads._
import play.api.libs.json.Json.JsValueWrapper
implicit val objectMapFormat = new Format[Map[String, Object]] {
def writes(map: Map[String, Object]): JsValue =
Json.obj(map.map{case (s, o) =>
val ret:(String, JsValueWrapper) = o match {
case _:String => s -> JsString(o.asInstanceOf[String])
case _ => s -> JsArray(o.asInstanceOf[List[String]].map(JsString(_)))
}
ret
}.toSeq:_*)
def reads(jv: JsValue): JsResult[Map[String, Object]] =
JsSuccess(jv.as[Map[String, JsValue]].map{case (k, v) =>
k -> (v match {
case s:JsString => s.as[String]
case l => l.as[List[String]]
})
})
}
Sample code:
val jv = Json.toJson(Map("val1" -> "xxx", "val2" -> List("a", "b", "c"), "val3" -> "sss", "val4" -> List("d", "e", "f")))
println(jv)
val jr = Json.fromJson[Map[String, Object]](jv)
println(jr.get)
The output:
> {"val1":"xxx","val2":["a","b","c"],"val3":"sss","val4":["d","e","f"]}
> Map(val1 -> xxx, val2 -> List(a, b, c), val3 -> sss, val4 -> List(d, e, f))