I'm reading data from a KV store (Redis) in this case. The data returned is in the following format.
{ "key1":"value1", "key2":"value2", "key3":"value3" ...}
Key is String and value is Int. I want to convert it into a Map[String,Int]
I looked at the json4s JSON API and my current code looks like the following. Is there a better/easier/cleaner way of doing this?
//send a async query to Redis to
val queryFuture = redis.zrangebyscore[String](tablename, keymin, keymax )
queryFuture.onComplete {
case Success(datarows) =>
println(s"Got %d rows of type %s for the query successfully".format(datarows.length))
val jsonrows = for { x <- datarows.toList }
yield parse(x)
println("Got json rows %d".format(jsonrows.size))
val mapdata = jsonrows.map(x => x.extract[Map[String,String]]).map( x => x.mapValues(_.toInt))
//need to do something with this data now
case Failure(y) =>
println(s" there was some failure in getting the data from Redis")
}
This looks to me like the simplest way to do it:
val map = parse("""{"a":"1","b":"2","c":"3"}""")
.children
.collect { case JField(k, JString(v)) => (k, v.toInt) }
.toMap
Your Json4s solution looks fine. Alternatively you can use mapField to transform the fields of a JObject and after that extract value of type Map[String, Int].
val json1 = parse(
"""
|{
| "key1": "1024",
| "key2": "2048",
| "key3": "4096"
|}
""".stripMargin)
val json2 = json1.mapField {
case (key, JString(value)) => (key, JInt(value.toInt))
case x => x
}
val res = json2.extract[Map[String, Int]]
println(res)
// Map(key1 -> 1024, key2 -> 2048, key3 -> 4096)
Not knowing json4s, and unfortunately you ommited the types, but guessing that jsonrows is probably something like a List[(String, String)] you could do
List(("key1" -> "1"),("key2" -> "2")).map { case (k, v) => (k, v.toInt)}.toMap
BTW, if you say need to do something with this data now in your onComplete - that could only be a side effecting operation. Better map over the future until your processing is complete.
Related
So I have a list of maps with dynamic field and values.
E.g.
val sampleMap = List(
Map("field1" -> 1, "field2" -> "helloworld"),
Map("field3" -> "abcd", "field4" -> 123.34212543))
So basically, I have a variable that is List[Map[String, Object]].
How can I convert this whole thing into a JSON circe?
Nevermind, I found the answer.
Basically we need to match the datatype of the object and convert it to JSON.
def toJson(obj: Object): Json = {
obj match {
case null => Json.Null
case b: java.lang.Boolean => Json.fromBoolean(b)
case i: java.lang.Integer => Json.fromInt(i)
case d: java.lang.Double =>
Json.fromDouble(d).getOrElse(Json.fromDouble(0d).get)
case l: java.lang.Long => Json.fromLong(l)
case t: TemporalAccessor =>
Json.fromString(DtFormatter.format(t))
case u => Json.fromString(String.valueOf(u))
}
}
I am new to scala and I am trying to parse a JSON shown below
val result = JSON.parseFull("""
{"name": "Naoki", "lang": ["Java", "Scala"] , "positionandvalue": ["5:21", "6:24", "7:6"]}
""")
result: Option[Any] = Some(Map(name -> Naoki, lang -> List(Java, Scala), positionandvalue -> List(5:21, 6:24, 7:6)))
And get the parsed values in a Map
val myMap = result match {
case Some(e) => e
case None => None
}
myMap: Any = Map(name -> Naoki, lang -> List(Java, Scala), positionandvalue -> List(5:21, 6:24, 7:6))
What I need is
1. To get the key as a new variable (to be used as metadata to validate the file) with its corresponding value assigned to it. Something like,
val name = "Naoki"
positionandvalue -> List(5:21, 6:24, 7:6). This variable indicates the List of(Position of string delimited a in file:length of string in position). How can I use this variable to satisfy the requirement.
you cannot dynamically create the variables name and positionandvalue from the Map key. However they can be statically created using the below approach.
val result: Option[Any] = Some(Map("name" -> "Naoki", "lang" -> List("Java", "Scala"), "positionandvalue" -> List("5:21", "6:24", "7:6")))
val myMap: Map[String, Any] = result match {
case Some(e: Map[String, Any] #unchecked) => e
case _ => Map()
}
val name = myMap.get("name") match {
case Some(x: String) => x
case _ => throw new RuntimeException("failure retrieving name key")
}
val positionandvalue = myMap.get("positionandvalue") match {
case Some(x: List[String] #unchecked) => x.map(y => (y.split(":") match {case Array(x1,x2) => x1 -> x2})).toMap
case _ => throw new RuntimeException("failure retrieving positionandvalue key")
}
positionandvalue: scala.collection.immutable.Map[String,String] = Map(5 -> 21, 6 -> 24, 7 -> 6)
If I have a JSON object like:
{
"test": 3
}
Then I would expect that extracting the "test" field as a String would fail because the types don't line up:
import org.json4s._
import org.json4s.jackson.JsonMethods
import org.json4s.JsonAST.JValue
def getVal[T: Manifest](json: JValue, fieldName: String): Option[T] = {
val field = json findField {
case JField(name, _) if name == fieldName => true
case _ => false
}
field.map {
case (_, value) => value.extract[T]
}
}
val json = JsonMethods.parse("""{"test":3}""")
val value: Option[String] = getVal[String](json, "test") // Was Some(3) but expected None
Is this automatic conversion from a JSON numeric to a String expected in Json4s? If so, are there any workarounds for this where the extracted field has to be of the same type that is specified in the type parameter to the extract method?
This is the default nature of most if not all of the parsers. If you request a value of type T and if the value can be safely cast to that specific type then the library would cast it for you. for instance take a look at the typesafe config with the similar nature of casting Numeric field to String.
import com.typesafe.config._
val config = ConfigFactory parseString """{ test = 3 }"""
val res1 = config.getString("test")
res1: String = 3
if you wanted not to automatically cast Integer/Boolean to String you could do something like this manually checking for Int/Boolean types as shown below.
if(Try(value.extract[Int]).isFailure || Try(value.extract[Boolean]).isFailure) {
throw RuntimeException(s"not a String field. try Int or Boolean")
} else {
value.extract[T]
}
One simple workaround is to create a custom serializer for cases where you want "strict" behavior. For example:
import org.json4s._
val stringSerializer = new CustomSerializer[String](_ => (
{
case JString(s) => s
case JNull => null
case x => throw new MappingException("Can't convert %s to String." format x)
},
{
case s: String => JString(s)
}
))
Adding this serializer to your implicit formats ensures the strict behavior:
implicit val formats = DefaultFormats + stringSerializer
val js = JInt(123)
val str = js.extract[String] // throws MappingException
I am using Scala with Play. I have a JSON file with all the countries in the world and their respective cities. The JSON looks like this:
{
"CountryA": ["City1","city2"],
"CountryB": ["City1"]
}
I parse it accordingly:
val source: String = Source.fromFile("app/assets/jsons/countriesToCities.json").getLines.mkString
val json: JsValue = Json.parse(source)
My ultimate goal is to convert the json contents into a Scala MultiMap where the key is a String - the country, and the value is a Set[String] - the cities.
Thanks in advance!
Here's a long-winded solution that won't throw if the JSON structure doesn't match what you're expecting:
val source: String =
"""
|{
| "CountryA": ["City1","city2"],
| "CountryB": ["City1"]
|}
""".stripMargin
val json: JsValue = Json.parse(source)
import scala.collection.breakOut
val map: Map[String, Set[String]] = json.asOpt[JsObject] match {
case Some(obj) =>
obj.fields.toMap.mapValues { v =>
v.asOpt[JsArray] match {
case Some(JsArray(cities)) => cities.flatMap(_.asOpt[String])(breakOut)
case _ => Set.empty[String]
}
}
case _ => Map.empty[String, Set[String]]
}
map must beEqualTo(Map("CountryA" -> Set("City1", "city2"), "CountryB" -> Set("City1")))
If you're confident about the structure of the JSON and don't mind using as (which could throw) instead of asOpt (which won't):
val map2: Map[String, Set[String]] = {
json.as[JsObject].fields.toMap.mapValues {
_.as[JsArray].value.map(_.as[String])(breakOut)
}
}
map2 must beEqualTo(Map("CountryA" -> Set("City1", "city2"), "CountryB" -> Set("City1")))
I have some history data, that I want to convert to json. So these are Lists of lists. Their type is List[List[Position]] where Position is a simple case class. I wrote a formatter to help Json.toJson cope. I was expecting an output of exactly one outer array with two inner arrays that contain 3 objects each. What I got instead was this. Note the additional array wrappings.
[[[[{"amount":1.0,"minAmount":2.0,"price":3.0,"volume":4.0},
{"amount":5.0,"minAmount":6.0,"price":7.0,"volume":8.0},
{"amount":9.0,"minAmount":10.0,"price":11.0,"volume":12.0}]],
[[{"amount":0.1,"minAmount":0.2,"price":0.3,"volume":0.4},
{"amount":5.0,"minAmount":6.0,"price":7.0,"volume":8.0},
{"amount":9.0,"minAmount":10.0,"price":11.0,"volume":12.0}]]]]
I don't know where the wrapping arrays come from. Can somebody help me out here? This is a test with the wrapper I am using:
class ApplicationSpec extends Specification {
implicit object PositionFormat extends Format[List[List[Position]]] {
def writes(historyList: List[List[Position]]) : JsValue = {
Json.arr(historyList.map{
o => Json.arr(o.map{ p =>
Json.obj(
"amount" -> JsNumber(p.amount),
"minAmount" -> JsNumber(p.minAmount),
"price" -> JsNumber(p.price),
"volume" -> JsNumber(p.volume)
)
})
})
}
def reads(json: JsValue): JsResult[List[List[Position]]] = ???
}
"Application" should {
"Convert position data to json" in {
val l1 = ListBuffer(new Position(1.0D,2.0D,3.0D,4.0D),
new Position(5.0D,6.0D,7.0D,8.0D),
new Position(9.0D,10.0D,11.0D,12.0D)).toList
val l2 = ListBuffer(new Position(0.1D,0.2D,0.3D,0.4D),
new Position(5.0D,6.0D,7.0D,8.0D),
new Position(9.0D,10.0D,11.0D,12.0D)).toList
val obj = ListBuffer(l1,l2).toList
val json = Json.toJson(obj)
var string: String = json.toString()
println(string)
}
}
}
It seems that Json.arr takes it's arguments and returns a JsValue for a JSON array of them. It looks like you could simply do with Json.toJson:
Here's how arr is meant to be used:
// Json.arr
Json.arr("one", "two")
// Gives you
// play.api.libs.json.JsArray = ["one","two"]
If you instead do:
// vs
val l = List("one", "two")
Json.arr(l)
// Gives you
// play.api.libs.json.JsArray = [["one","two"]]
// ... a nested array, which is what you don't want.
What you need is:
// Json.toJson
Json.toJson(l)
// Gives you:
// play.api.libs.json.JsValue = ["one","two"]