I am using Scala with Play. I have a JSON file with all the countries in the world and their respective cities. The JSON looks like this:
{
"CountryA": ["City1","city2"],
"CountryB": ["City1"]
}
I parse it accordingly:
val source: String = Source.fromFile("app/assets/jsons/countriesToCities.json").getLines.mkString
val json: JsValue = Json.parse(source)
My ultimate goal is to convert the json contents into a Scala MultiMap where the key is a String - the country, and the value is a Set[String] - the cities.
Thanks in advance!
Here's a long-winded solution that won't throw if the JSON structure doesn't match what you're expecting:
val source: String =
"""
|{
| "CountryA": ["City1","city2"],
| "CountryB": ["City1"]
|}
""".stripMargin
val json: JsValue = Json.parse(source)
import scala.collection.breakOut
val map: Map[String, Set[String]] = json.asOpt[JsObject] match {
case Some(obj) =>
obj.fields.toMap.mapValues { v =>
v.asOpt[JsArray] match {
case Some(JsArray(cities)) => cities.flatMap(_.asOpt[String])(breakOut)
case _ => Set.empty[String]
}
}
case _ => Map.empty[String, Set[String]]
}
map must beEqualTo(Map("CountryA" -> Set("City1", "city2"), "CountryB" -> Set("City1")))
If you're confident about the structure of the JSON and don't mind using as (which could throw) instead of asOpt (which won't):
val map2: Map[String, Set[String]] = {
json.as[JsObject].fields.toMap.mapValues {
_.as[JsArray].value.map(_.as[String])(breakOut)
}
}
map2 must beEqualTo(Map("CountryA" -> Set("City1", "city2"), "CountryB" -> Set("City1")))
Related
I've got a List which holds some Personalization-objects. The latter is defined like this:
sealed case class Personalization(firstname: String, lastname: String, keycardId: String)
I need to map this list to a Json-Array structure which has to look like this:
"personalization": [
{
"id": "firstname",
"value": "John"
},
{
"id": "lastname",
"value": "Doe"
}...
I am struggling with the part of mapping the field information to id/value pairs. Normally, I would create a play.api.libs.json.Format out of the Personalization class and let it map automatically -> Json.format[Personalization] - but this time, I need to create an array where an entry can hold n attributes.
Therefore I am asking for advice, if there is a possibility to use the Scala Play-Framework?
Any input is much appreciated, thank you!
Writing as such JSON representation is not quite complex, using Writes.transform.
import play.api.libs.json._
case class Personalization(firstname: String, lastname: String, keycardId: String) // No need to seal a case class
implicit def writes: Writes[Personalization] = {
val tx: JsValue => JsValue = {
case JsObject(fields) => Json.toJson(fields.map {
case (k, v) => Json.obj("id" -> k, "value" -> v)
})
case jsUnexpected => jsUnexpected // doesn't happen with OWrites
}
Json.writes[Personalization].transform(tx)
}
Which can be tested as bellow.
val personalization = Personalization(
firstname = "First",
lastname = "Last",
keycardId = "Foo")
val jsonRepr = Json.toJson(personalization)
// => [{"id":"firstname","value":"First"},{"id":"lastname","value":"Last"},{"id":"keycardId","value":"Foo"}]
Reading is a little bit tricky:
implicit def reads: Reads[Personalization] = {
type Field = (String, Json.JsValueWrapper)
val fieldReads = Reads.seq(Reads[Field] { js =>
for {
id <- (js \ "id").validate[String]
v <- (js \ "value").validate[JsValue]
} yield id -> v
})
val underlying = Json.reads[Personalization]
Reads[Personalization] { js =>
js.validate(fieldReads).flatMap { fields =>
Json.obj(fields: _*).validate(underlying)
}
}
}
Which can be tested as bellow.
Json.parse("""[
{"id":"firstname","value":"First"},
{"id":"lastname","value":"Last"},
{"id":"keycardId","value":"Foo"}
]""").validate[Personalization]
// => JsSuccess(Personalization(First,Last,Foo),)
Note that is approach can be used for any case class format.
Probably it is possible to do it a more elegant way I did, but you can use the following snippet:
case class Field(id: String, value: String)
object Field {
implicit val fieldFormatter: Format[Field] = Json.format[Field]
}
sealed case class Personalization(firstname: String, lastname: String, keycardId: String)
object Personalization {
implicit val personalizationFormatter: Format[Personalization] = new Format[Personalization] {
override def reads(json: JsValue): JsResult[Personalization] =
Try {
val data = (json \ "personalization").as[JsValue]
data match {
case JsArray(value) =>
val fields = value.map(_.as[Field]).map(f => f.id -> f.value).toMap
val firstname = fields.getOrElse("firstname", throw new IllegalArgumentException("Mandatory field firstname is absent."))
val lastname = fields.getOrElse("lastname", throw new IllegalArgumentException("Mandatory field lastname is absent."))
val keycardId = fields.getOrElse("keycardId", throw new IllegalArgumentException("Mandatory field keycardId is absent."))
Personalization(firstname, lastname, keycardId)
case _ => throw new IllegalArgumentException("Incorrect json format for Personalization.")
}
}.toEither.fold(e => JsError(e.getMessage), JsSuccess(_))
override def writes(o: Personalization): JsValue = {
val fields = List(Field("firstname", o.firstname), Field("lastname", o.lastname), Field("keycardId", o.keycardId))
JsObject(List("personalization" -> Json.toJson(fields)))
}
}
}
It converts {"personalization":[{"id":"firstname","value":"John"},{"id":"lastname","value":"Doe"},{"id":"keycardId","value":"1234"}]} to Personalization(John,Doe,1234) and vice versa
I am new to Scala. I want to parse JSON data in scala store to database table.
My GET method looks like this (Please ignore the permissions):
def Classes = withAuth { username =>
implicit request =>
User.access(username, User.ReadXData).map { user =>
implicit val writer = new Writes[Class] {
def writes(entry: Class): JsValue = Json.obj(
"id" -> entry.id,
"name" -> entry.name
)
}
val classes = (Class.allAccessible(user))
Ok(Json.obj("success" -> true, "classes" -> classes))
}.getOrElse(Forbidden(Application.apiMessage("Not authorised"))) }
This GET method returns the json below:
"success":true,"schools":[{"id":93,"name":"Happy unniversity",}]}
I'm currently rendering the JSOn in a datatables js (editor) grid - with success
HOWEVER, I'm unable to parse and POST the JSON and store it to the database (mysql) table.
Thank you for your guidance!
Looks you are using play-json.
For class User
import play.api.libs.json.Json
final case class User(id: String, name: String)
object User {
implicit val userFormat = Json.format[User]
}
object UserJson {
def main(args: Array[String]): Unit = {
val user = User("11", "Peter")
val json = Json.toJson(user).toString()
println("json ===> " + json)
val user2 = Json.parse(json).as[User]
println("name ===> " + user2.name)
}
}
I definitely recommend this lib: "de.heikoseeberger" %% "akka-http-jackson" % "1.27.0" for akka-http.
If I have a JSON object like:
{
"test": 3
}
Then I would expect that extracting the "test" field as a String would fail because the types don't line up:
import org.json4s._
import org.json4s.jackson.JsonMethods
import org.json4s.JsonAST.JValue
def getVal[T: Manifest](json: JValue, fieldName: String): Option[T] = {
val field = json findField {
case JField(name, _) if name == fieldName => true
case _ => false
}
field.map {
case (_, value) => value.extract[T]
}
}
val json = JsonMethods.parse("""{"test":3}""")
val value: Option[String] = getVal[String](json, "test") // Was Some(3) but expected None
Is this automatic conversion from a JSON numeric to a String expected in Json4s? If so, are there any workarounds for this where the extracted field has to be of the same type that is specified in the type parameter to the extract method?
This is the default nature of most if not all of the parsers. If you request a value of type T and if the value can be safely cast to that specific type then the library would cast it for you. for instance take a look at the typesafe config with the similar nature of casting Numeric field to String.
import com.typesafe.config._
val config = ConfigFactory parseString """{ test = 3 }"""
val res1 = config.getString("test")
res1: String = 3
if you wanted not to automatically cast Integer/Boolean to String you could do something like this manually checking for Int/Boolean types as shown below.
if(Try(value.extract[Int]).isFailure || Try(value.extract[Boolean]).isFailure) {
throw RuntimeException(s"not a String field. try Int or Boolean")
} else {
value.extract[T]
}
One simple workaround is to create a custom serializer for cases where you want "strict" behavior. For example:
import org.json4s._
val stringSerializer = new CustomSerializer[String](_ => (
{
case JString(s) => s
case JNull => null
case x => throw new MappingException("Can't convert %s to String." format x)
},
{
case s: String => JString(s)
}
))
Adding this serializer to your implicit formats ensures the strict behavior:
implicit val formats = DefaultFormats + stringSerializer
val js = JInt(123)
val str = js.extract[String] // throws MappingException
I'm reading data from a KV store (Redis) in this case. The data returned is in the following format.
{ "key1":"value1", "key2":"value2", "key3":"value3" ...}
Key is String and value is Int. I want to convert it into a Map[String,Int]
I looked at the json4s JSON API and my current code looks like the following. Is there a better/easier/cleaner way of doing this?
//send a async query to Redis to
val queryFuture = redis.zrangebyscore[String](tablename, keymin, keymax )
queryFuture.onComplete {
case Success(datarows) =>
println(s"Got %d rows of type %s for the query successfully".format(datarows.length))
val jsonrows = for { x <- datarows.toList }
yield parse(x)
println("Got json rows %d".format(jsonrows.size))
val mapdata = jsonrows.map(x => x.extract[Map[String,String]]).map( x => x.mapValues(_.toInt))
//need to do something with this data now
case Failure(y) =>
println(s" there was some failure in getting the data from Redis")
}
This looks to me like the simplest way to do it:
val map = parse("""{"a":"1","b":"2","c":"3"}""")
.children
.collect { case JField(k, JString(v)) => (k, v.toInt) }
.toMap
Your Json4s solution looks fine. Alternatively you can use mapField to transform the fields of a JObject and after that extract value of type Map[String, Int].
val json1 = parse(
"""
|{
| "key1": "1024",
| "key2": "2048",
| "key3": "4096"
|}
""".stripMargin)
val json2 = json1.mapField {
case (key, JString(value)) => (key, JInt(value.toInt))
case x => x
}
val res = json2.extract[Map[String, Int]]
println(res)
// Map(key1 -> 1024, key2 -> 2048, key3 -> 4096)
Not knowing json4s, and unfortunately you ommited the types, but guessing that jsonrows is probably something like a List[(String, String)] you could do
List(("key1" -> "1"),("key2" -> "2")).map { case (k, v) => (k, v.toInt)}.toMap
BTW, if you say need to do something with this data now in your onComplete - that could only be a side effecting operation. Better map over the future until your processing is complete.
I'm writing a simple scala application that opens a flat file of json data, parses it and finally prints it out to the screen.
The next step will require that I stop at each object and add another item (string) to the front of it. My question is how can I add a new string per object in this list?
The following is my JSON implementation (credit goes to the init author here)
import scala.util.parsing.combinator._
class JSON extends JavaTokenParsers {
def obj: Parser[Map[String, Any]] =
"{"~> repsep(member, ",") <~"}" ^^ (Map() ++ _)
def arr: Parser[List[Any]] =
"["~> repsep(value, ",") <~"]"
def member: Parser[(String, Any)] =
stringLiteral~":"~value ^^
{ case name~":"~value => (name, value) }
def value: Parser[Any] = (
obj
| arr
| stringLiteral
| floatingPointNumber ^^ (_.toInt)
| "null" ^^ (x => null)
| "true" ^^ (x => true)
| "false" ^^ (x => false)
)
}
Next I call this w/ a flat file like so
import java.io.FileReader
import scala23.JSON
class JSONTest extends JSON {
def main(args: String) {
val reader = new FileReader(args)
println(parseAll(value, reader))
}
}
Then I get a valid println of the json contents. Instead I would like to pass this parse method a String and have it append it or create a new json object that has the string at the front of each object inside
Update
My current attempt looks something like the below
class JSONTest extends JSON {
def main(args: String) {
val reader = new FileReader(args)
val header = ("abc", "def")
// println(parseAll(value, reader).map(addHeader(_, header)))
println(parseAll(value, reader).map(addHeader(_.asInstanceOf[Map[String, Any]], header)))
}
def addHeader(xyz:Map[String, Any], header:(String, Any)):Map[String, Any] = {
xyz.map {
case (k, m:Map[String, Any]) => (k, addHeader(m))
case e => e
} + header
}
}
But I'm currently getting a few errors in Intellij
error: missing parameter type for expanded function ((x$1) => x$1.asInstanceOf[Map[String, Any]])
println(parseAll(value, reader).map(addHeader(_.asInstanceOf[Map[String, Any]], header)))
AND
error: not enough arguments for method addHeader: (xyz: Map[String,Any],header: (String, Any))Map[String,Any].
Unspecified value parameter header.
case (k, m:Map[String, Any]) => (k, addHeader(m))
Any help would be much appreciated (thank you in advance!)
Have you tried using map on the parser output instead.
Edit: this compiles on my machine
import java.io.FileReader
import scala23.JSON
class JSONTest extends JSON {
def main(args: String) {
val reader = new FileReader(args)
val header = ("abc", "def")
// println(parseAll(value, reader).map(addHeader(_, header)))
println(parseAll(value, reader).map(addHeader(_, header)))
}
def addHeader(xyz:Any, header:(String, Any)):Any = xyz match {
case obj:Map[String, Any] => obj.map {
case (k, m:Map[String, Any]) => (k, addHeader(m, header))
case e => e
} + header
case arr:List[Any] => arr.map(addHeader(_, header))
case e => e
}
}
It should be handling the varied output of the parse better.