In an akka scala applicatoon I consume a rest endpoint. Hence, I want to map its responses to case classes, yet I also want to ease working with those case classes by transforming certain properties, e.g. those containing dates.
So given a Json:
{
"id": "20180213165959sCdJr",
"createdAt": "2018-02-13T16:59:59.570+0000",
"finishedAt": "2018-02-13T17:00:18.118+0000"
}
I want to create such a clase class out of it:
case class FinishedRun
(
id: String,
createdAt: Date,
finishedAt: Date
)
I created this construtor:
object FinishedRun {
def apply(id: String,
createdAt: String,
finishedAt: String
): FinishedRun = {
val getDate = (jsonValue: String) => {
val format = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ")
format.parse(jsonValue)
}
new FinishedRun(id, createdAt = getDate(createdAt), finishedAt = getDate(finishedAt))
}
}
While this works for initializing a case class from scratch, I have trouble extracting this case class with the help of the json4s libary through the parse(json).as[FinishedRun] approach.
It appears that json4s does not call the case class' constructor and hence cannot extract it, throwing:
No usable value for createdAt
Invalid date '2018-02-13T16:59:59.570+0000'
org.json4s.package$MappingException: No usable value for createdAt
Invalid date '2018-02-13T16:59:59.570+0000'
at org.json4s.reflect.package$.fail(package.scala:95)
What am I missing to have Json4s parse the Date properly?
Here is my test case:
import org.json4s._
import org.json4s.native.JsonMethods._
import org.scalatest.FlatSpec
class MarshallingTest extends FlatSpec {
implicit val formats = DefaultFormats
it should "marshall json object with date iso strings into a case class with Date properties" in {
val json =
"""
|{
| "id": "20180213165959sCdJr",
| "createdAt": "2018-02-13T16:59:59.570+0000",
| "finishedAt": "2018-02-13T17:00:18.118+0000"
|}
""".stripMargin
val expected = FinishedRun(
id = "20180213165959sCdJr",
createdAt = "2018-02-13T16:59:59.570+0000",
finishedAt = "2018-02-13T17:00:18.118+0000"
)
val actual = parse(json).extract[FinishedRun]
assert(actual == expected)
}
}
You need to define your CustomSerializer(1), CustomSerializer(2). I changed the date type from Date to ZonedDateTime:
import java.time.ZonedDateTime
import java.time.format.DateTimeFormatter
import org.json4s._
import org.json4s.JsonAST._
import org.json4s.JsonDSL._
import org.json4s.native.JsonMethods._
import org.scalatest.FlatSpec
case class FinishedRun
(
id: String,
createdAt: ZonedDateTime,
finishedAt: ZonedDateTime
)
object FinishedRunSerializer {
val dateTimeFmt = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSSZ")
}
class FinishedRunSerializer extends CustomSerializer[FinishedRun](
format => ( {
case jObj: JObject =>
implicit val fmt = format
val id = (jObj \ "id").extract[String]
val created = ZonedDateTime.parse((jObj \ "createdAt").extract[String],
FinishedRunSerializer.dateTimeFmt)
val finished = ZonedDateTime.parse((jObj \ "finishedAt").extract[String],
FinishedRunSerializer.dateTimeFmt)
FinishedRun(id, created, finished)
}, {
case finishedRun: FinishedRun =>
("id" -> finishedRun.id) ~
("createdAt" -> finishedRun.createdAt.format(FinishedRunSerializer.dateTimeFmt)) ~
("finishedAt" -> finishedRun.finishedAt.format(FinishedRunSerializer.dateTimeFmt))
}
))
In your test or the place when you use it do not forget to bring FinishedRunSerializer:
implicit val formats = DefaultFormats + new FinishedRunSerializer()
A simple solution to your problem might be to use the Serialization trait in org.json4s. You should be able to do something like this:
val finishedRun = read[FinishedRun](json)
Please refer to this link for details and examples: https://github.com/json4s/json4s#serializing-polymorphic-lists
Related
I am using the Play Framework and trying to build JSON validator for a class with abstract members. Shown below, the DataSource class is the base class which I am trying to validate the format against.
// SourceTypeConfig Trait.
trait SourceTypeConfig
final case class RDBMSConfig(...) extends SourceTypeConfig
object RDBMSConfig { implicit val fmt = Json.format[RDBMSConfig] }
final case class DirectoryConfig(
path: String,
pathType: String // Local, gcloud, azure, aws, etc.
) extends SourceTypeConfig
object DirectoryConfig { implicit val fmt = Json.format[DirectoryConfig] }
// FormatConfig trait.
trait FormatConfig
final case class SQLConfig(...) extends FormatConfig
object SQLConfig { implicit val fmt = Json.format[SQLConfig]}
final case class CSVConfig(
header: String,
inferSchema: String,
delimiter: String
) extends FormatConfig
object CSVConfig { implicit val fmt = Json.format[CSVConfig]}
// DataSource base class.
case class DataSource(
name: String,
sourceType: String,
sourceTypeConfig: SourceTypeConfig,
format: String,
formatConfig: FormatConfig
)
What I am hoping to accomplish:
val input: JsValue = Json.parse(
"""
{
"name" : "test1",
"sourceType" : "directory",
"sourceTypeConfig" : {"path" : "gs://test/path", "pathType" "google"},
"format" : "csv",
"formatConfig" : {"header" : "yes", "inferSchema" : "yes", "delimiter" : "|"}
}
"""
)
val inputResult = input.validate[DataSource]
What I am struggling with is building the DataSource object and defining its reads/writes/format. I would like it to contain a match based on the sourceType and format values that direct it to point towards the associated sourceTypeConfig and formatConfig's formats so it can parse out the JSON.
Instead of building a parser at the DataSource level, I defined parsers at the SourceConfig and FormatConfig levels, similar to what is shown below.
sealed trait SourceConfig{val sourceType: String}
object SourceConfig{
implicit val fmt = new Format[SourceConfig] {
def reads(json: JsValue): JsResult[SourceConfig] = {
def from(sourceType: String, data: JsObject): JsResult[SourceConfig] = sourceType match {
case "RDBMS" => Json.fromJson[RDBMSConfig](data)(RDBMSConfig.fmt)
case "directory" => Json.fromJson[DirectoryConfig](data)(DirectoryConfig.fmt)
case _ => JsError(s"Unknown source type: '$sourceType'")
}
for {
sourceType <- (json \ "sourceType").validate[String]
data <- json.validate[JsObject]
result <- from(sourceType, data)
} yield result
}
def writes(source: SourceConfig): JsValue =
source match {
case b: RDBMSConfig => Json.toJson(b)(RDBMSConfig.fmt)
case b: DirectoryConfig => Json.toJson(b)(DirectoryConfig.fmt)
}
}
}
Then, DataSource could be simply defined as:
object DataSource { implicit val fmt = Json.format[DataSource] }
Another option is to use play-json-derived-codecs library:
libraryDependencies += "org.julienrf" %% "play-json-derived-codecs" % "4.0.0"
import julienrf.json.derived.flat
implicit val format1: OFormat[RDBMSConfig] = Json.format[RDBMSConfig]
implicit val format2: OFormat[DirectoryConfig] = Json.format[DirectoryConfig]
implicit val format3: OFormat[SourceTypeConfig] = flat.oformat((__ \ "sourceType").format[String])
I am not much familier with spray json, but I have to convert the below json into Array[myTest]
Below is the code, but it doesnt work. It throws the following errors: How do I fix them?
Error:(19, 54) Cannot find JsonReader or JsonFormat type class for Array[A$A61.this.myTest]
lazy val converted= trainingDataRef.toJson.convertTo[Array[myTest]]
^
Error:(19, 54) not enough arguments for method convertTo: (implicit evidence$1: spray.json.JsonReader[Array[A$A61.this.myTest]])Array[A$A61.this.myTest].
Unspecified value parameter evidence$1.
lazy val converted= trainingDataRef.toJson.convertTo[Array[myTest]]
^
Error:(10, 61) could not find implicit value for evidence parameter of type spray.json.DefaultJsonProtocol.JF[Map[String,Any]]
implicit val format: RootJsonFormat[myTest] = jsonFormat3(myTest.apply)
^
Code: ^
import spray.json.DefaultJsonProtocol._
import spray.json._
case class myTest (
id: String,
classDetails: Map[String, Any],
school: Map[String, Any])
object myTest {
implicit val format: RootJsonFormat[myTest] = jsonFormat3(myTest.apply)
}
val trainingDataRef = """[{"id":"my-id","classDetails":{"sec":"2","teacher":"John"},"school":{"name":"newschool"}}]"""
println(trainingDataRef.getClass)
val converted= trainingDataRef.toJson.convertTo[Array[myTest]]
println(converted)
spray-json has good documentation, try take a look there. Basically, you have to define your case classes and implement JsonFormat for them:
import spray.json.DefaultJsonProtocol._
import spray.json._
case class ClassDetails(sec: String, teacher: String)
object ClassDetails {
implicit val format: RootJsonFormat[ClassDetails] = jsonFormat2(ClassDetails.apply)
}
case class School(name: String)
object School {
implicit val format: RootJsonFormat[School] = jsonFormat1(School.apply)
}
case class ClassInfo
(
id: String,
classDetails: ClassDetails,
school: School
)
object ClassInfo {
implicit object ClassInfoFormat extends RootJsonFormat[ClassInfo] {
def write(c: ClassInfo): JsValue = JsObject(
"id" -> JsString(c.id),
"classDetails" -> c.classDetails.toJson,
"school" -> c.school.toJson
)
def read(value: JsValue): ClassInfo = {
value.asJsObject.getFields("id", "classDetails", "school") match {
case Seq(JsString(name), details, school) =>
new ClassInfo(name, details.convertTo[ClassDetails], school.convertTo[School])
case _ => throw new DeserializationException("ClassInfo expected")
}
}
}
}
val json = """[{"id":"my-id","classDetails":{"sec":"2","teacher":"John"},"school":{"name":"newschool"}}]"""
// JSON string to case classes
val classInfos = json.parseJson.convertTo[Seq[ClassInfo]]
classInfos.zipWithIndex.foreach { case (c, idx) =>
println(s"$idx => $c")
}
println
// Seq[ClassInfo] to JSON
println(s"$classInfos: ")
println(classInfos.toJson.prettyPrint)
I am trying to use akka-http-spray-json 10.0.9
My model:
case class Person(id: Long, name: String, age: Int)
I get json string jsonStr with list of persons and try to parse it:
implicit val personFormat: RootJsonFormat[Person] = jsonFormat3(Person)
val json = jsonStr.parseJson
val persons = json.convertTo[Seq[Person]]
Error:
Object expected in field 'id'
Probably i need to create implicit object extends RootJsonFormat[List[Person]] and override read and write methods.
implicit object personsListFormat extends RootJsonFormat[List[Person]] {
override def write(persons: List[Person]) = ???
override def read(json: JsValue) = {
// Maybe something like
// json.map(_.convertTo[Person])
// But there is no map or similar method :(
}
}
P.S. Sorry for my english, it's not my native.
UPD
jsonStr:
[ {"id":6,"name":"Martin Ordersky","age":50}, {"id":8,"name":"Linus Torwalds","age":43}, {"id":9,"name":"James Gosling","age":45}, {"id":10,"name":"Bjarne Stroustrup","age":59} ]
I get perfectly expected results with:
import spray.json._
object MyJsonProtocol extends DefaultJsonProtocol {
implicit val personFormat: JsonFormat[Person] = jsonFormat3(Person)
}
import MyJsonProtocol._
val jsonStr = """[{"id":1,"name":"john","age":40}]"""
val json = jsonStr.parseJson
val persons = json.convertTo[List[Person]]
persons.foreach(println)
If I have a JSON object like:
{
"test": 3
}
Then I would expect that extracting the "test" field as a String would fail because the types don't line up:
import org.json4s._
import org.json4s.jackson.JsonMethods
import org.json4s.JsonAST.JValue
def getVal[T: Manifest](json: JValue, fieldName: String): Option[T] = {
val field = json findField {
case JField(name, _) if name == fieldName => true
case _ => false
}
field.map {
case (_, value) => value.extract[T]
}
}
val json = JsonMethods.parse("""{"test":3}""")
val value: Option[String] = getVal[String](json, "test") // Was Some(3) but expected None
Is this automatic conversion from a JSON numeric to a String expected in Json4s? If so, are there any workarounds for this where the extracted field has to be of the same type that is specified in the type parameter to the extract method?
This is the default nature of most if not all of the parsers. If you request a value of type T and if the value can be safely cast to that specific type then the library would cast it for you. for instance take a look at the typesafe config with the similar nature of casting Numeric field to String.
import com.typesafe.config._
val config = ConfigFactory parseString """{ test = 3 }"""
val res1 = config.getString("test")
res1: String = 3
if you wanted not to automatically cast Integer/Boolean to String you could do something like this manually checking for Int/Boolean types as shown below.
if(Try(value.extract[Int]).isFailure || Try(value.extract[Boolean]).isFailure) {
throw RuntimeException(s"not a String field. try Int or Boolean")
} else {
value.extract[T]
}
One simple workaround is to create a custom serializer for cases where you want "strict" behavior. For example:
import org.json4s._
val stringSerializer = new CustomSerializer[String](_ => (
{
case JString(s) => s
case JNull => null
case x => throw new MappingException("Can't convert %s to String." format x)
},
{
case s: String => JString(s)
}
))
Adding this serializer to your implicit formats ensures the strict behavior:
implicit val formats = DefaultFormats + stringSerializer
val js = JInt(123)
val str = js.extract[String] // throws MappingException
I am writing a case class that will map to a JSON result that I am getting from an external API response.
The datetime looks like: 2016-05-30T00:23:27.070Z
What type should I use to map to this datetime string?
I want to use playframework's json automapper so I can just do:
implicit val userReads = Json.reads[User]
case class User(createdAt: ?????)
There is already predefined Format for dates DefaultLocalDateTimeReads:
import java.time.LocalDateTime
val json = Json.parse("""{"date": "2016-05-30T00:23:27.070Z"}""")
(json \ "date").as[LocalDateTime]
In case you need some other dateTime library/format, you could write custom reader like this one:
import org.joda.time.DateTime
import play.api.libs.json.{JsError, _}
implicit object DateTimeReads extends Reads[DateTime] {
val Format = org.joda.time.format.DateTimeFormat
.forPattern("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")
def reads(json: JsValue) = json match {
case JsString(x) => JsSuccess(Format.parseDateTime(x))
case _ => JsError(s"Can't read $json as DateTime")
}
}
(json \ "date").as[DateTime]
res0: org.joda.time.DateTime = 2016-05-30T00:23:27.070+03:00
import java.time.LocalDateTime
case class User(createdAt: LocalDateTime)
implicit val userReads = Json.reads[User]