I am writing a case class that will map to a JSON result that I am getting from an external API response.
The datetime looks like: 2016-05-30T00:23:27.070Z
What type should I use to map to this datetime string?
I want to use playframework's json automapper so I can just do:
implicit val userReads = Json.reads[User]
case class User(createdAt: ?????)
There is already predefined Format for dates DefaultLocalDateTimeReads:
import java.time.LocalDateTime
val json = Json.parse("""{"date": "2016-05-30T00:23:27.070Z"}""")
(json \ "date").as[LocalDateTime]
In case you need some other dateTime library/format, you could write custom reader like this one:
import org.joda.time.DateTime
import play.api.libs.json.{JsError, _}
implicit object DateTimeReads extends Reads[DateTime] {
val Format = org.joda.time.format.DateTimeFormat
.forPattern("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")
def reads(json: JsValue) = json match {
case JsString(x) => JsSuccess(Format.parseDateTime(x))
case _ => JsError(s"Can't read $json as DateTime")
}
}
(json \ "date").as[DateTime]
res0: org.joda.time.DateTime = 2016-05-30T00:23:27.070+03:00
import java.time.LocalDateTime
case class User(createdAt: LocalDateTime)
implicit val userReads = Json.reads[User]
Related
I'm new to Akka HTTP and I want to get rid of unnecessary fields from a JSON response and take only the necessary fields. For example, I use this endpoint to get the response and it contains a bunch of fields. For the moment I only need 'name' and 'versions'. I would like to know how to deserialize this into a case class containing only 'name' and 'versions'. I coded the following lines to get the response as a string.
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{HttpRequest, HttpResponse}
import akka.stream.scaladsl.{Flow, Sink, Source}
import akka.stream.{ActorMaterializer, OverflowStrategy}
import scala.concurrent.Future
import scala.concurrent.duration.DurationInt
import scala.language.postfixOps
import scala.util.{Failure, Success}
object SoftwareRegistry extends App {
implicit val system = ActorSystem("NPMRegistry")
implicit val materializer = ActorMaterializer()
import system.dispatcher
case class NPMPackage(name: String)
// reading the packages
val filename = "B:\\Scala\\NPMRegistry\\src\\main\\resources\\packages.txt"
val bufferedSource = scala.io.Source.fromFile(filename)
val listOfPackages: List[NPMPackage] = (for (line <- bufferedSource.getLines) yield {
NPMPackage(line.trim)
}).toList
bufferedSource.close()
// source
val sourceList = Source(listOfPackages)
// sink
val sink = Sink.foreach[NPMPackage] { p =>
// https request
val responseFuture: Future[HttpResponse] =
Http().singleRequest(HttpRequest(uri = s"https://registry.npmjs.org/${p.name}"))
val x = responseFuture
.flatMap(_.entity.toStrict(2 seconds))
.map(_.data.utf8String)
x.onComplete {
case Success(res) => println(res)
case Failure(_) => sys.error("Something went wrong")
}
}
// flow to slow things down and streaming sink to time-delayed operations
val bufferedFlow = Flow[NPMPackage]
.buffer(10, overflowStrategy = OverflowStrategy.backpressure)
.throttle(1, 3 seconds)
sourceList.async
.via(bufferedFlow).async
.to(sink)
.run()
}
And it prints the following output
For parsing json you need to use some library. In akka-http docs they use spray-json. Add the following dependency to your build.sbt with appropriate akkaHttpVersion.
"com.typesafe.akka" %% "akka-http-spray-json" % akkaHttpVersion
Now you need serializers and deserializers for your data. I am using a simple model, change it as needed.
trait Formatter extends DefaultJsonProtocol {
implicit object jsonFormat extends JsonFormat[Versions] {
override def read(json: JsValue): Versions = json match {
case JsObject(fields) =>
Versions(fields.keys.toList)
}
override def write(obj: Versions): JsValue = JsonParser(obj.toString)
}
implicit val formatterPackage: RootJsonFormat[Package] = jsonFormat2(Package)
case class Package(name: String, versions: Versions)
case class Versions(versions: List[String])
}
Finally sink:
//needed import with others
import spray.json._
object SoftwareRegistry extends App with Formatter {
//existing code
//---------
val sink = Sink.foreach[NPMPackage] { p =>
// https request
val responseFuture: Future[HttpResponse] =
Http().singleRequest(HttpRequest(uri = s"https://registry.npmjs.org/${p.name}"))
val packages = responseFuture
.flatMap(
_.entity
.dataBytes
.via(JsonFraming.objectScanner(Int.MaxValue))
.map(_.utf8String)
.map(_.parseJson.convertTo[Package])
.toMat(Sink.seq)(Keep.right)
.run()
)
packages.onComplete {
case Success(res) => println(res)
case Failure(_) => sys.error("Something went wrong")
}
}
//existing code
//---------
}
In an akka scala applicatoon I consume a rest endpoint. Hence, I want to map its responses to case classes, yet I also want to ease working with those case classes by transforming certain properties, e.g. those containing dates.
So given a Json:
{
"id": "20180213165959sCdJr",
"createdAt": "2018-02-13T16:59:59.570+0000",
"finishedAt": "2018-02-13T17:00:18.118+0000"
}
I want to create such a clase class out of it:
case class FinishedRun
(
id: String,
createdAt: Date,
finishedAt: Date
)
I created this construtor:
object FinishedRun {
def apply(id: String,
createdAt: String,
finishedAt: String
): FinishedRun = {
val getDate = (jsonValue: String) => {
val format = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ")
format.parse(jsonValue)
}
new FinishedRun(id, createdAt = getDate(createdAt), finishedAt = getDate(finishedAt))
}
}
While this works for initializing a case class from scratch, I have trouble extracting this case class with the help of the json4s libary through the parse(json).as[FinishedRun] approach.
It appears that json4s does not call the case class' constructor and hence cannot extract it, throwing:
No usable value for createdAt
Invalid date '2018-02-13T16:59:59.570+0000'
org.json4s.package$MappingException: No usable value for createdAt
Invalid date '2018-02-13T16:59:59.570+0000'
at org.json4s.reflect.package$.fail(package.scala:95)
What am I missing to have Json4s parse the Date properly?
Here is my test case:
import org.json4s._
import org.json4s.native.JsonMethods._
import org.scalatest.FlatSpec
class MarshallingTest extends FlatSpec {
implicit val formats = DefaultFormats
it should "marshall json object with date iso strings into a case class with Date properties" in {
val json =
"""
|{
| "id": "20180213165959sCdJr",
| "createdAt": "2018-02-13T16:59:59.570+0000",
| "finishedAt": "2018-02-13T17:00:18.118+0000"
|}
""".stripMargin
val expected = FinishedRun(
id = "20180213165959sCdJr",
createdAt = "2018-02-13T16:59:59.570+0000",
finishedAt = "2018-02-13T17:00:18.118+0000"
)
val actual = parse(json).extract[FinishedRun]
assert(actual == expected)
}
}
You need to define your CustomSerializer(1), CustomSerializer(2). I changed the date type from Date to ZonedDateTime:
import java.time.ZonedDateTime
import java.time.format.DateTimeFormatter
import org.json4s._
import org.json4s.JsonAST._
import org.json4s.JsonDSL._
import org.json4s.native.JsonMethods._
import org.scalatest.FlatSpec
case class FinishedRun
(
id: String,
createdAt: ZonedDateTime,
finishedAt: ZonedDateTime
)
object FinishedRunSerializer {
val dateTimeFmt = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSSZ")
}
class FinishedRunSerializer extends CustomSerializer[FinishedRun](
format => ( {
case jObj: JObject =>
implicit val fmt = format
val id = (jObj \ "id").extract[String]
val created = ZonedDateTime.parse((jObj \ "createdAt").extract[String],
FinishedRunSerializer.dateTimeFmt)
val finished = ZonedDateTime.parse((jObj \ "finishedAt").extract[String],
FinishedRunSerializer.dateTimeFmt)
FinishedRun(id, created, finished)
}, {
case finishedRun: FinishedRun =>
("id" -> finishedRun.id) ~
("createdAt" -> finishedRun.createdAt.format(FinishedRunSerializer.dateTimeFmt)) ~
("finishedAt" -> finishedRun.finishedAt.format(FinishedRunSerializer.dateTimeFmt))
}
))
In your test or the place when you use it do not forget to bring FinishedRunSerializer:
implicit val formats = DefaultFormats + new FinishedRunSerializer()
A simple solution to your problem might be to use the Serialization trait in org.json4s. You should be able to do something like this:
val finishedRun = read[FinishedRun](json)
Please refer to this link for details and examples: https://github.com/json4s/json4s#serializing-polymorphic-lists
I am trying to use akka-http-spray-json 10.0.9
My model:
case class Person(id: Long, name: String, age: Int)
I get json string jsonStr with list of persons and try to parse it:
implicit val personFormat: RootJsonFormat[Person] = jsonFormat3(Person)
val json = jsonStr.parseJson
val persons = json.convertTo[Seq[Person]]
Error:
Object expected in field 'id'
Probably i need to create implicit object extends RootJsonFormat[List[Person]] and override read and write methods.
implicit object personsListFormat extends RootJsonFormat[List[Person]] {
override def write(persons: List[Person]) = ???
override def read(json: JsValue) = {
// Maybe something like
// json.map(_.convertTo[Person])
// But there is no map or similar method :(
}
}
P.S. Sorry for my english, it's not my native.
UPD
jsonStr:
[ {"id":6,"name":"Martin Ordersky","age":50}, {"id":8,"name":"Linus Torwalds","age":43}, {"id":9,"name":"James Gosling","age":45}, {"id":10,"name":"Bjarne Stroustrup","age":59} ]
I get perfectly expected results with:
import spray.json._
object MyJsonProtocol extends DefaultJsonProtocol {
implicit val personFormat: JsonFormat[Person] = jsonFormat3(Person)
}
import MyJsonProtocol._
val jsonStr = """[{"id":1,"name":"john","age":40}]"""
val json = jsonStr.parseJson
val persons = json.convertTo[List[Person]]
persons.foreach(println)
I'm using play.api.libs.json._ library. I have this kind of Scala class. I need to read / write this class in Json format. As there is no implicit reader/ writer for Timestamp. I have to provide my own. I tried couple ways unfortunately none of them worked. Could you please suggest me how it is done? Thanks in advance!
case class Event(id: Long, startTime: Option[java.sql.Timestamp] = None, endTime: Option[java.sql.Timestamp] = None)
I would like to POST / GET in following Json format
{
"id": 1,
"startTime": "2011-10-02 18:48:05.123456",
"endTime": "2011-10-02 20:48:05.123456"
}
just add before Json Reader or Json Format for Event class
import play.api.libs.json.Json._
import play.api.libs.json._
def timestampToDateTime(t: Timestamp): DateTime = new DateTime(t.getTime)
def dateTimeToTimestamp(dt: DateTime): Timestamp = new Timestamp(dt.getMillis)
implicit val timestampFormat = new Format[Timestamp] {
def writes(t: Timestamp): JsValue = toJson(timestampToDateTime(t))
def reads(json: JsValue): JsResult[Timestamp] = fromJson[DateTime](json).map(dateTimeToTimestamp)
}
What I did code for a personal project:
implicit object timestampFormat extends Format[Timestamp] {
val format = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SS'Z'")
def reads(json: JsValue) = {
val str = json.as[String]
JsSuccess(new Timestamp(format.parse(str).getTime))
}
def writes(ts: Timestamp) = JsString(format.format(ts))
}
And don't forget to import this:
import java.sql.Timestamp
import java.text.SimpleDateFormat
import play.api.Play.current
import play.api.libs.json._
It respects Javascript dates standard.
Source: https://github.com/BinaryBrain/Gamers/blob/master/server/play/app/models/Package.scala
I'm using my own implicit implementation of JSON serializer and deserializer for my case object
My case class looks like (it's just a code snippet)
sealed trait MyTrait
case object MyCaseClass extends MyTrait
I want to write my own ser. and deser. of JSON for MyTrait
implicit val myTraitFormat = new JsonFormat[MyTrait] {
override def read(json: JsValue): MyTrait = json.asJsObject.getFields("value") match {
case Seq(JsString("TEST")) ⇒ MyCaseClass
case _ ⇒ throw new DeserializationException(s"$json is not a valid extension of my trait")
}
override def write(myTrait: MyTrait): JsValue = {
myTrait match {
case MyCaseClass => JsObject("value" -> JsString("TEST"))
}
}
}
Now my test is failing by throwing DeserializationException:
"The my JSON format" when {
"deserializing a JSON" must {
"return correct object" in {
val json = """{"value": "TEST"}""".asJson
json.convertTo[MyTrait] must equal (MyCaseClass)
}
}
}
Obviously json.asJsObject.getFields("value")can not be matched to Seq(JsString("TEST")). Maybe this is related to using traits?
But I have found example on official spray-json site https://github.com/spray/spray-json#providing-jsonformats-for-other-types
Any ideas how to properly match on field in JsObject?
Thanks!
Best
Write variable name instead of string:
override def read(json: JsValue): MyCaseClass = json.asJsObject.getFields("value") match {
case Seq(JsString(value)) ⇒ MyCaseClass
case _ ⇒ throw new DeserializationException(s"$json is not a valid case class")
}
Does that work for you?
Update:
Yes, your test is wrong. I tried it with the following (note parseJson instead of asJson) and it worked:
scala> val json = """{"value": "TEST"}""".parseJson
json: spray.json.JsValue = {"value":"TEST"}
scala> json.convertTo[MyCaseClass]
res2: MyCaseClass = MyCaseClass()
Update 2: Tried it with trait:
scala> import spray.json._
import spray.json._
scala> import spray.json.DefaultJsonProtocol._
import spray.json.DefaultJsonProtocol._
[Your code]
scala> val json = """{"value": "TEST"}""".parseJson
json: spray.json.JsValue = {"value":"TEST"}
scala> json.convertTo[MyTrait]
res1: MyTrait = MyCaseClass