Scala Circe with generics - json

I am trying to use the scala json library Circe, wrapping it in a simple trait to provide conversion to/from json for which I have the following:
import io.circe.generic.auto._
import io.circe.parser._
import io.circe.syntax._
trait JsonConverter {
def toJson[T](t : T) : String
def fromJson[T](s: String) : T
}
case class CirceJsonConverter() extends JsonConverter{
override def toJson[T](t: T): String = t.asJson.noSpaces
override def fromJson[T](s: String): T = decode[T](s).getOrElse(null).asInstanceOf[T]
}
The aim of this is to simply be able to call JsonConverter with any object and have it convert it to/from json as such jsonConverter.toJson(0) must equalTo("0") , however when I try to compile it I get the following:
[error] could not find implicit value for parameter encoder: io.circe.Encoder[T]
[error] override def toJson[T](t: T): String = t.asJson.noSpaces
[error] ^
[error] could not find implicit value for parameter decoder: io.circe.Decoder[T]
[error] override def fromJson[T](s: String): T = decode[T](s).getOrElse(null).asInstanceOf[T]
[error] ^
[error] two errors found
I can of course have a class that everything I intend to put through the converter inherit from, but I had the impression that circe could auto generate the encoders/decoders?

What you want is not going to work unless you can implement a strategy for turning any object into Json... which seems unlikely. Circe (and many other libs) instead choose to use a common pattern called Type Classes to make it convenient to define how you want to do something, in this case Encoder/Decoder, for a specific type.
I recommend researching Type Classes if you are unfamiliar with them. And then take a look at the Circe docs to see how you can implement Encoders/Decoders specifically.

Following Idan Waisman answer and C4stor answer in my duplicate question I used the Type Classes pattern. For brevity I provide sample code only for decoding json. Encoding can be implemented in exactly the same way.
First, let's define the trait that will be used to inject json decoder dependency:
trait JsonDecoder[T] {
def apply(s: String): Option[T]
}
Next we define object that creates instance implementing this trait:
import io.circe.Decoder
import io.circe.parser.decode
object CirceDecoderProvider {
def apply[T: Decoder]: JsonDecoder[T] =
new JsonDecoder[T] {
def apply(s: String) =
decode[T](s).fold(_ => None, s => Some(s))
}
}
As you can notice apply requires implicit io.circe.Decoder[T] to be in scope when it called.
Then we copy io.circe.generic.auto object content and create a trait (I made PR to have this trait available as io.circe.generic.Auto):
import io.circe.export.Exported
import io.circe.generic.decoding.DerivedDecoder
import io.circe.generic.encoding.DerivedObjectEncoder
import io.circe.{ Decoder, ObjectEncoder }
import io.circe.generic.util.macros.ExportMacros
import scala.language.experimental.macros
trait Auto {
implicit def exportDecoder[A]: Exported[Decoder[A]] = macro ExportMacros.exportDecoder[DerivedDecoder, A]
implicit def exportEncoder[A]: Exported[ObjectEncoder[A]] = macro ExportMacros.exportEncoder[DerivedObjectEncoder, A]
}
Next in the package (e.g. com.example.app.json) that uses json decoding a lot we create package object if does not exist and make it extend Auto trait and provide implicit returning JsonDecoder[T] for given type T:
package com.example.app
import io.circe.Decoder
package object json extends Auto {
implicit def decoder[T: Decoder]: JsonDecoder[T] = CirceDecoderProvider[T]
}
Now:
all source files in com.example.app.json has Auto implicits in scope
you can get JsonDecoder[T] for any type T that has io.circe.Decoder[T] or for which it can be generated with Auto implicits
you do not need to import io.circe.generic.auto._ in every file
you can switch between json libraries by only changing com.example.app.json package object content.
For example you can switch to json4s (though I did the opposite and switched to circe from json4s). Implement provider for JsonDecoder[T]:
import org.json4s.Formats
import org.json4s.native.JsonMethods._
import scala.util.Try
case class Json4SDecoderProvider(formats: Formats) {
def apply[T: Manifest]: JsonDecoder[T] =
new JsonDecoder[T] {
def apply(s: String) = {
implicit val f = formats
Try(parse(s).extract[T]).toOption
}
}
}
And change com.example.app.json package object content to:
package com.example.app
import org.json4s.DefaultFormats
package object json {
implicit def decoder[T: Manifest]: JsonDecoder[T] = Json4SDecoderProvider(DefaultFormats)[T]
}
With Type Classes pattern you get compile-time dependency injection. That gives you less flexibility than runtime dependency injection but I doubt that you need to switch json parsers in runtime.

Related

Extending AutoDerivation in Circe does not work

My question concerns the second solution offered by mixel here: Scala Circe with generics
Note that the trait named Auto in Circe has been renamed to AutoDerivation in the current version of Circe.
I am using the solution mixel provides in his StackOverflow solution but have not been able to get it to work. I have tried things like updating my Circe version to the most recent one and making sure the Macro Paradise plugin is imported, but still no luck.
Here is my code. The first is its own file, called CirceGeneric.
import io.circe._
import io.circe.parser._
import io.circe.generic.extras._
object CirceGeneric {
trait JsonEncoder[T] {
def apply(in: T): Json
}
trait JsonDecoder[T] {
def apply(s: String): Either[Error, T]
}
object CirceEncoderProvider {
def apply[T: Encoder]: JsonEncoder[T] = new JsonEncoder[T] {
def apply(in: T) = Encoder[T].apply(in)
}
}
object CirceDecoderProvider {
def apply[T: Decoder]: JsonDecoder[T] = new JsonDecoder[T] {
def apply(s: String) = decode[T](s)
}
}
}
object Generic extends AutoDerivation {
import CirceGeneric._
implicit def encoder[T: Encoder]: JsonEncoder[T] = CirceEncoderProvider[T]
implicit def decoder[T: Decoder]: JsonDecoder[T] = CirceDecoderProvider[T]
}
The second is a method for unit testing that uses the Akka function responseAs. The method appears in a class called BaseServiceTest.
def responseTo[T]: T = {
def response(s: String)(implicit d: JsonDecoder[T]) = {
d.apply(responseAs[String]) match {
case Right(value) => value
case Left(error) => throw new IllegalArgumentException(error.fillInStackTrace)
}
}
response(responseAs[String])
}
The idea is to convert the result of responseAs[String] (which returns a string) into a decoded case class.
The code is not behaving as expected. Intellij does not detect any missing implicits, but when compilation time comes around, I am getting problems. I should mention that the BaseServiceTest file contains imports for CirceGeneric._ and Generic._, so a missing import statement is not the problem.
[error] [...]/BaseServiceTest.scala:59: could not find implicit value for parameter d: [...]CirceGeneric.JsonDecoder[T]
[error] response(responseAs[String])
Either the implicit conversion from Decoder[T] to JsonDecoder[T] is not happening, or the Decoder[T] instance is not being created. Either way, something is wrong.
You still need a Decoder or JsonDecoder context bound on responseTo.
def responseTo[T : Decoder]: T = ...
This is because all your code, and indeed mixel's code in the linked answer, is about abstracting from a Decoder out to a JsonDecoder trait which can be used for cross-library support. But you still don't have any way of constructing one without an underlying Decoder instance.
Now, there are some ways of automatically generating Decoders for (for instance) case classes contained in circe.generics.auto, but at this point in your code
def responseTo[T]: T = {
def response(s: String)(implicit d: JsonDecoder[T]) = ...
...
}
you're asking the compiler to be able to provide an implicit JsonDecoder (i.e., in your setup, Decoder) instance for any arbitrary type. As the accepted answer to the linked question explains, that's not possible.
You need to delay the implicit resolution to the point where you know what type you're dealing with - in particular, that you can provide a Decoder[T] instance for it.
EDIT: In your response to your comment regarding what the point is if you can't create JsonDecoders for all types...
My understanding of the linked question is that they're trying to abstract away the circe library in order to allow swapping out the JSON library implementation. This is done as follows:
add the JsonDecoder type class
have a package json which contains implicits (using Circe) for constructing them automatically via the package object extending AutoDerivation
have external code only refer to JsonDecoder and import the implicits in the json package
Then all the JSON serialization and implicit resolution works out without ever needing the calling code to reference io.circe, and it's easy to switch over the json/JsonDecoder to another JSON library if desired. But you're still going to have to use the JsonDecoder context bound, and be restricted to working with types where such an implicit can be constructed. Which is not every type.

Get a Json format for a Seq of a generic type

I have an abstract class with a generic type which gets a Json format for that generic type from its subclass. But the abstract class also needs a Json format of a sequence of that type. Is there any way in Scala to get a Json format of a sequence of things based only on the format of those things?
I'm using the Play Json framework.
Here's an example that doesn't follow my case exactly but provides a good indication of what I want to achieve:
package scalatest
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Await
import scala.concurrent.duration.Duration
import java.util.UUID
import scala.util.control.NonFatal
import play.api.libs.json.Format
import play.api.libs.json.Json
object Banana {
def main(args: Array[String]): Unit = {
val f: Format[Seq[Banana]] = getSeqFormat(Json.format[Banana])
}
def getSeqFormat[T](format: Format[T]): Format[Seq[T]] = {
??? // TODO implement
}
}
case class Banana(color: String)
If you're just trying to serialize bananas into JSON objects then the only thing you need to do is define the Banana implicit json format, the others (like Seq format for example) are built-in within play:
import play.api.libs.json.Json
case class Banana(color: String)
object Banana {
implicit val jsonFormat = Json.writes[Banana]
}
object PlayJsonTest extends App {
val bananas = Seq(Banana("yellow"), Banana("green"))
println(Json.toJson(bananas)) // [{"color":"yellow"},{"color":"green"}]
}
This also works for other types because the Json#toJson method is defined as follows:
// Give me an implicit `Writes[T]` and I know how to serialize it
def toJson[T](o: T)(implicit tjs: Writes[T]): JsValue = tjs.writes(o)
The defaults are implicitly used and those include a format for most of the collections. You can find them here.
I hope that helps you.

Mapping Seq to JSON in Play

I'm trying to map a Scala case class to JSON using Play 2.x. This works for simple versions of the case class, but not when there's a Seq or List of objects involved: then I get 'no implicit format' and 'no unapply function found' errors.
The code I'm using for this is the following:
case class Book(title: String, authors: Seq[Author])
case class Author(name: String)
I've used the Json.format macro to generate the Reads and Writes for this:
implicit val bookFormat = Json.format[Book]
implicit val authorFormat = Json.format[Author]
But now when I'm compiling my code, I get the following error:
Error:(25, 40) Play 2 Compiler:
/Users/erikp/Userfiles/projects/play/booksearch/app/models/user.scala:25: No implicit format for Seq[models.Author] available.
implicit val bookFormat = Json.format[Book]
^
Without the Seq it works nicely, but with the Seq, it fails. I tried adding implicit val authorsFormat = Json.format[Seq[Author]] to the implicit converters, but that has no effect.
Define the formatters respecting their dependency order, for each class in the graph that needs to be serialized.
Formatting Book requires formatting Author, so define the Author formatter before the Book formatter.
For example, with this Models.scala file:
package models
import play.api.libs.json._
case class Book(title: String, authors: Seq[Author])
case class Author(name: String)
object Formatters {
implicit val authorFormat = Json.format[Author]
implicit val bookFormat = Json.format[Book]
}
and this JsonExample.scala file:
package controllers
import models._
import models.Formatters._
import play.api.mvc._
import play.api.libs.json._
object JsonExample extends Controller {
def listBooks = Action {
val books = Seq(
Book("Book One", Seq(Author("Author One"))),
Book("Book Two", Seq(Author("Author One"), Author("Author Two")))
)
val json = Json.toJson(books)
Ok(json)
}
}
a request to listBooks will produce this result:
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Content-Length: 133
<
[{"title":"Book One","authors":[{"name":"Author One"}]},{"title":"Book Two","authors":[{"name":"Author One"},{"name":"Author Two"}]}]
For more advanced formatting, including partial serialization to avoid having to declare formatters for classes that should not be serialized, see JSON Reads/Writes/Format Combinators.
It should be kept in mind that the classes to be serialized don't necessarily have to be the domain model classes. It may be helpful to declare data transfer object (DTO) classes that reflect the desired JSON structure, and instantiate them from the domain model. This way, serialization is straightforward with Json.format and there isn't the issue of partial serialization, with the added benefit of a typesafe representation of the JSON API.
For example, this BookDTO.scala file defines a BookDTO data transfer object that uses only types that can be serialized to JSON without requiring further definition:
package dtos
import models._
import play.api.libs.json.Json
case class BookDTO (title: String, authors: Seq[String])
object BookDTO {
def fromBook(b: Book) = BookDTO(b.title, b.authors.map(_.name))
implicit val bookDTOFormat = Json.format[BookDTO]
}
and this JsonExample2.scala file shows how to use this pattern:
package controllers
import dtos._
import dtos.BookDTO._
import models._
import play.api.mvc._
import play.api.libs.json._
import play.api.libs.functional.syntax._
object JsonExample2 extends Controller {
def listBooks = Action {
val books = Seq(
Book("Book One", Seq(Author("Author One"))),
Book("Book Two", Seq(Author("Author One"), Author("Author Two")))
)
val booksDTO = books.map(BookDTO.fromBook(_))
Ok(Json.toJson(booksDTO))
}
}

Rendering JSON with Play! and Scala

I have a simple question regarding rendering JSON object from a Scala class. Why do I have to implemet deserializer ( read, write ).
I have the following case class:
case class User(firstname:String, lastname:String, age:Int)
And in my controller:
val milo:User = new User("Sam","Fisher",23);
Json.toJson(milo);
I get compilation error: No Json deserializer found for type models.User. Try to implement an implicit Writes or Format for this type.
In my previous project I had to implement a reader,writer object in the class for it to work and I find it very annoying.
object UserWebsite {
implicit object UserWebsiteReads extends Format[UserWebsite] {
def reads(json: JsValue) = UserWebsite(
(json \ "email").as[String],
(json \ "url").as[String],
(json \ "imageurl").as[String])
def writes(ts: UserWebsite) = JsObject(Seq(
"email" -> JsString(ts.email),
"url" -> JsString(ts.url),
"imageurl" -> JsString(ts.imageurl)))
}
}
I really recommend to upgrade to play 2.1-RC1 because here, JSON writers/readers are very simple to be defined (more details here)
But in order to help you to avoid some errors, I will give you a hint with imports:
- use these imports only! (notice that json.Reads is not included)
import play.api.libs.json._
import play.api.libs.functional.syntax._
import play.api.libs.json.Writes._
and you only have to write this code for write/read your class to/from Json (of course you will have User instead of Address:
implicit val addressWrites = Json.writes[Address]
implicit val addressReads = Json.reads[Address]
Now, they will be used automatically:
Example of write:
Ok(Json.toJson(entities.map(s => Json.toJson(s))))
Example of read(I put my example of doing POST for creating an entity by reading json from body) please notice addressReads used here
def create = Action(parse.json) { request =>
request.body.validate(addressReads).map { entity =>
Addresses.insert(entity)
Ok(RestResponses.toJson(RestResponse(OK, "Succesfully created a new entity.")))
}.recover { Result =>
BadRequest(RestResponses.toJson(RestResponse(BAD_REQUEST, "Unable to transform JSON body to entity.")))
}
}
In conclusion, they tried (and succeded) to make things very simple regarding JSON.
If you are using play 2.0.x you can do
import com.codahale.jerkson.Json._
generate(milo)
generate uses reflection to do it.
In play 2.1 you can use Json.writes to create a macro for that implicit object you had to create. No runtime reflection needed!
import play.api.libs.json._
import play.api.libs.functional.syntax._
implicit val userWrites = Json.writes[User]
Json.toJson(milo)
I have been using jerkson (which basically is wrapper to jackson) in my project to convert objects to json string.
The simplest way to do that is:
import com.codehale.jerkson.Json._
...
generate(milo)
...
If you need to configure the ObjectMapper (e.g. adding custom serializer/deserializer, configuring output format, etc.), you can do it by creating object which extends com.codehale.jerkson.Json class.
package utils
import org.codehaus.jackson.map._
import org.codehaus.jackson.{Version, JsonGenerator, JsonParser}
import com.codahale.jerkson.Json
import org.codehaus.jackson.map.module.SimpleModule
import org.codehaus.jackson.map.annotate.JsonSerialize
object CustomJson extends Json {
val module = new SimpleModule("CustomSerializer", Version.unknownVersion())
// --- (SERIALIZERS) ---
// Example:
// module.addSerializer(classOf[Enumeration#Value], EnumerationSerializer)
// --- (DESERIALIZERS) ---
// Example:
// module.addDeserializer(classOf[MyEnumType], new EnumerationDeserializer[MyEnumType](MyEnumTypes))
mapper.setSerializationInclusion(JsonSerialize.Inclusion.NON_NULL)
mapper.setSerializationConfig(mapper.getSerializationConfig.without(SerializationConfig.Feature.WRITE_NULL_MAP_VALUES))
mapper.registerModule(module)
}
To use it in your codes:
import utils.CustomJson._
...
generate(milo)
...
In fact, this is very simple. Firstly import:
import play.api.libs.json._
Thanks to the fact, that User is a case class you can automatically create Json writes using Json.writes[]:
val milo:User = new User("Millad","Dagdoni",23)
implicit val userImplicitWrites = Json.writes[User]
Json.toJson(milo)
I haven't found it in the docs, but here is the link to the api: http://www.playframework.com/documentation/2.2.x/api/scala/index.html#play.api.libs.json.Json$
In your case, I'd use the JSON.format macro.
import play.api.libs.json._
implicit val userFormat = Json.format[User]
val milo = new User("Sam", "Fisher", 23)
val json = Json.toJson(milo)

Play2 does not find my implicit Reads or Format for JSON

This is my Search Object:
package models.helper
import play.api.libs.json.Format
import play.api.libs.json.JsValue
import play.api.libs.json.JsObject
import play.api.libs.json.JsString
case class Search (name: String, `type`:String){
implicit object SearchFormat extends Format[Search] {
def reads(json: JsValue): Search = Search(
(json \ "name").as[String],
(json \ "type").as[String]
)
def writes(s: Search): JsValue = JsObject(Seq(
"name" -> JsString(s.name),
"type" -> JsString(s.`type`)
))
}
}
I'm trying ot use this class when calling a webservice using WS:
val search = response.json.as[Search]
But the scala compiler keeps complaining on this line:
No Json deserializer found for type models.helper.Search. Try to
implement an implicit Reads or Format for this type.
Could anybody tell me what I'm doing wrong?
got the example from https://sites.google.com/site/play20zh/scala-developers/working-with-json
this thread discusses the same issue but gives no solution, what example on what site? https://groups.google.com/forum/?fromgroups#!topic/play-framework/WTZrmQi5XxY
Indeed the example is wrong. You need your implicit Format[Search] value to be available in the implicit scope.
In your case the Format[Search] is defined as a nested value of the class Search, so you can reach it only from an instance of Search.
So, what you want to do is to define it in another place, where it could be referenced without having to create an instance of Search, e.g. in a Formats object:
object Formats {
implicit SearchFormat extends Format[Search] {
…
}
}
Then you can use it as follows:
import Formats.SearchFormat
val search = response.json.as[Search]
You can also get rid of the import tax by defining the Format[Search] value in the companion object of the Search class. Indeed the Scala compiler automatically looks in companion objects of type parameters when it needs an implicit value of a given type:
case class Search(name: String, `type`: String)
object Search {
implicit object SearchFormat extends Format[Search] {
…
}
}
Then you can use it without having to import it:
val search = response.json.as[Search]