It looks like when I try to serialize a case class extending from Seq it gets handeled as a Seq and all additional fields are omitted.
Serializing just leads to a Standard Sequence without the additional fields. Whereas deserialization fails.
Anybody knows what I'am doing wrong here?
Code:
// Definition of class
case class IntSeq(count: Int, ints: Seq[Int]) extends Seq[Int] {
def length: Int = ints.length
def iterator: Iterator[Int] = ints.iterator
def apply(idx: Int) = ints(idx)
}
// type Hints
implicit val hints = Serialization.formats( ShortTypeHints(List(classOf[IntSeq])) )
lazy val testIntSeq = IntSeq(5, List(1,2,3,4,5))
val ser = Serialization.write[IntSeq](testIntSeq)
// ser: String = [1,2,3,4,5]
Serialization.read[IntSeq](ser) // Hurray: org.json4s.package$MappingException: unknown error
Without discussing why such a class is useful... How would you manually encode such object?
In JSON you can have dictionaries and lists, so you cannot expect a list to have extra named attributes.
You can make your IntSeq class to have a list attribute and not extend Seq itself (and maybe provide an automatic conversion), so your JSON could look like:
{'length' : 5, 'ints' : [1, 2, 3, 4, 5]}
Related
I'm trying to write a method that will allow Jackson ObjectMapper readValue on a json string to a parameterized object type. Something like this
case class MyObj(field1: String, field2: String)
val objectMapper: ObjectMapper = new ObjectMapper().registerModule(new DefaultScalaModule)
def fromJson[T](jsonString: String, objTyp: T): T = {
objectMapper.readValue(jsonString, classOf[T])
}
val x = fromJson("""{"field1": "something", "field2": "something"}""", MyObj)
This of course returns an error of
class type required but T found
i've looked at this issue Scala classOf for type parameter
but it doesn't seem to help. It seems like this is possible to do somehow. Looking for any help
You have to give it the actual runtime class to parse into, not just a type parameter.
One way to do it is passing the class directly:
def fromJson[T](json: String, clazz: Class[T]) = objectMapper.readValue[T](json, clazz)
val x = fromJson("""...""", classOf[MyObj])
Alternatively, you can use ClassTag, which looks a bit messier in implementation, but kinda prettier at call site:
def fromJson[T : ClassTag](json: String): T = objectMapper.readValue[T](
json,
implicitly[ClassTag[T]].runtimeClass.asInstanceOf[Class[T]]
)
val x = fromJson[MyObj]("""{"field1": "something", "field2": "something"}""")
i've looked at this issue Scala classOf for type parameter but it doesn't seem to help.
In the very first answer there it's written classTag[T].runtimeClass as a replacement of classOf[T]. This should help.
Regarding the signature
def fromJson[T](jsonString: String, objTyp: T): T
You should notice that MyObj has type MyObj.type (companion-object type), not MyObj (case-class type).
Class companion object vs. case class itself
So if you call fromJson("""...""", MyObj) then the types in these two places
def fromJson[...](jsonString: String, objTyp: ???): ???
^^^ ^^^ <--- HERE
can't be the same.
If it's enough for you to call
fromJson("""...""", classOf[MyObj])
or
fromJson[MyObj]("""...""")
(normally it should be enough) then please see #Dima's answer, you should prefer those options, they're easier.
Just in case, if you really want to call like fromJson("""...""", MyObj) then for example you can use the type class ToCompanion (this is more complicated) from
Invoke construcotr based on passed parameter
Get companion object of class by given generic type Scala (answer)
// ToCompanion should be defined in a different subproject
def fromJson[C, T](jsonString: String, objTyp: C)(implicit
toCompanion: ToCompanion.Aux[C, T],
classTag: ClassTag[T]
): T =
objectMapper.readValue(jsonString, classTag.runtimeClass.asInstanceOf[Class[T]])
val x = fromJson("""{"field1": "something", "field2": "something"}""", MyObj)
// MyObj(something,something)
How to configure the spray-json parsing on parsing options?
Similarly as Jackson Parsing Features.
For example, I am parsing a json that has a field that my case class has not, and it is breaking:
spray.json.DeserializationException: Object is missing required member 'myfield'
UPDATE :
A simple example:
case class MyClass(a: String, b: Long);
and try to parse an incomplete json like
val data = "{a: \"hi\"}"
with a spray-json format like:
jsonFormat2(MyClass.apply)
// ...
data.parseJson.convertTo[MyClass]
(simplified code).
But the question goes further, I want to ask about configuration options like in other parsers. More examples:
Be able to ignore fields that exist in the JSON but not in the case class.
Ways of managing nulls or nonexistent values.
etc.
SprayJson allows you to define custom parsers like so:
case class Foo(a: String, b: Int)
implicit object FooJsonFormat extends RootJsonFormat[Foo] {
override def read(json: JsValue): Foo = {
json.asJsObject.getFields("name", "id") match {
case Seq(JsString(name), id) =>
Foo(name, id.convertTo[Int])
}
}
override def write(obj: Foo): JsValue = obj.toJson
}
This allows you to parse any arbitrary payload and pull out the fields "name" and "id" - other fields are ignored. If those fields are not guaranteed you can add something like:
case Seq(JsString(name), JsNull) =>
Foo(name, 0)
You should look at what's available in JsValue.scala - in particular JsArray may come in handy if you're getting payloads with anonymous arrays (i.e. the root is [{...}] instead of {"field":"value"...})
Spray Json doesn't support default parameters. So You cannot have a case class like
case class MyClass(a: String, b: Int = 0)
and then parse json like {"a":"foo"}
However if you make the second parameter as Option. then it works.
import spray.json._
case class MyClass(a: String, b: Option[Int] = None)
object MyProtocol extends DefaultJsonProtocol {
implicit val f = jsonFormat2(MyClass)
}
import MyProtocol.f
val mc1 = MyClass("foo", Some(10))
val strJson = mc1.toJson.toString
val strJson2 = """{"a": "foo"}"""
val mc2 = strJson2.parseJson.convertTo[MyClass]
println(mc2)
In scala, I have the following trait and classes defined (names used only for illustrative purposes):
trait Entity {
def x() : Collection
}
case class X(x : Int, y : Int) extends Entity {
def x() : Collection = XCollection()
}
case class Y(x : Int, y : Int) extends Entity {
def x() : Collection = YCollection()
}
While the class instances are created by parsing the response from a web service REST API.
Although the approach using the play-json library works in the case parsing the response, and returning a class representation of the response, I've been struggling with the following: having a generic function taking a type parameter, whereas T : Entity, and returning an instance of type T.
For example, consider the following:
def parse[T <: Entity](json : String) : Option[T] = Json.parse(json).asOpt[T](Variants.format[T])
Given a type T, I would like to parse the JSON string and produce an instance of type T, whereas the instance is a derivation of the trait Entity. However, I keep getting a compile error in regard to the reflection API:
Error:(25, 96) exception during macro expansion:
scala.ScalaReflectionException: type T is not a class
at scala.reflect.api.Symbols$SymbolApi$class.asClass(Symbols.scala:323)
at scala.reflect.internal.Symbols$SymbolContextApiImpl.asClass(Symbols.scala:73)
at julienrf.variants.Variants$Impl$.baseAndVariants(Variants.scala:132)
at julienrf.variants.Variants$Impl$.formatDiscriminator(Variants.scala:99)
at julienrf.variants.Variants$Impl$.format(Variants.scala:94)
def parse[T <: Entity](json : String) : Option[T] = Json.parse(json).asOpt[T](Variants.format[T])
^
Therefore, I would appreciate for some help!
Thanks
You better allow implicits solve your format requirements, than using some kind of factory:
def parse[T <: Entity](json: String)(implicit r: Reads[T]): Option[T] =
Json.parse(json).asOpt[T]
Then if you have a format in current context implicitly defined, parse will work:
implicit val XFormat = Json.format[X]
parse[X](Json.stringify(Json.toJson(X(1, 2))) // returns X(1, 2)
Update
You can make it as a factory, if you really want it. I would question whether it's worth doing that, but in theory I could imagine some distinct situations where you would not want to use implicit mechanics. Still I think if implicits does not work for you, you might have an architecture problem in your code
import play.api.libs.json.{Json, Reads}
import scala.reflect.runtime.universe._
trait Entity
case class X(x : Int, y : Int) extends Entity
case class Y(x : Int, y : Int) extends Entity
val mapping = Map[Type, Reads[_]](typeOf[X] -> Json.format[X], typeOf[Y] -> Json.format[Y])
def getFormat[T](tpe: Type): Reads[T] =
mapping(tpe).asInstanceOf[Reads[T]]
def parse[T : TypeTag](json: String): Option[T] = {
val map = mapping(implicitly[TypeTag[T]].tpe)
Json.parse(json).asOpt[T](getFormat(implicitly[TypeTag[T]].tpe))
}
println(parse[X]("""{"x": 5, "y": 6}"""))
println(parse[Y]("""{"x": 5, "y": 6}"""))
I'm new to scala and play framework.
Why does scala not have something like this??
class Customer (idx: Int, emailx: String) {
val id: Int = idx
val email: String = emailx
}
....
def customers = Action {
val customer = new Customer(1, "Customer1")
Ok(Json.toJson(customer))
}
I like play frameowrk (with scala, its productivity)
But,
Why should I map each field of my object manulay to json field?? Was it so hard for scala to implement this future like in Java or C#, even php has json_encode.
Is there any way to achieve this simple goal (return object as json) without any additional manipulations?
Macros are slick and perfect for generating simple case class formats
implicit val jsonFormat = Json.format[Customer]
Typically you put this declaration in your companion object to the type you are generating a format for. This way it is implicitly in scope in any file that you import your type (Customer). Like this:
case class Customer(...)
object Customer {
implicit val jsonFormat = Json.format[Customer]
}
Then in your controller you can do
Json.toJson(customer)
which will produce the JsValue type expected by Play.
For my classes I define a convertor, so that I can write exactly what you have written, e.g. Json.toJson(customer), but the convertor, though simple, does currently have to be written once. E.g.
implicit val customerWrites = new Writes[Customer] {
def writes(customer:Customer) = Json.obj(
"id" -> customer.id,
"email" -> customer.email
)
}
Perhaps macros, into which I have not delved, could do this more automatically...
I need to serialize/deserialize a Scala class with structure something like the following:
#JsonIgnoreProperties(ignoreUnknown = true, value = Array("body"))
case class Example(body: Array[Byte]) {
lazy val isNativeText = bodyIsNativeText
lazy val textEncodedBody = (if (isNativeText) new String(body, "UTF-8") else Base64.encode(body))
def this(isNativeText: Boolean, textEncodedBody: String) = this((if(isNativeText) str.getBytes("UTF-8") else Base64.decode(textEncodedBody)))
def bodyIsNativeText: Boolean = // determine if the body was natively a string or not
}
It's main member is an array of bytes, which MIGHT represent a UTF-8 encoded textual string, but might not. The primary constructor accepts an array of bytes, but there is an alternate constructor which accepts a string with a flag indicating whether this string is base64 encoded binary data, or the actual native text we want to store.
For serializing to a JSON object, I want to store the body as a native string rather than a base64-encoded string if it is native text. That's why I use #JsonIgnoreProperties to not include the body property, and instead have a textEncodedBody that gets echoed out in the JSON.
The problem comes when I try to deserialize it like so:
val e = Json.parse[Example]("""{'isNativeText': true, 'textEncodedBody': 'hello'}""")
I receive the following error:
com.codahale.jerkson.ParsingException: Invalid JSON. Needed [body],
but found [isNativeText, textEncodedBody].
Clearly, I have a constructor that will work...it just is not the default one. How can I force Jerkson to use this non-default constructor?
EDIT: I've attempted to use both the #JsonProperty and #JsonCreator annotation, but jerkson appears to disregard both of those.
EDIT2: Looking over the jerkson case class serialization source code, it looks like a case class method with the same name as its field will be used in the way that a #JsonProperty would function - that is, as a JSON getter. If I could do that, it would solve my problem. Not being super familiar with Scala, I have no idea how to do that; is it possible for a case class to have a user-defined method with the same name as one of its fields?
For reference, here is the code below that leads me to this conclusion...
private val methods = klass.getDeclaredMethods
.filter { _.getParameterTypes.isEmpty }
.map { m => m.getName -> m }.toMap
def serialize(value: A, json: JsonGenerator, provider: SerializerProvider) {
json.writeStartObject()
for (field <- nonIgnoredFields) {
val methodOpt = methods.get(field.getName)
val fieldValue: Object = methodOpt.map { _.invoke(value) }.getOrElse(field.get(value))
if (fieldValue != None) {
val fieldName = methodOpt.map { _.getName }.getOrElse(field.getName)
provider.defaultSerializeField(if (isSnakeCase) snakeCase(fieldName) else fieldName, fieldValue, json)
}
}
json.writeEndObject()
}
Correct me if I'm wrong, but it looks like Jackson/Jerkson will not support arbitrarily nested JSON. There's an example on the wiki that uses nesting, but it looks like the target class must have nested classes corresponding to the nested JSON.
Anyway, if you're not using nesting with your case classes then simply declaring a second case class and a couple implicit conversions should work just fine:
case class Example(body: Array[Byte]) {
// Note that you can just inline the body of bodyIsNativeText here
lazy val isNativeText: Boolean = // determine if the body was natively a string or not
}
case class ExampleRaw(isNativeText: Boolean, textEncodedBody: String)
implicit def exampleToExampleRaw(ex: Example) = ExampleRaw(
ex.isNativeText,
if (ex.isNativeText) new String(ex.body, "UTF-8")
else Base64.encode(ex.body)
)
implicit def exampleRawToExample(raw: ExampleRaw) = Example(
if (raw.isNativeText) raw.textEncodedBody.getBytes("UTF-8")
else Base64.decode(textEncodedBody)
)
Now you should be able to do this:
val e: Example = Json.parse[ExampleRaw](
"""{'isNativeText': true, 'textEncodedBody': 'hello'}"""
)
You could leave the original methods and annotations you added to make the JSON generation continue to work with the Example type, or you could just convert it with a cast:
generate(Example(data): ExampleRaw)
Update:
To help catch errors you might want to do something like this too:
case class Example(body: Array[Byte]) {
// Note that you can just inline the body of bodyIsNativeText here
lazy val isNativeText: Boolean = // determine if the body was natively a string or not
lazy val doNotSerialize: String = throw new Exception("Need to convert Example to ExampleRaw before serializing!")
}
That should cause an exception to be thrown if you accidentally pass an instance of Example instead of ExampleRaw to a generate call.