How do you exclude fields in Jackson json serialisation (Scala) - json

I am trying to exclude the _passthroughFields property in the example below. When I use the debugger, it looks like my PropertyFilter is never used. What am I doing wrong?
import java.util
import com.fasterxml.jackson.databind.ser.impl.SimpleBeanPropertyFilter.SerializeExceptFilter
import com.fasterxml.jackson.databind.ser.impl.SimpleFilterProvider
import com.fasterxml.jackson.databind.{DeserializationFeature, ObjectMapper, ObjectWriter}
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
import org.scalatest.{Matchers, WordSpec}
import scala.collection.immutable.Map
class PassthroughFieldsSpec extends WordSpec with Matchers {
"JacksonParser" when {
"given an Object and undesired fields" should {
"not include those fields in the json response" in {
trait Foo {
def id: String
def _passthroughFields: Map[String, String] = Map.empty
}
class Bar(val id: String, override val _passthroughFields: Map[String, String]) extends Foo
val item = new Bar("abcd", Map.empty)
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
val excludes = new util.HashSet[String](1)
excludes.add("_passthroughFields")
excludes.add("get_passthroughFields")
val filters = new SimpleFilterProvider()
.addFilter("filter properties by name", new SerializeExceptFilter(excludes))
val writer: ObjectWriter = mapper.writer(filters)
val json = writer.writeValueAsString(item)
json.contains("_passthroughFields") shouldBe false
}
}
}
}

I think you can exclude it using something like #JsonIgnore or make the field transient.
Otherwise if you need to define it outside of the case class code (like in your example) you can do it with Genson.
import com.owlike.genson._
import com.owlike.genson.reflect.VisibilityFilter
// Note that if you use case classes you don't need the visibility filter stuff
// it is used to indicate that private fields should be ser/de
val genson = new GensonBuilder()
.withBundle(ScalaBundle())
.useFields(true, VisibilityFilter.PRIVATE)
.exclude("_passthroughFields")
.create()
genson.toJson(item)
Disclaimer: I am Gensons author.

Related

how to return custom json in scalatra

Scalatra Code:
import org.scalatra._
import org.json4s.{DefaultFormats, Formats}
import org.scalatra.json._
class AppServlet extends AppStack with JacksonJsonSupport{
protected implicit lazy val jsonFormats: Formats = DefaultFormats
private def generateJSON():((String, String),(String, String)) = {
val json = ("Firstname" -> "joe", "LastName" -> "cole")
json
}
before() {
contentType = formats("json")
}
get("/") {
generateJSON
}
}
I am trying to return simple json using scalatra framework and the output is something like this {"_1":{"Firstname":"joe"},"_2":{"LastName":"cole"}}. I dont need _1 or _2 to be printed. Please note i am not trying to return any objects. I just need to make my own json and then return it. It is not associated with any data model. Any help is appreciated.
What you create is a tuple of (String, String), that's not surprising the output is like that. You should either create a case class, or, since you use json4s, return:
// don't forget this:
// import org.json4s.JsonDSL._
("Firstname" -> "joe") ~ ("LastName" -> "cole")
import org.scalatra._
import org.json4s.{DefaultFormats, Formats}
import org.scalatra.json._
import org.json4s._
import org.json4s.JsonDSL._
class AppServlet extends AppStack with JacksonJsonSupport{
protected implicit lazy val jsonFormats: Formats = DefaultFormats
private def generateJSON():JObject = {
val json = ("Firstname" -> "joe") ~ ("LastName" -> "cole")
json
}
get("/") {
generateJSON
}
}

remove tpe from json after serialization via scala-pickling

Is there an easy way to serialize to json without "tpe" field inside an object?
I need to serialize case classes to json structures and then, send them over the wire (They won't been deserialized in future). I have a specific api, so.. I don't need additional fields.
For class Person illustrated below:
case class Person(Name: String, Age: int, CandyLover: Boolean)
I'd like to see following:
{
"Name": "Paul",
"Age": 23,
"CandyLover": true
}
I would suggest you to take a look at spray-json or argonaut. Or play-json if you are already using Play.
Scala pickling is not really a json library, it was not created to generate JSON for public API, it's not flexible, you can't define JSON protocol first and then provide pickling serialization to match protocol. Pickling is for computer-to-computer serialization where no one is really care about bytes going between apps.
Probably the simplest way to do it is to pass somehow the Hint into JSONPickleBuilder. Quick way to do it is to override builders in JSONPickleFormat by calling hintStaticallyElidedType() method from PickleTools.
Here is kind of code snippet to demonstrate it:
import org.specs2.matcher.JsonMatchers
import org.specs2.mutable.Specification
import org.specs2.specification.Scope
import scala.pickling.Defaults._
import scala.pickling.json.{JSONPickleBuilder, JSONPickleFormat, JsonFormats}
import scala.pickling.{Output, PBuilder, PickleTools, StringOutput}
class PersonJsonFormatsTest extends Specification with JsonMatchers {
trait Context extends Scope with PersonJsonFormats
trait PersonJsonFormats extends JsonFormats {
override implicit val pickleFormat: JSONPickleFormat = new PersonJSONPickleFormat
}
class PersonJSONPickleFormat extends JSONPickleFormat {
override def createBuilder() = new JSONPickleBuilder(this, new StringOutput) with PickleTools {
hintStaticallyElidedType()
}
override def createBuilder(out: Output[String]): PBuilder = new JSONPickleBuilder(this, out) with PickleTools {
hintStaticallyElidedType()
}
}
"Pickle" should {
"Serialize Person without $type field in resulting JSON" in new Context {
case class Person(Name: String, Age: Int, CandyLover: Boolean)
val pickledPersonObject = Person("Paul", 23, CandyLover = true).pickle
println(pickledPersonObject.value)
pickledPersonObject.value must not */("$type" → ".*".r)
}
}
}
The output of println(pickledPersonObject.value) will be as you need:
{
"Name": "Paul",
"Age": 23,
"CandyLover": true
}
This way you will stay aligned with further pickle updates.
P.S. If someone knows more elegant and convenient way to reach the same behaviour - please let us know :-)
For scala-pickling 0.10.x:
import scala.pickling._
import scala.pickling.json.{JSONPickle, JSONPickleBuilder, JSONPickleFormat, JsonFormats}
import scala.pickling.pickler.AllPicklers
object serialization extends JsonFormats with Ops with AllPicklers {
override implicit val pickleFormat: JSONPickleFormat = new JSONPickleFormat {
private def setHints(h: Hintable): Unit = {
h.hintStaticallyElidedType()
h.hintDynamicallyElidedType()
}
override def createBuilder(): JSONPickleBuilder = {
val builder = super.createBuilder()
setHints(builder)
builder
}
override def createBuilder(out: Output[String]): PBuilder = {
val builder = super.createBuilder(out)
setHints(builder)
builder
}
override def createReader(pickle: JSONPickle): PReader = {
val reader = super.createReader(pickle)
setHints(reader)
reader
}
}
}
object SerializationTest extends App {
import serialization._
case class Person(firstName: String, lastName: String)
val pickle: JSONPickle = Person("Evelyn", "Patterson").pickle
val jsonString: String = pickle.value // {"firstName": "Evelyn","lastName": "Patterson"}
val person: Person = jsonString.unpickle[Person]
}
For scala-pickling 0.11.x:
import scala.pickling._
import scala.pickling.json.{JSONPickle, JSONPickleBuilder, JSONPickleFormat}
import scala.pickling.pickler.AllPicklers
object serialization extends AllPicklers {
private final class CustomJSONPickleFormat(tag: FastTypeTag[_]) extends JSONPickleFormat {
private def setHints(h: Hintable) {
h.hintElidedType(tag)
}
override def createBuilder(): JSONPickleBuilder = {
val b = super.createBuilder()
setHints(b)
b
}
override def createBuilder(out: Output[String]): PBuilder = {
val b = super.createBuilder(out)
setHints(b)
b
}
override def createReader(pickle: JSONPickle): PReader = {
val b = super.createReader(pickle)
setHints(b)
b
}
}
implicit val staticOnly = static.StaticOnly // for compile time serialization methods generation
implicit final class EncodeDecodeOps[T](picklee: T) {
def encode(implicit pickler: Pickler[T]): String = {
val pickleFormat = new CustomJSONPickleFormat(pickler.tag)
functions.pickle(picklee)(pickleFormat, pickler).value
}
def decode[A](implicit c: T => String, unpickler: Unpickler[A]): A = {
val pickleFormat = new CustomJSONPickleFormat(unpickler.tag)
functions.unpickle[A](json.JSONPickle(picklee))(unpickler, pickleFormat)
}
}
}
case class Person(firstName: String, lastName: String) {
#transient var x = "test"
}
object SerializationTest extends App {
import serialization._
val jsonString = Person("Lisa", "Daniels").encode
println(jsonString)
val person = jsonString.decode[Person]
println(person)
}

Recursive data types and custom serializers in spray-json

I have a recursive data structure that I want to write a custom spray-json serializer for.
case class Counts(var count: Int, var properties: mutable.Map[String, Counts])
object MyJsonProtocol extends DefaultJsonProtocol {
import DefaultJsonProtocol._
implicit object CountsJsonFormat extends RootJsonFormat[Counts] {
def read(json: JsValue) = ???
def write(c: Counts) = {
// Flatten count and properties into the same object.
val properties = c.properties.toJson.asJsObject
val fields = properties.fields + ("count" -> JsNumber(c.count))
JsObject(fields.toSeq: _*)
}
}
}
I've seen the documentation for how to do this for a case class if you use the builtin serialization logic, but I have no idea how to apply that to a custom serializer. I get this compiler error:
Cannot find JsonWriter or JsonFormat type class for scala.collection.mutable.Map[String,co.asku.acuity.EventCounter.Counts]
val properties = c.properties.toJson.asJsObject
^
spray-json formats can't handle mutable Maps by default (see this disussion that has happened a while ago on the mailing list). Change the type of properties to be an immutable Map (which I think is better anyways) and your format will work as expected.
To add to edi's answer, using toMap would have worked in the example I posted, to convert the mutable map to immutable.
However, I actually ran into a more complex usecase using nested mutable maps, so I just added a format to serialize them like this:
object JsonProtocol extends DefaultJsonProtocol {
import DefaultJsonProtocol._
implicit def mutableMapFormat[K :JsonFormat, V :JsonFormat] = new RootJsonFormat[mutable.Map[K, V]] {
def read(value: JsValue) = ???
def write(m: mutable.Map[K, V]) = m.toMap.toJson
}
implicit object CountsJsonFormat extends RootJsonFormat[Counts] {
// ...
}
}
This code provides serialization and deserialization support for mutable maps (and can be modified trivially for other mutable collections):
import spray.json._
import spray.json.DefaultJsonProtocol._
import scala.collection.mutable
...
implicit def mutableMapFormat[K : JsonFormat, V : JsonFormat] = new RootJsonFormat[mutable.Map[K, V]] {
def write(m : mutable.Map[K, V]) = mapFormat[K, V].write(m.toMap)
def read(value : JsValue) = mutable.Map.empty[K, V] ++ mapFormat[K, V].read(value)
}

Spray json marshalling

I'm on a quest to create a JSON API where some of the models could be nicely generalized. I'm a newbie in Spray, so I started out a spike with an over simplified example.
However I can't figure out what is going on with the bellow code...
I have imported both
my custom implicits and
spray.httpx.SprayJsonSupport._
As I understand this is what I have to do in order to have an implicit in scope that can convert from JsonFormat to Marshaller.
Compiler error:
TestService.scala:15: could not find implicit value for parameter um: spray.httpx.unmarshalling.FromRequestUnmarshaller[my.company.Test[my.company.X]]
Code:
package my.company
import spray.routing.HttpService
import spray.json.{JsValue, JsObject, JsonFormat, DefaultJsonProtocol}
trait TestService extends HttpService {
import my.company.TestImplicits._
import spray.httpx.SprayJsonSupport._
val test =
path("test") {
post {
entity(as[Test[X]]) {
test => {
complete(s"type: ${test.common}")
}
}
}
}
}
trait Common {
def commonData: String
}
case class X(id: Long, commonData: String) extends Common
case class Y(commonData: String) extends Common
case class Test[T <: Common](comment: String, common: T)
object TestImplicits extends DefaultJsonProtocol {
implicit val xFormat = jsonFormat2(X)
implicit val yFormat = jsonFormat1(Y)
implicit val yTestFormat: JsonFormat[Test[Y]] = new JsonFormat[Test[Y]] {
def write(test: Test[Y]) = JsObject()
def read(js: JsValue) = Test("test", Y("y"))
}
implicit val xTestFormat: JsonFormat[Test[X]] = new JsonFormat[Test[X]] {
def write(test: Test[X]) = JsObject()
def read(js: JsValue) = Test("test", X(1L, "y"))
}
}
I would appreciate any help. Thanks in advance.
SOLVED
Solution was (as #jrudolp suggested) both to:
Move implicit definitions on top of the file (surprising)
Create RootJsonFormat rather than JsonFormat.

JSON deserialization using Jackson in Scala

I'm new to Jackson. I am encountering the following error when using Jackson to deserialize a JSON string if I don't explicitly specify a class type during deserialization:
com.fasterxml.jackson.databind.JsonMappingException: Can not instantiate abstract type [simple type, class scala.runtime.Nothing$] (need to add/enable type information?)
Is there a way to serialize / deserialize JSON in Jackson without having to specify the class type?
Here is my test code:
import java.lang.reflect.{Type, ParameterizedType}
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.core.`type`.TypeReference;
import org.junit._
import org.junit.Assert._
case class Person (name:String)
case class Address (city:String, state:String)
class JSONTest {
#Test
def jsonTest () = {
val p = new Person ("Bob")
val json = JacksonWrapper.serialize(p)
println ("json= " + json)
val obj = JacksonWrapper.deserialize[Person](json)
println ("obj = " + obj)
// fails since class type isn't explictly specified.
// is there a way to do it so that class type is automatically determined?
val obj2 = JacksonWrapper.deserialize(json)
println ("obj= " + obj2)
}
}
//http://stackoverflow.com/questions/12591457/scala-2-10-json-serialization-and-deserialization
object JacksonWrapper {
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
def serialize(value: Any): String = {
import java.io.StringWriter
val writer = new StringWriter()
mapper.writeValue(writer, value)
writer.toString
}
def deserialize[T: Manifest](value: String) : T =
mapper.readValue(value, typeReference[T])
private [this] def typeReference[T: Manifest] = new TypeReference[T] {
override def getType = typeFromManifest(manifest[T])
}
private [this] def typeFromManifest(m: Manifest[_]): Type = {
if (m.typeArguments.isEmpty) { m.erasure }
else new ParameterizedType {
def getRawType = m.erasure
def getActualTypeArguments = m.typeArguments.map(typeFromManifest).toArray
def getOwnerType = null
}
}
}
You have to provide the type information somehow. Jackson serializes to JSON only your object's fields, it doesn't store additional information about the type anywhere, so for your Person class, the JSON probably looks like this: { "name": "Bob"}. Without you providing the information that you want to deserialize as an instance of Person, Jackson can't known the type. You could have another class with the field name as well, and JSON doesn't say which one you need. That's why you need to provide the type when deserializing - in Java by passing an argument of type Class and in Scala by providing the type parameter to ObjectMapper and friends.