Extracting lift-json into a case class with an upper bound - json

I have spent the last day searching and reading various sites and articles to try to find the answer to this question myself and I haven't found anything that helps. I'm not even sure if this is feasible. My problem is that I am trying to parse and extract a Json Response using lift-json. The response consists of 4 parts, where the first 3 parts are always the same for every response to every type of request I make. The last part varies depending on the type of request, but it is always going to be a list of some type. I was hoping to do something like this:
abstract class MyObjects
case class Apple(id: Int, name: String, color: String) extends MyObjects
case class Orange(id: Long, name: String, state: String) extends MyObjects
abstract class MyResponse
case class Fruits[T <: MyObjects](aisle: Int, bin: Int, hasWhat: Option[List[T]])
Where if I wanted to know what all the apples are, I would make a request for that and get back a response with a list of apples. When I try to extract this example:
myJson.extract[Fruits[Apple]]
I get this error:
net.liftweb.json.MappingException: do not know how to get type parameter from T
at net.liftweb.json.Meta$.fail(Meta.scala:128)
at net.liftweb.json.Meta$Reflection$.term$1(Meta.scala:206)
at net.liftweb.json.Meta$Reflection$.typeParameters(Meta.scala:220)
at net.liftweb.json.Meta$.mkContainer$1(Meta.scala:91)
at net.liftweb.json.Meta$.fieldMapping$1(Meta.scala:101)
at net.liftweb.json.Meta$.mkContainer$1(Meta.scala:90)
at net.liftweb.json.Meta$.fieldMapping$1(Meta.scala:107)
at net.liftweb.json.Meta$.toArg$1(Meta.scala:117)
at net.liftweb.json.Meta$$anonfun$constructors$1$1$$anonfun$apply$1.apply(Meta.scala:83)
at net.liftweb.json.Meta$$anonfun$constructors$1$1$$anonfun$apply$1.apply(Meta.scala:82)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:...
I'm using lift-json 2.1 & scala 2.8. I do have a way to work around it, by specifically creating a case class for each type of response, but I thought what I was trying to do was cleaner. Just wanted to know if
a) is this even possible?
b) if so, what am I doing wrong?
EDIT...sample application:
val apples = """{ "aisle" : 1, "bin" : 1,
"hasWhat" : [{ "id" : 4, "name" : "Granny", "color" : "green"},
{ "id" : 4, "name" : "Fuji", "color" : "red"}] }"""
val oranges = """ { "aisle" : 3, "bin" : 2,
"hasWhat" : [{ "id" : 2, "name" : "Navel", "state" : "FL" },
{ "id" : 2, "name" : "Clementine", "state" : "Spain" }]}"""
scala> val aJson = parse(apples)
aJson: net.liftweb.json.JsonAST.JValue = JObject(List(JField(aisle,JInt(1)), JField(bin,JInt(1)), JField(hasWhat,JArray(List(JObject(List(JField(id,JInt(4)), JField(name,JString(Granny)), JField(color,JString(green)))), JObject(List(JField(id,JInt(4)), JField(name,JString(Fuji)), JField(color,JString(red)))))))))
scala> val oJson = parse(oranges)
oJson: net.liftweb.json.JsonAST.JValue = JObject(List(JField(aisle,JInt(3)), JField(bin,JInt(2)), JField(hasWhat,JArray(List(JObject(List(JField(id,JInt(2)), JField(name,JString(Navel)), JField(state,JString(FL)))))))))
scala> val doesntWork = aJson.extract[Fruits]
doesntWork: org.spin.node.gogrid.objects.Fruits = Fruits(1,1,None)
scala> val works = aJson.extract[AFruit]
works: org.spin.node.gogrid.objects.AFruit = AFruit(1,1,Some(List(Apple(4,Granny,green), Apple(4,Fuji,red))))
I want doesntWork to be like works, Where:
case class AFruit(aisle: Int, bin: Int, hasWhat: Option[List[Apple]])
Thanks!
-newbie

Extracting parameterized case class is not yet supported. One workaround (not sure if this works for your case though) is to make Fruits a concrete type and add the type information into JSON.
import net.liftweb.json._
import net.liftweb.json.Extraction._
import net.liftweb.json.JsonAST._
import net.liftweb.json.Printer._
abstract class MyObjects
case class Apple(id: Int, name: String, color: String) extends MyObjects
case class Orange(id: Long, name: String, state: String) extends MyObjects
case class Fruits(aisle: Int, bin: Int, hasWhat: Option[List[MyObjects]])
object Test extends Application {
// This configuration adds an extra field for MyObjects to JSON
// which tells the real type of a MyObject.
implicit val formats = Serialization.formats(FullTypeHints(List(classOf[MyObjects])))
val fs = Fruits(0, 0, Some(List(
Apple(1, "Granny Smith", "green"),
Apple(2, "Grenade", "red"))))
val json = decompose(fs)
println(pretty(render(json)))
assert (json.extract[Fruits] == fs)
}
That prints:
{
"aisle":0,
"bin":0,
"hasWhat":[{
"jsonClass":"Apple",
"id":1,
"name":"Granny Smith",
"color":"green"
},{
"jsonClass":"Apple",
"id":2,
"name":"Grenade",
"color":"red"
}]
}

Related

How do I get circe to decode nested json with kebab-case attribute names

I started with the accepted answer to SO question 53573659 which has a nested list of attrs and uses the auto-parser to get the data into case classes. I want to be able to handle the same data but with the nested fields having kebab-case rather than camel case.
Here is the same input JSON with the kebab-case fields
val sampleKebab="""{
"parent" : {
"name" : "title",
"items" : [
{
"foo" : "foo1",
"attrs" : {
"attr-a" : "attrA1",
"attr-b" : "attrB1"
}
},
{
"foo" : "foo2",
"attrs" : {
"attr-a" : "attrA2",
"attr-b" : "attrB2",
"attr-c" : "attrC2"
}
}
]
}
}"""
I can decode the attrs data by itself using the following example
import io.circe.derivation.deriveDecoder
import io.circe.{Decoder, derivation}
import io.circe.generic.auto._
import io.circe.parser._
val attrKebabExample = """{
"attr-a": "attrA2",
"attr-b": "attrB2",
"attr-c": "attrC2"
}"""
case class AttrsKebab(attrA: String, attrB: String)
implicit val decoder: Decoder[AttrsKebab] = deriveDecoder(derivation.renaming.kebabCase)
val attrKebabData = decode[AttrsKebab](attrKebabExample)
attrKebabData decodes to
Either[io.circe.Error,AttrsKebab] = Right(AttrsKebab(attrA2,attrB2))
When I try to tie this decoder into the case class hierarchy from the original question, it exposes some glue that I am missing to hold it all together
case class ItemKebab(foo: String, attrs : AttrsKebab)
case class ParentKebab(name: String, items: List[ItemKebab])
case class DataKebab(parent : ParentKebab)
case class Data(parent : Parent)
val dataKebab=decode[DataKebab](sample)
In this case, dataKebab contains a DecodingFailure
Either[io.circe.Error,DataKebab] = Left(DecodingFailure(Attempt to decode value on failed cursor, List(DownField(attr-a), DownField(attrs), DownArray, DownField(items), DownField(parent))))
My guess is that either the decoder I defined is being ignored, or I need to explicitly define more of the decode process, but I'm looking for some help to find what the solution might be.

Trying unparse json string, but getting Expected start of the object '{', but had 'EOF' instead

I am trying to parse a json file into a list using kotlin serializable.
Here are my data classes.
#Serializable
data class Book(
val epub : String,
val fb2 : String,
val mobi : String,
val djvu : String,
val title : String,
val author : String,
val anotation: String,
val cover_uri : String,
)
#Serializable
data class Books(
#Serializable (with = BookListSerializer::class)
val books : List<Book>
)
object BookListSerializer : JsonTransformingSerializer < List < Book >> ( ListSerializer ( Book.serializer ()))
Here I am trying to parse a string
val books = Json.decodeFromString<Books>(stringJson)
Here my Json String
[
{
"anotation": "Этот город",
"author": "Чарльз Плэтт",
"cover_uri": "null",
"djvu": "null",
"epub": "/b/301494/epub",
"fb2": "/b/301494/fb2",
"mobi": "/b/301494/mobi",
"title": "New York Times (Пульс Нью-Йорка) (fb2)"
},
{
"anotation": "Способна л",
"author": "Триш Уайли",
"cover_uri": "/i/45/390445/cover.jpg",
"djvu": "null",
"epub": "/b/390445/epub",
"fb2": "/b/390445/fb2",
"mobi": "/b/390445/mobi",
"title": "Лучший мужчина Нью-Йорка (fb2)"
}
]
And i always getting this error
kotlinx.serialization.json.internal.JsonDecodingException: Expected start of the object '{', but had 'EOF' instead
JSON input: .....2","mobi":"/b/49442/mobi","title":"I love New York (fb2)"}]
I would be very glad and grateful for any help
tl;dr
Exchange this
val books = Json.decodeFromString<Books>(stringJson)
with this
val books = Json.decodeFromString<List<Book>>(stringJson)
You're trying to deserialize an JSON array [ ... ] but declare an object of type Books as target when calling decodeFromString, thus something like { books: [ ... ] }.
You either have to wrap your JSON array in the property books of an JSON object or change the expected type during deserialization to List<Book>.
Thus, besides the above solution, you could also do the following:
val wrappedStringJson = """
{
"books": $stringJson
}
""".trimIndent()
val books = Json.decodeFromString<Books>(wrappedStringJson)
I experienced the same issue during testing on Ktor Server.
fun testFun() = testApplication { ....
val response = client.get("/boruto/heroes")
val actual = Json.decodeFromString<ApiResponse>(response.content.toString())
....
}
The issue was that I was using this content instead of body.
val actual = Json.decodeFromString<T>(response.content.toString())
I changed it to this and the test passed
val actual = Json.decodeFromString<T>(response.body())
Leaving this here in case someone encounters the same issue.

How can the polymorphic with internal property type can be handled in Kotlin?

Good day, Mr. Freeman! I'm trying to make a non trivial polymorphic dto for Json / Object conversion for an external api. The dto contains the property, that can be of two types, but it depends on it's internal value... Let's say that i have a such json:
[{
"Id": 1,
"Age": 2,
"Car": {
"MaxPassengers": 20,
"Model": "Audi",
"UniqueAudyTechnology": true
},
"Vendor": "VOLKSWAGEN AUTO GROUP (VAG)"
},
{
"Id": 2,
"Age": 1,
"Car": {
"MaxPassengers": 5,
"Model": "Skoda",
"SkodaRentalProgramId": 100
},
"Vendor": "VOLKSWAGEN AUTO GROUP (VAG)"
}]
So, in "Car" field i can have any car class, but to define it i need to use Car.Model property.
I've made a common interface and data classes:
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.EXISTING_PROPERTY,
property = "Model",
visible = true
)
#JsonSubTypes(value = [
JsonSubTypes.Type(value = Audi::class, name = "Audi"),
JsonSubTypes.Type(value = Skoda::class, name = "Skoda")
])
#ApiModel(description = "Used car")
interface UsedCar {
#get:JsonProperty("Id")
val id: Long
#get:JsonProperty("Age")
val age: Int
#get:JsonProperty("Vendor")
val vendor: String
}
and the data classes:
data class Audi(
#JsonProperty("UniqueAudyTechnology")
val hasUniqueAudyTechnology: boolean,
#JsonProperty("Model")
val model: String,
#JsonProperty("MaxPassengers")
val maxPassengers: Int
)
data class Skoda(
#JsonProperty("SkodaRentalProgramId")
val skodaRentalProgramId: Int,
#JsonProperty("Model")
val model: String,
#JsonProperty("MaxPassengers")
val maxPassengers: Int
)
In fact, i want to make jackson resolve subtype by subtypes property Model. I keep trying all the day, but i can't understand what did i miss...?
P.S. the code may not work because i've removed implementation of UsedCar by Audi and Skoda... sorry... no idea how to handle it...
IMHO this is the easiest solution for an API that sends multiple unused fields.
#JsonIgnoreProperties(ignoreUnknown = true)
data class Car(
#JsonProperty("Id")
val id: Long,
#JsonProperty("Age")
val age: Int,
#JsonProperty("Vendor")
val vendor: String,
#JsonProperty("UniqueAudiTechnology")
val hasUniqueAudiTechnology: Boolean,
#JsonProperty("SkodaRentalProgramId")
val skodaRentalProgramId: Int,
#JsonProperty("Model")
val model: String,
#JsonProperty("MaxPassengers")
val maxPassengers: Int)
It can get really tedious if there are many fields, but that's a bad API design in the first place.
Check this answer for more info in the above way.

json4s parse json partially

I have a json model, where contents of certain attribute depend on the other attribute. Something like this:
"paymentMethod": "CREDIT_CARD",
"metaData": {
"cardType": "VISA",
"panPrefix": "",
"panSuffix": "",
"cardHolder": "",
"expiryDate": ""
}
So when paymentMethod equals to CREDIT_CARD, the metadata object will contain attributes as described. In case of other payment method, there'll be different metadata.
I want to handle this situation in a future-proof way. What I'm trying to do is to not parse the metadata field right away, but keep it somehow "unparsed" until I've parsed the paymentMethod field. Then I'd take the metadata and applied appropriate parsing approach.
However I don't know which type to use for a Scala class field for such "late parsed" attributes. I've tried String, JsonInput, JObject, and they all are not suitable (either don't compile or can't be parsed). Any ideas which type can I use? Or, in other words:
case class CreditCardMetadata(
cardType: String,
panPrefix: String,
panSuffix: String,
cardHolder: String,
expiryDate: String)
case class PaypalMetadata(...) // etc.
case class PaymentGatewayResponse(
paymentMethod: String,
metadata: ???)
You could create a CustomSerializer to parse the metadata directly. Something like :
case class PaymentResponse(payment: Payment, otherField: String)
sealed trait Payment
case class CreditCardPayment(cardType: String, expiryDate: String) extends Payment
case class PayPalPayment(email: String) extends Payment
object PaymentResponseSerializer extends CustomSerializer[PaymentResponse]( format => (
{
case JObject(List(
JField("paymentMethod", JString(method)),
JField("metaData", metadata),
JField("otherField", JString(otherField))
)) =>
implicit val formats = DefaultFormats
val payment = method match {
case "CREDIT_CARD" => metadata.extract[CreditCardPayment]
case "PAYPAL" => metadata.extract[PayPalPayment]
}
PaymentResponse(payment, otherField)
},
{ case _ => throw new UnsupportedOperationException } // no serialization to json
))
Which can be used as:
implicit val formats = DefaultFormats + PaymentResponseSerializer
val json = parse("""
{
"paymentMethod": "CREDIT_CARD",
"metaData": {
"cardType": "VISA",
"expiryDate": "2015"
},
"otherField": "hello"
}
""")
val json2 = parse("""
{
"paymentMethod": "PAYPAL",
"metaData": {
"email": "foo#bar.com"
},
"otherField": "world"
}
""")
val cc = json.extract[PaymentResponse]
// PaymentResponse(CreditCardPayment(VISA,2015),hello)
val pp = json2.extract[PaymentResponse]
// PaymentResponse(PayPalPayment(foo#bar.com),world)
You can use a Map[String, String].
It will contain anything you may need.
The answer by Peter Neyens has inspired me to implement my own solution. It's not as generic as his, but in my case I needed something really simple and ad-hoc. Here's what I've done:
It's possible to define a case class with the field of unknown type is represented by a JObject type. Something like this:
case class PaymentGatewayResponse(
default: Boolean,
paymentMethod: String,
visibleForCustomer: Boolean,
active: Boolean,
metaData: JObject)
When such json is parsed into such case class, this field is not parsed immediately, but contains all the necessary information. Then it's possible parse it in a separate step:
case class CreditCardMetadata(
cardType: String,
cardObfuscatedNumber: String,
cardHolder: String,
expiryDate: String)
val response: PaymentGatewayResponse = doRequest(...)
response.map { r =>
r.paymentMethod match {
case "CREDIT_CARD" => r.metaData.extract[CreditCardMetadata]
case unsupportedType: String => throw new UnsupportedPaymentMethodException(unsupportedType)
}
}

Jackson Scala JSON Deserialization to case classes

I have a JSON which has following form:
{
"inventory": [
{
"productType": "someProduct1",
"details": {
"productId": "Some_id",
"description": "some description"
}
},
{
"productType": "someProduct2",
"details": {
"productId": "Some_id",
"description":{"someKey":"somevalue"}
}
}
]
}
The case classes that I want the above json to deserialize look like following:
case class Inventory(products:List[Product])
case class Product(productType:String,details:ProductDetails)
abstract class ProductDetails
case class ProductDetailsSimple(productId:String,description:String) extends ProductDetails
case class ProductDetailsComplex(productId:String,description:Map[String,String]) extends ProductDetails
I am using jackson-scala module to deserialize the above JSON string as follows:
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.readValue(jsonBody, classOf[Inventory])
The error I get is as follows:
"Unexpected token (END_OBJECT), expected FIELD_NAME: missing property '#details' that is to contain type id (for class ProductDetails)\n at [Source: java.io.StringReader#12dfbabd; line: 9, column: 5]"
I have been through jackson documentation on Polymorphic deserialization and have tried combinations as mentioned but with no luck.
I would like to understand what I am doing wrong here, which needs correction with respect to deserialization using jackson module.
I think there's a few separate problems to address here, so I've listed three separate approaches.
TL;DR
Either use Jackson polymorphism correctly or, in your case, go to a simpler approach and remove the need for the polymorphism. See my code on github.
1. Custom Deserializer
Your formatted JSON is:
{ inventory:
[ { productType: 'someProduct1',
details:
{ productId: 'Some_id',
description: 'some description' } },
{ productType: 'someProduct2',
details:
{ productId: 'Some_id',
description: { someKey: 'somevalue' }
}
}
]
}
The field productType is misplaced, in my opinion, but if this format is a strict requirement then you could write your own deserializer that looks at the productType field and instantiates a different concrete class.
I don't think this would be the best solution so I didn't write example code, but I like the Joda date-time package as a reference for custom serialize/deserialize
2. Jackson Polymorphism
You've separated Product from ProductDetails with a type field:
case class Product(productType:String,details:ProductDetails)
abstract class ProductDetails
I think you've confused how Jackson's polymorphic data type handling works and complicated your class design as a result.
Perhaps your business rules require that a product has a "type", in which case I'd name it "kind" or some other non-code label, and put it into what you've called ProductDetails.
But if "type" was included in an attempt to get type polymorphism working, then it isn't the right way.
I've included the below as a working example of Jackson polymorphism in Scala:
/**
* The types here are close to the original question types but use
* Jackson annotations to mark the polymorphic JSON treatment.
*/
import scala.Array
import com.fasterxml.jackson.annotation.JsonSubTypes.Type
import com.fasterxml.jackson.annotation.{JsonSubTypes, JsonTypeInfo}
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.PROPERTY,
property = "type")
#JsonSubTypes(Array(
new Type(value = classOf[ProductDetailsSimple], name = "simple"),
new Type(value = classOf[ProductDetailsComplex], name = "complex")
))
abstract class Product
case class ProductDetailsSimple(productId: String, description: String) extends Product
case class ProductDetailsComplex(productId: String, description: Map[String, String]) extends Product
case class PolymorphicInventory(products: List[Product])
Note that I removed the Product vs ProductDetails distinction, so an Inventory now just as a list of Product. I left the names ProductDetailsSimple and ProductDetailsComplex though I think they should be renamed.
Example usage:
val inv = PolymorphicInventory(
List(
ProductDetailsSimple(productId="Some_id", description="some description"),
ProductDetailsComplex(productId="Some_id", description=Map("someKey" -> "somevalue"))
)
)
val s = jsonMapper.writerWithDefaultPrettyPrinter().writeValueAsString(inv)
println("Polymorphic Inventory as JSON: "+s)
Output:
Polymorphic Inventory as JSON: {
"products" : [ {
"type" : "simple",
"productId" : "Some_id",
"description" : "some description"
}, {
"type" : "complex",
"productId" : "Some_id",
"description" : {
"someKey" : "somevalue"
}
} ]
}
3. Remove the polymorphism
I suggest that polymorphism in this case isn't needed at all, and that the error is in trying to make "description" either a single string or a key/value map when they are really fields with distinct intentions.
Perhaps there is a data legacy issue involved (in which case see the custom deser suggestion), but if the data is in your control, I vote for "go simpler":
case class Product(productId: String,
description: String="",
attributes: Map[String, String]=Map.empty)
case class PlainInventory(products: List[Product])
I's more "scala-rific" to use Option to indicate the absence of a value, so:
case class Product(productId: String,
description: Option[String]=None,
attributes: Option[Map[String, String]]=None)
Example usage:
val inv = PlainInventory(
List(
Product(productId="Some_id", description=Some("some description")),
Product(productId="Some_id", attributes=Some(Map("someKey" -> "somevalue")))
)
)
val s = jsonMapper.writerWithDefaultPrettyPrinter().writeValueAsString(inv)
println("Plain Inventory as JSON: "+s)
Output:
Plain Inventory as JSON: {
"products" : [ {
"productId" : "Some_id",
"description" : "some description"
}, {
"productId" : "Some_id",
"attributes" : {
"someKey" : "somevalue"
}
} ]
}
Working minimal code on github.