I'm trying to create an option type customer serialiser here is the JsonConverter
I've got:
type OptionConverter<'T> =
inherit JsonConverter<'T>
override __.WriteJson (writer: Utf8JsonWriter, value: 'T option, serializer: JsonSerializerOptions) : unit =
match value with
| Some optionValue ->
JsonSerializer.Serialize(writer, optionValue, serializer)
| None ->
writer.WriteNullValue()
override __.ReadJson (reader: byref<Utf8JsonWriter>, typeToConvert: Type, serializer: JsonSerializerOptions): 'T =
failwith "Not implemented"
type OptionConverterFactory =
inherit JsonConverterFactory
override __.CanConvert(typeToConvert: Type) =
let isOption = if typeToConvert.GetGenericTypeDefinition() = typeof<option<_>> then true else false
isOption
override __.CreateConverter(typeToConvert: Type, options: JsonSerializerOptions) : JsonConverter =
let optionType = typeToConvert.GetGenericArguments().[0]
let converter = Activator.CreateInstance(typeof<OptionConverter<_>>.MakeGenericType([| optionType |])) :?> JsonConverter
converter
I get red line underneath WriteJson saying no abstract or interface member found that corresponds with this override...
Also, if I change WriteJson and ReadJson to member then OptionConvert<'T> gets red underline saying that WriteJson and ReadJson have not been implemented...
How do I fix this?
I think there are two problems here:
The wrong methods are being overloaded
You need to constrain your 'T to be an option.
This compiled:
open System
open System.Text.Json
open System.Text.Json.Serialization
type OptionConverter<'T> =
inherit JsonConverter<Option<'T>>
override _.Write(writer, value, serializer) =
match value with
| Some optionValue ->
JsonSerializer.Serialize(writer, optionValue, serializer)
| None ->
writer.WriteNullValue()
override _.Read(reader, typeToConvert, options) =
failwith "Not implemented"
Related
I have two different maps that are created in my project. I have used type alias for both.
type alias1 = Map[String, ScalaObject]
type alias2 = Map[String, String]
I have a match-case situation where I want to differentiate between the two because different operations need to happen on both.
val obj: T = fromJson[T](jacksonMapper, json)
obj match {
case _: alias1 => operation1()
case _: alias2 => operation2()
case _ => obj
}
Any idea how to differentiate the two?
ADT would be to create a sealed trait with some case classes for each case; so one for a Map[String, String] and other for Map[String, ScalaObject] and you can pattern match on the case classes.
So like this:
sealed trait MyType extends Product with Serializable
final case class ObjMap(data: Map[String, ScalaObject]) extends MyType
final case class StrMap(data: Map[String, String]) extends MyType
val obj: MyType = ???
obj match {
case ObjMap(_) => operation1()
case StrMap(_) => operation2()
}
The Typeclass approach may be too complex for this.
The two key concepts to understand why the two maps cannot be differentiated at runtime are
type erasure where compile-time parameteriseed type Map[String, String] becomes runtime class Map
pattern matching translates to runtime isInstanceOf and asInstanceOf calls
Consider the following simplified example
case class Foo[T](i: T)
val fooInt = Foo[Int](42)
val fooStr = Foo[String]("")
fooInt.isInstanceOf[Foo[String]]
// val res0: Boolean = true
Note how isInstanceOf could not check at runtime what was the compile-time type parameter T and so it cannot differentiate between fooInt and fooStr. Effectively the best we can do is something like
fooInt.isInstanceOf[Foo[_]]
where the underscore _ serves to communicate the fact of type erasure.
Next consider how the following pattern match
(fooInt: Any) match { case str: Foo[String] => "oops :(" }
// val res1: String = oops :(
effectively becomes something like
if (fooInt.isInstanceOf[Foo[String]]) "oops :("
which again due to type erasure incorrectly evaluates to "oops :(".
An alternative approach is to try to do as much as possible at compile-time before type arguments are discarded. Typeclasses can be thought of as kind of compile-time pattern matching that happens before type erasure.
Is there any way of pattern matching objects where the objects may be Set[Foo] or Set[Bar] when the matching object can be any Object.
Given the below code, trying to pattern match on Set[Bar] will result in a match of Set[Foo] because of type erasure.
import play.api.libs.json._
import scala.collection.immutable.HashMap
case class Foo(valOne: Int, valTwo: Double)
object Foo {
implicit val writesFoo = Json.writes[Foo]
}
case class Bar(valOne: String)
object Bar {
implicit val writesBar = Json.writes[Bar]
}
case class TestRequest(params: Map[String, Object])
object TestRequest {
import play.api.libs.json.Json.JsValueWrapper
implicit val writeAnyMapFormat = new Writes[Map[String, Object]] {
def writes(map: Map[String, Object]): JsValue = {
Json.obj(map.map {
case (s, a) => {
val ret: (String, JsValueWrapper) = a match {
case _: String => s -> JsString(a.asInstanceOf[String])
case _: java.util.Date => s -> JsString(a.asInstanceOf[String])
case _: Integer => s -> JsString(a.toString)
case _: java.lang.Double => s -> JsString(a.toString)
case None => s -> JsNull
case foo: Set[Foo] => s -> Json.toJson(a.asInstanceOf[Set[Foo]])
case bar: Set[Bar] => s -> Json.toJson(a.asInstanceOf[Set[Bar]])
case str: Set[String] => s -> Json.toJson(a.asInstanceOf[Set[String]])
}
ret
}}.toSeq: _*)
}
}
implicit val writesTestRequest = Json.writes[TestRequest]
}
object MakeTestRequest extends App {
val params = HashMap[String, Object]("name" -> "NAME", "fooSet" -> Set(Foo(1, 2.0)), "barSet" -> Set(Bar("val1")))
val testRequest = new TestRequest(params)
println(Json.toJson(testRequest))
}
Trying to serialise the TestRequest will result in:
Exception in thread "main" java.lang.ClassCastException: Bar cannot be cast to Foo
Delegating the pattern matching of Sets to another method in an attempt to get the TypeTag,
case _ => s -> matchSet(a)
results in the type, unsurprisingly, of Object.
def matchSet[A: TypeTag](set: A): JsValue = typeOf[A] match {
case fooSet: Set[Foo] if typeOf[A] =:= typeOf[Foo] => Json.toJson(set.asInstanceOf[Set[Foo]])
case barSet: Set[Bar] if typeOf[A] =:= typeOf[Bar] => Json.toJson(set.asInstanceOf[Set[Bar]])
}
The runtime error being:
Exception in thread "main" scala.MatchError: java.lang.Object (of class scala.reflect.internal.Types$ClassNoArgsTypeRef)
A workaround could be to check the instance of the first element in the Set but this seems inefficient and ugly. Could also match on the key eg fooSet or barSet but if the keys are the same name eg both called set, then this wouldn't work.
In 2.11 s there any way to get at the type/class the Set has been created with?
You could use Shapeless typeable. Note that this is still not 100% safe (e.g. empty lists of different types cannot be distinguished at runtime, because that information literally doesn't exist); under the hood it's doing things like checking the types of the elements using reflection, just with a nicer interface on top.
In general it's better to carry the type information around explicitly, e.g. by using a a case class (or a shapeless HMap) rather than an untyped Map. Less good, but still better than nothing, is using wrapper case classes for the different types of Set that are possible, so that each one is a different type at runtime.
(Also half the point of the pattern match is to avoid the asInstanceOf; you should use e.g. foo rather than a.asInstanceOf[Set[Foo]])
I want to convert a case class with an Option[DateTime] parameter to a spray-json object which can be served by an API. Using spray-json I have a custom JsonFormat as such
object JsonImplicits extends DefaultJsonProtocol {
implicit object PostJsonFormat extends RootJsonFormat[Post] {
def write(p: Post) = JsObject(
"title" -> JsString(p.title),
"content" -> JsString(p.content),
"author" -> JsString(p.author),
"creationDate" -> JsString(p.creationDate.getOrElse(DateTime.now))
)
}
}
But I get:
overloaded method value apply with alternatives:
(value: String)spray.json.JsString <and>
(value: Symbol)spray.json.JsString
cannot be applied to (com.github.nscala_time.time.Imports.DateTime)
"creationDate" -> JsString(p.creationDate.getOrElse(DateTime.now))
when I try to compile it and no matter what I try I can't seem to convert the DateTime object to a string. For instance, when I try calling toString I get
ambiguous reference to overloaded definition,
both method toString in class AbstractDateTime of type (x$1: String, x$2: java.util.Locale)String
and method toString in class AbstractDateTime of type (x$1: String)String
match expected type ?
"creationDate" -> JsString(p.creationDate.getOrElse(DateTime.now.toString)))
You have several problems here.
First, the toString() method in AbstractDateTime requires one or several arguments see here.
But I would advise you against this path and recommend using properly Spray-Json.
Spray-json does not know how to serialize Option[DateTime], therefore you have to provide a RootJsonFormat for it.
This is what I am doing.
implicit object DateJsonFormat extends RootJsonFormat[DateTime] {
private val parserISO : DateTimeFormatter = ISODateTimeFormat.dateTimeNoMillis();
override def write(obj: DateTime) = JsString(parserISO.print(obj))
override def read(json: JsValue) : DateTime = json match {
case JsString(s) => parserISO.parseDateTime(s)
case _ => throw new DeserializationException("Error info you want here ...")
}
}
Adapt it as you want if you do not want to use ISO formatting.
I have one simply question: Is it possible to parse F# Map type from json? Because when I try it (With F# Map<string, string>), it is easy to serialize and it looks how it have to, but when I try to deserialize it is throwing an exception.
Newtonsoft.Json.JsonSerializationException: Unable to find a default constructor to use for type Microsoft.FSharp.Collections.FSharpMap`2[System.Int32,System.String]. Path '1', line 2, position 7.
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewDictionary (Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonDictionaryContract contract, System.Boolean& createdFromNonDefaultConstructor) [0x00000] in <filename unknown>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00000] in <filename unknown>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00000] in <filename unknown>:0
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, Boolean checkAdditionalContent) [0x00000] in <filename unknown>:0
And it is deserializing from classic:
Map.ofList [ ("1", "one"); ("2", "two"); ("3", "three") ]
The resulting JSON looks like C# dictionary
{
"1": "one",
"2": "two",
"3": "three"
}
It is serializing without settings (Only indentation). So is it possible to serialize this, or is there some working workaround?
Thanks for answer
You can make your own converter to do this. It's a lot of reflection and constructing appropriate generic types, but it can be done.
You first deserialize to a Dictionary<Key, Val>, then create and fill a List<Tuple<Key, Val>> manually via reflection (because the Map constructor requires Tuples, not KeyValuePairs), then finally pass that into the Map constructor.
Not sure if there's an easier way, but this is what I came up with:
open System
open System.Collections
open System.Collections.Generic
open Newtonsoft.Json
let mapConverter = {
new JsonConverter() with
override x.CanConvert(t:Type) =
t.IsGenericType && t.GetGenericTypeDefinition() = typedefof<Map<_, _>>
override x.WriteJson(writer, value, serializer) =
serializer.Serialize(writer, value)
override x.ReadJson(reader, t, _, serializer) =
let genArgs = t.GetGenericArguments()
let generify (t:Type) = t.MakeGenericType genArgs
let tupleType = generify typedefof<Tuple<_, _>>
let listType = typedefof<List<_>>.MakeGenericType tupleType
let create (t:Type) types = (t.GetConstructor types).Invoke
let list = create listType [||] [||] :?> IList
let kvpType = generify typedefof<KeyValuePair<_, _>>
for kvp in serializer.Deserialize(reader, generify typedefof<Dictionary<_, _>>) :?> IEnumerable do
let get name = (kvpType.GetProperty name).GetValue(kvp, null)
list.Add (create tupleType genArgs [|get "Key"; get "Value"|]) |> ignore
create (generify typedefof<Map<_, _>>) [|listType|] [|list|]
}
Once you have your converter, then you just pass it into the DeserializeObject method and JsonConvert will use it wherever appropriate.
let str = JsonConvert.SerializeObject (Map<_, _> [333, 1234])
JsonConvert.DeserializeObject<Map<int, int>>(str, mapConverter)
The nice thing about doing it this way is that if you've got a big/deep record where your Map is just a single field, then it'll work with that too--you don't have to go changing your record structure to use Dictionaries just to support serialization.
This functionality became part of JSON.Net in version 6.0.3. (April 30th, 2014)
But, if you are stuck for some reason using an earlier version then a simplified (and more efficient as less reflection) version of Dax Fohl's version could be:
type mapConvert<'f,'t when 'f : comparison>() =
static member readJson (reader:JsonReader, serializer:JsonSerializer) =
serializer.Deserialize<Dictionary<'f, 't>> (reader)
|> Seq.map (fun kv -> kv.Key, kv.Value)
|> Map.ofSeq
let mapConverter = {
new JsonConverter() with
override __.CanConvert (t:Type) =
t.IsGenericType && t.GetGenericTypeDefinition() = typedefof<Map<_, _>>
override __.WriteJson (writer, value, serializer) =
serializer.Serialize(writer, value)
override __.ReadJson (reader, t, _, serializer) =
let converter =
typedefof<mapConvert<_,_>>.MakeGenericType (t.GetGenericArguments())
let readJson =
converter.GetMethod("readJson")
readJson.Invoke(null, [| reader; serializer |])
}
The problem is that json.net can't construct a Map<int,string>. However if you deserialize to a regular .net Dictionary<int,string> it will work, as the json is the same.
You can't serialize F#'s Map directly, since it has no default contructor (constructor with no parameter) at all.
This is the original documentation of F# map: (from http://msdn.microsoft.com/en-us/library/ee353686%28v=vs.110%29.aspx)
[<Sealed>]
type Map<[<EqualityConditionalOnAttribute>] 'Key,[<ComparisonConditionalOnAttribute>] [<EqualityConditionalOnAttribute>] 'Value (requires comparison)> =
class
interface IEnumerable
interface IComparable
interface IEnumerable
interface ICollection
interface IDictionary
new Map : seq<'Key * 'Value> -> Map< 'Key, 'Value>
member this.Add : 'Key * 'Value -> Map<'Key, 'Value>
member this.ContainsKey : 'Key -> bool
member this.Remove : 'Key -> Map<'Key, 'Value>
member this.TryFind : 'Key -> 'Value option
member this.Count : int
member this.IsEmpty : bool
member this.Item ('Key) : 'Value
end
As you see above, Map doesn't have default constructor but the serializer need a class with default constructor.
The best way to serialize a map is mapping the map to be regular .NET dictionary, but then the new dictionary doesn't have all of the advantages of F#'s Map, especially the immutability of F#'s Map.
Given this simple example the third test "de-serialize" fails with the message.
Can not construct instance of com.egc.ost.pricing.contracts.response.A, problem: abstract types either need to be mapped to concrete types, have custom deserializer, or be instantiated with additional type information
at [Source: java.io.StringReader#1f03691; line: 2, column: 29]
com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.egc.ost.pricing.contracts.response.A, problem: abstract types either need to be mapped to concrete types, have custom deserializer, or be instantiated with additional type information
at [Source: java.io.StringReader#1f03691; line: 2, column: 29]
Even though the second test "de-serialize_a" proves that Jackson can resolve the correct polymorphic types.
I am using Jackson version and scala module
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.10</artifactId>
<version>2.2.2</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-joda</artifactId>
<version>2.2.2</version>
</dependency>
Code:
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.EXTERNAL_PROPERTY, property = "type")
#JsonSubTypes(Array(
new Type(value = classOf[B], name = "B"),
new Type(value = classOf[C], name = "C")))
trait A{}
case class B(value : Double) extends A
case class C(value : String) extends A
case class Owner( results: Seq[(String, A)])
class ATest extends FlatSpec with ShouldMatchers {
behavior of "A"
it should "serialise" in {
val owner : Owner = Owner(Seq(("ExampleB",B(1.0)),("ExampleC",C("One"))))
val serialize: String = JsonMarshall.serialize(owner)
println(serialize)
}
it should "de-serialize_a" in {
val a: A = JsonMarshall.deserialize[A]("""{
| "type" : "C",
| "value" : "One"
| }""".stripMargin)
println(a)
}
val json = """{
| "results" : [ [ "ExampleB", {
| "type" : "B",
| "value" : 1.0
| } ], [ "ExampleC", {
| "type" : "C",
| "value" : "One"
| } ] ]
|}""".stripMargin
it should "de-serialize" in {
val owner: Owner = JsonMarshall.deserialize[Owner](json)
println(owner)
}
}
I believe there a two bugs here one related to Map the other to Seq(tuple)
1) Maps not exporting the type information on serialization
2) Seq(Tuple) ignores type information on De-Serialization
Workround: - Use a wrapper class to replace tuple.
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.EXTERNAL_PROPERTY, property = "type")
#JsonSubTypes(Array(
new Type(value = classOf[B], name = "B"),
new Type(value = classOf[C], name = "C")))
trait A{}
case class B(value : Double) extends A
case class C(value : String) extends A
case class ContainerWithMap( results: Map[String, A])
case class ContainerWithTupleSeq( results: Seq[(String, A)])
case class ContainerWithWrappedSeq( results: Seq[WrapperClass])
case class WrapperClass(s : String, a : A)
class ATest extends FlatSpec with ShouldMatchers {
behavior of "A"
val map: Map[String, A] = Map("ExampleB" -> B(1.0), "ExampleC" -> C("One"))
val seq: Seq[WrapperClass] = Seq(WrapperClass("ExampleB", B(1.0)), WrapperClass( "ExampleC",C("One")))
it should "fail not supporting reciprocal serialize de-serialize behaviour for maps" in {
val owner : ContainerWithMap = ContainerWithMap(map)
val serialize: String = JsonMarshall.serialize(owner)
println(serialize) // types not exported on serialization
val thrown = evaluating{JsonMarshall.deserialize[ContainerWithMap](serialize)} should produce [JsonMappingException]
thrown.getMessage should startWith("Unexpected token (END_OBJECT), expected FIELD_NAME: missing property 'type' that is to contain type id")
}
it should "fail not supporting reciprocal serialize de-serialize behaviour for sequence of tuples" in {
val owner : ContainerWithTupleSeq = ContainerWithTupleSeq(map.toSeq)
val serialize: String = JsonMarshall.serialize(owner)
println(serialize) // types ignored on de-Serialization
val thrown = evaluating{JsonMarshall.deserialize[ContainerWithTupleSeq](serialize)} should produce [JsonMappingException]
thrown.getMessage should startWith("Can not construct instance of com.egc.ost.pricing.contracts.response.A, problem: " +
"abstract types either need to be mapped to concrete types, have custom deserializer, or be instantiated with additional type information")
}
it should "work if using a wrapper class" in {
val owner : ContainerWithWrappedSeq = ContainerWithWrappedSeq(seq)
val serialize: String = JsonMarshall.serialize(owner)
println(serialize)
val deserialize: ContainerWithWrappedSeq = JsonMarshall.deserialize[ContainerWithWrappedSeq](serialize)
println(deserialize)
deserialize should be(owner)
}
}
A fix for this issue came in version 2.2.3. Using this version avoids the need for a wrapper class.