Jackson JSON Deserialization of MongoDB ObjectId - json

Ok, so first off here's the JSON that's returning from my web service. I'm trying to deserialize it into pojos after an asynchronous query in a ResponseHandler in my Android ContentProvider.
{"exampleList" : [{
"locationId" : "00001" ,
"owners" : [
{
"paidID" : { "$oid" : "50a9c951300493f64fbffdb6"} ,
"userID" : { "$oid" : "50a9c951300493f64fbffdb6"}
} ,
{
"paidID" : { "$oid" : "50a9c951300493f64fbffdb7"} ,
"userID" : { "$oid" : "50a9c951300493f64fbffdb7"}
}
]
}]}
At first, I was confused about the problem I was seeing, since I use the same Jackson-annotated beans for my web service as I do in my Android app--but then I realized that the owners object was never getting sent in the sample JSON to my web service (it skips the POJOs on my web service and gets added into the documents in mongoDB through atomic updates from the DAO).
So OK. Up to now, Jackson wasn't having to handle the owners object, and now that it is it is choking on it, namely:
JsonMappingException: Can not deserialize instance of java.lang.String out of
START_OBJECT token at [char position where you can find "userID" and "paidID"]
through reference chain [path to my Jackson bean which contains the owners class]
My Jackson bean has a wrapper, which is what that "exampleList" is all about:
public class Examples extends HashMap<String, ArrayList<Example>> {
}
And then the actual Example class:
#JsonIgnoreProperties(ignoreUnknown = true)
public class Example implements Comparable<Example> {
#ObjectId #Id
private String id;
#JsonProperty(Constants.Example.location)
private String location;
#JsonProperty(Constants.Example.OWNERS)
private List<Owners> owners;
public int compareTo(Example _o) {
return getId().compareTo(_o.getId());
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getLocation() {
return location;
}
public void setLocation(String location) {
this.location = location;
}
public List<Example.Owners> getOwners() {
return owners;
}
public void setOwners(List<Example.Owners> owners) {
this.owners = owners;
}
public Example() {
}
#JsonCreator
public Example(#Id #ObjectId String id) {
this.id = id;
}
#JsonIgnoreProperties(ignoreUnknown = true)
public static class Owners implements Comparable<Owners> {
#JsonProperty(Constants.Example.USERID)
private String userID;
#JsonProperty(Constants.Example.PAIDID)
private String paidID;
public Owners() {
}
public int compareTo(Owners _o) {
return getUserID().compareTo(_o.getUserID());
}
#ObjectId
public String getUserID() {
return userID;
}
#ObjectId
public void setUserID(String userID) {
this.userID = userID;
}
#ObjectId
public String getPaidID() {
return paidID;
}
#ObjectId
public void setPaidID(String paidID) {
this.paidID = paidID;
}
}
}
And finally, the code in the ResponseHandler where this is all failing (the 2nd line produces the JsonMappingException):
objectMapper = MongoJacksonMapperModule.configure(objectMapper);
mExamples = objectMapper.readValue(jsonParser, Examples.class);
I have a feeling the issue is that Jackson still doesn't know how to map those $oid, which are the mongoDB ObjectIds. The MongoJacksonMapper library is supposed to help that by providing the #ObjectId annotation and a way to configure the ObjectMapper to use that library, but it still isn't working. For some reason, it's still looking for the userID or paidID as a String, not an ObjectId. Any ideas?

Another alternative is
com.fasterxml.jackson.databind.ser.std.ToStringSerializer.
#Id
#JsonSerialize(using = ToStringSerializer.class)
private final ObjectId id;
This will result in:
{
"id": "5489f420c8306b6ac8d33897"
}

For future users: Use a custom jackson deserializer to convert $oid back to ObjectId.
public class ObjectIdDeserializer extends JsonDeserializer<ObjectId> {
#Override
public ObjectId deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
JsonNode oid = ((JsonNode)p.readValueAsTree()).get("$oid");
return new ObjectId(oid.asText());
}
}
How to use:
ObjectMapper mapper = new ObjectMapper();
SimpleModule mod = new SimpleModule("ObjectId", new Version(1, 0, 0, null, null, null));
mod.addDeserializer(ObjectId.class, new ObjectIdDeserializer());
mapper.registerModule(mod);
YourClass obj = mapper.readValue("{your json with $oid}", YourClass.class);

My code had at least two problems that were pretty tough to track down answers to online, so I'll make sure to link here. Basically, child classes need a constructor in the parent class that calls Jackson's readValue() to map the child. As far as mongoDB $oid's go, you should create a separate MongoId class to represent these mongo objects, and follow a similar pattern as with the child class to map the data when it comes in for deserialization. Here's a blog post I found that describes this well and provides some examples.

Jackson does not know how to serialize an ObjectId. I tweaked Arny's code to serialize any ObjectId and provide this working example:
public class SerialiserTest {
private ObjectMapper mapper = new ObjectMapper();
public static class T {
private ObjectId objectId;
public ObjectId getObjectId() {
return objectId;
}
public void setObjectId(ObjectId objectId) {
this.objectId = objectId;
}
}
#Test
public final void serDeser() throws IOException {
T t = new T();
t.setObjectId(new ObjectId());
List<T> ls = Collections.singletonList(t);
String json = mapper.writeValueAsString(ls);
System.out.println(json);
SimpleModule mod = new SimpleModule("ObjectId", new Version(1, 0, 0, null, null, null));
mod.addDeserializer(ObjectId.class, new ObjectIdDeserializer());
mapper.registerModule(mod);
JavaType type = mapper.getTypeFactory().
constructCollectionType(List.class, T.class);
List<?> l = mapper.readValue(json, type);
System.out.println(l);
}
}
public class ObjectIdDeserializer extends JsonDeserializer<ObjectId> {
#Override
public ObjectId deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
JsonNode n = (JsonNode)p.readValueAsTree();
return new ObjectId(n.get("timestamp").asInt(), n.get("machineIdentifier").asInt(), (short) n.get("processIdentifier").asInt(), n.get("counter").asInt());
}
}

There's an even easier way documented here which was a lifesaver for me. Now you can use the ObjectId in Java but when you go to/from JSON it'll be a String.
public class ObjectIdJsonSerializer extends JsonSerializer<ObjectId> {
#Override
public void serialize(ObjectId o, JsonGenerator j, SerializerProvider s) throws IOException, JsonProcessingException {
if(o == null) {
j.writeNull();
} else {
j.writeString(o.toString());
}
}
}
And then in your beans:
#JsonSerialize(using=ObjectIdJsonSerializer.class)
private ObjectId id;

I did it like this:
#Configuration
public class SpringWebFluxConfig {
#Bean
#Primary
ObjectMapper objectMapper() {
Jackson2ObjectMapperBuilder builder = new Jackson2ObjectMapperBuilder();
builder.serializerByType(ObjectId.class, new ToStringSerializer());
builder.deserializerByType(ObjectId.class, new JsonDeserializer() {
#Override
public Object deserialize(JsonParser p, DeserializationContext ctxt)
throws IOException {
Map oid = p.readValueAs(Map.class);
return new ObjectId(
(Integer) oid.get("timestamp"),
(Integer) oid.get("machineIdentifier"),
((Integer) oid.get("processIdentifier")).shortValue(),
(Integer) oid.get("counter"));
}
});
return builder.build();
}
}

Related

Is it possible to make Jackson serialize a nested object as a string

Given these classes:
#Value
private static class Message {
private final String type;
private final MyType message;
}
#Value
public class MyType {
private final String foo;
}
Jackson will produce:
{
"Type" : "Test",
"Message" : {"foo" : "bar"}
}
Is there some type of annotation or instruction I can give to Jackson to ask it to serialize the nested complex type as a string, e.g. the desired JSON would be:
{
"Type" : "Test",
"Message" : "{\"foo\" : \"bar\"}"
}
I tried both of these annotations on the message field:
#JsonFormat(shape = JsonFormat.Shape.STRING)
#JsonSerialize(as=String.class)
Neither has the desired impact. For now my "hack" is to do this at construction time:
return new Message("Test", mapper.writeValueAsString(new MyType("bar")));
I guess I could write a custom serializer, but I wondered if this is some type of standard behaviour that is built in. My use case is that I'm constructing a JSON payload which is expected to have a string message contained within it that itself contains JSON.
Environment
Jackson version is 2.9.0 using Spring Boot 2 on Java 10.
It can be done with custom serializer:
class EscapedJsonSerializer extends StdSerializer<Object> {
public EscapedJsonSerializer() {
super((Class<Object>) null);
}
#Override
public void serialize(Object value, JsonGenerator gen, SerializerProvider provider) throws IOException {
StringWriter str = new StringWriter();
JsonGenerator tempGen = new JsonFactory().setCodec(gen.getCodec()).createGenerator(str);
if (value instanceof Collection || value.getClass().isArray()) {
tempGen.writeStartArray();
if (value instanceof Collection) {
for (Object it : (Collection) value) {
writeTree(gen, it, tempGen);
}
} else if (value.getClass().isArray()) {
for (Object it : (Object[]) value) {
writeTree(gen, it, tempGen);
}
}
tempGen.writeEndArray();
} else {
provider.defaultSerializeValue(value, tempGen);
}
tempGen.flush();
gen.writeString(str.toString());
}
#Override
public void serializeWithType(Object value, JsonGenerator gen, SerializerProvider serializers, TypeSerializer typeSer) throws IOException {
StringWriter str = new StringWriter();
JsonGenerator tempGen = new JsonFactory().setCodec(gen.getCodec()).createGenerator(str);
writeTree(gen, value, tempGen);
tempGen.flush();
gen.writeString(str.toString());
}
private void writeTree(JsonGenerator gen, Object it, JsonGenerator tempGen) throws IOException {
ObjectNode tree = ((ObjectMapper) gen.getCodec()).valueToTree(it);
tree.set("#class", new TextNode(it.getClass().getName()));
tempGen.writeTree(tree);
}
}
and deserializer:
class EscapedJsonDeserializer extends JsonDeserializer<Object> implements ContextualDeserializer {
private final Map<JavaType, JsonDeserializer<Object>> cachedDeserializers = new HashMap<>();
#Override
public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
throw new IllegalArgumentException("EscapedJsonDeserializer should delegate deserialization for concrete class");
}
#Override
public JsonDeserializer<?> createContextual(DeserializationContext ctxt, BeanProperty property) throws JsonMappingException {
JavaType type = (ctxt.getContextualType() != null) ?
ctxt.getContextualType() : property.getMember().getType();
return cachedDeserializers.computeIfAbsent(type, (a) -> new InnerDeserializer(type));
}
private class InnerDeserializer extends JsonDeserializer<Object> {
private final JavaType javaType;
private InnerDeserializer(JavaType javaType) {
this.javaType = javaType;
}
#Override
public Object deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
String string = p.readValueAs(String.class);
return ((ObjectMapper) p.getCodec()).readValue(string, javaType);
}
#Override
public Object deserializeWithType(JsonParser p, DeserializationContext ctxt, TypeDeserializer typeDeserializer)
throws IOException {
String str = p.readValueAs(String.class);
TreeNode root = ((ObjectMapper) p.getCodec()).readTree(str);
Class clz;
try {
clz = Class.forName(((TextNode) root.get("#class")).asText());
Object newJsonNode = p.getCodec().treeToValue(root, clz);
return newJsonNode;
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
}
}
}
}
The field should be annotated with #JsonSerialize and #JsonDeserialize (if needed)
class Outer {
#JsonTypeInfo(include = JsonTypeInfo.As.PROPERTY, use = JsonTypeInfo.Id.CLASS)
#JsonSerialize(using = EscapedJsonSerializer.class)
#JsonDeserialize(using = EscapedJsonDeserializer.class)
public Foo val;
}
It works well with simple collections (list, arrays) and to some extent with polymorphism, although more elaborate solution may be needed for specific polymorphism related issues.
Example output looks like this:
{"val":"{\"foo\":\"foo\",\"#class\":\"org.test.Foo\"}"}
{"val":"{\"foo\":\"foo\",\"bar\":\"bar\",\"#class\":\"org.test.Bar\"}"}
I also couldn't find built-in solution and ended up writing custom converter:
public class ObjectToJsonStringConverter extends StdConverter<Object, String> {
private final ObjectMapper objectMapper = new ObjectMapper();
#Override
public String convert(Object value) {
try {
return objectMapper.writeValueAsString(value);
} catch (JsonProcessingException e) {
throw new IllegalStateException(e);
}
}
}
usage:
#Value
private static class Message {
private final String type;
#JsonSerialize(converter = ObjectToJsonStringConverter.class)
private final MyType message;
}

#JsonAnySetter, #JsonAnyGetter with DSL Json (De)/Serialization not taking values(always null)

Im using #JsonAnySetter and #JsonAnyGetter in my POJO class using my Custom serialization with DSL JSON class, the Map is initialized but the other properties are always null.
My POJO class:
#CompiledJson
public class Name {
String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
Map<String,String> properties = new HashMap<String,String>();
public Name() {
// TODO Auto-generated constructor stub
}
#JsonAnyGetter
public Map<String, String> get() {
return this.properties;
}
#JsonAnySetter
public void set(String key, String value) {
this.properties.put(key, value);
}
De/Serializing using DSLJson serialize() and deserialize() methods. I do not see any error also, but properties remains null in JSON. I doubt if Jackson annotations are supported by DSL Json. :/
Spring Boot App with DSL Json and Jackson Annotations
UPDATE
I want to parse MyClass, which is a part of RootClass:
#Compiledjson
public class RootClass {
private String id;
private List<MyClass> myclass;
private AnotherCLass class2;
//getters and setter here
}
#CompiledJson
public class MyClass implements JsonObject {
private String name;
private Map<String, String> properties; //want this to behave like Jackson's #JsonAnySetter/Getter annotation.
//The implementation of MapConverter serializer you mentioned below.
}
The entire code parses through custom Message reader and writer.
While sending my JSON Body, It'll be like this :
{
"id" : "1234",
"myclass" :
[
{
"name" : "abcd",
//any dynamic properties I want to add will go here
"test" : "test1",
"anything" : "anything"
}
],
"class2" : "test5"
}
Thank you :)
DSL-JSON doesn't support such get()/set(string, string) method pairs.
It does understand Map<String, String> so if you expose properties it will work on that. But not in this kind of setup.
As of v1.1 you have two options for solving such problems, both of them are covered in example project
If you wish to reuse existing converters, your solution can look like this:
public static class MyClass {
private String name;
private Map<String, String> properties;
#JsonConverter(target = MyClass.class)
public static class MyClassConverter {
public static final JsonReader.ReadObject<MyClass> JSON_READER = new JsonReader.ReadObject<MyClass>() {
public MyClass read(JsonReader reader) throws IOException {
Map<String, String> properties = MapConverter.deserialize(reader);
MyClass result = new MyClass();
result.name = properties.get("name");
result.properties = properties;
return result;
}
};
public static final JsonWriter.WriteObject<MyClass> JSON_WRITER = new JsonWriter.WriteObject<MyClass>() {
public void write(JsonWriter writer, MyClass value) {
MapConverter.serialize(value.properties, writer);
}
};
}
}

Simple way to strip outer array of responses in gson

I'm working with an api (Phillips Hue) that wraps all of it's json responses in an array with one entry (the content).
Example:
[{
"error": {
"type": 5,
"address": "/",
"description": "invalid/missing parameters in body"
}
}]
I usually write standard POJO's parsed by GSON to handle responses but since the response is not a json object I'm a bit stumped on the best way to deal with this. I didn't really want every object to actually be an array that I have to call .get(0) on.
Example of the POJO if it was a JSON obj and NOT wrapped in an array.
public class DeviceUserResponse {
private DeviceUser success;
private Error error;
public DeviceUser getSuccess() {
return success;
}
public Error getError() {
return error;
}
public static class Error {
private int type;
private String address;
private String description;
public int getType() {
return type;
}
public String getAddress() {
return address;
}
public String getDescription() {
return description;
}
#Override
public String toString() {
return "Type: " + this.type
+ " Address: " + this.address
+ " Description: " + this.description;
}
}
}
What I have to do right now:
ArrayList<DeviceUserResponse> response.get(0).getError();
Is there a way that I can strip this array for every response or am I just going to have to do a .get(0) in my POJO's and just not expose it?
I think you've to go with custom deserialization in order to "strip out" the array.
Here a possible solution.
An adapter for your response POJO:
public class DeviceUserResponseAdapter extends TypeAdapter<DeviceUserResponse> {
protected TypeAdapter<DeviceUserResponse> defaultAdapter;
public DeviceUserResponseAdapter(TypeAdapter<DeviceUserResponse> defaultAdapter) {
this.defaultAdapter = defaultAdapter;
}
#Override
public void write(JsonWriter out, DeviceUserResponse value) throws IOException {
defaultAdapter.write(out, value);
}
#Override
public DeviceUserResponse read(JsonReader in) throws IOException {
in.beginArray();
assert(in.hasNext());
DeviceUserResponse response = defaultAdapter.read(in);
in.endArray();
return response;
}
}
A factory for your adapter:
public class DeviceUserResponseAdapterFactory implements TypeAdapterFactory {
#Override
#SuppressWarnings("unchecked")
public <T> TypeAdapter<T> create(Gson gson, TypeToken<T> type) {
if (type.getRawType()!=DeviceUserResponse.class) return null;
TypeAdapter<DeviceUserResponse> defaultAdapter = (TypeAdapter<DeviceUserResponse>) gson.getDelegateAdapter(this, type);
return (TypeAdapter<T>) new DeviceUserResponseAdapter(defaultAdapter);
}
}
Then you've to register and user it:
DeviceUserResponseAdapterFactory adapterFactory = new DeviceUserResponseAdapterFactory();
GsonBuilder gsonBuilder = new GsonBuilder();
Gson gson = gsonBuilder.registerTypeAdapterFactory(adapterFactory).create();
DeviceUserResponse response = gson.fromJson(json, DeviceUserResponse.class);
System.out.println(response.getError());
This solution will not work if you have the DeviceUserResponse inside other complex JSON object. I that case the adapter will try to find an array and will terminate with an error.
Another solution is to parse it as array and then in your "communication" layer you get only the first element. This will preserve the GSon deserialization.
In the comment you're asking for a more generic solution, here one:
The adapter:
public class ResponseAdapter<T> extends TypeAdapter<T> {
protected TypeAdapter<T> defaultAdapter;
public ResponseAdapter(TypeAdapter<T> defaultAdapter) {
this.defaultAdapter = defaultAdapter;
}
#Override
public void write(JsonWriter out, T value) throws IOException {
defaultAdapter.write(out, value);
}
#Override
public T read(JsonReader in) throws IOException {
in.beginArray();
assert(in.hasNext());
T response = defaultAdapter.read(in);
in.endArray();
return response;
}
}
The factory:
public class ResponseAdapterFactory implements TypeAdapterFactory {
#Override
public <T> TypeAdapter<T> create(Gson gson, TypeToken<T> type) {
if ((type.getRawType().getSuperclass() != Response.class)) return null;
TypeAdapter<T> defaultAdapter = (TypeAdapter<T>) gson.getDelegateAdapter(this, type);
return (TypeAdapter<T>) new ResponseAdapter<T>(defaultAdapter);
}
}
Where Response.class is your super class from which all the service responses inherit.
The first solution advices are still valid.

Jackson Mapper Serialize/Deserialize ObjectId

My POJO is :
import org.jongo.marshall.jackson.id.Id;
public class User {
#Id
private String id;
private String name;
private int age;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
}
I get user from mongo database and want to output him into a file with jackson mapper
ObjectMapper mapper = new ObjectMapper();
mapper.writerWithDefaultPrettyPrinter().writeValue(new File("c:/user.txt"), user);
and I get something like this in my file
{
"name" : "John",
"age" : 23,
"_id" : {
"time" : 1358443593000,
"inc" : 660831772,
"machine" : 2028353122,
"new" : false,
"timeSecond" : 1358443593
}
}
I need id field to be stored into a file as string because when i deserialize this object my id field in pojo looks something like this
{
"time":1358443593000,
"inc":660831772,
"machine":2028353122,
"new":false,
"timeSecond":1358443593
}
Any help will be apreciated
Answering my own question. Found solution here Spring 3.2 and Jackson 2: add custom object mapper
I needed custom object mapper and ObjectId serializer.
public class ObjectIdSerializer extends JsonSerializer<ObjectId> {
#Override
public void serialize(ObjectId value, JsonGenerator jgen, SerializerProvider provider) throws IOException, JsonProcessingException {
jgen.writeString(value.toString());
}
}
public class CustomObjectMapper extends ObjectMapper {
public CustomObjectMapper() {
SimpleModule module = new SimpleModule("ObjectIdmodule");
module.addSerializer(ObjectId.class, new ObjectIdSerializer());
this.registerModule(module);
}
}
I found an easy attempt using springboot 2.5.4.
Just by adding a Jackson2ObjectMapperBuilderCustomizer bean will do the trick.
#Configuration
public class JacksonMapperConfiguration
{
#Bean
public Jackson2ObjectMapperBuilderCustomizer jsonCustomizer() {
return builder -> builder.serializerByType(ObjectId.class, new ToStringSerializer());
}
}
Here is a simple solution for serialization if you don't have a model for the object being stored:
client.getDatabase("db").getCollection("collection").find().onEach { it["_id"] = it["_id"].toString() }
"onEach" is a kotlin function. If you use Java, then change it to a simple foreach.
It's not efficient to iterate over the entire list just for the id. Only use it for small lists where performance is less important than short code.

Jackson in Spring: how to unmarshal a "generic" class w/o it thinking it's a LinkedHashMap?

So my entities look like this:
public class HappyClass<T>
{
private String id;
prviate int ver;
private Object obj;
public String getId()
{
return this.id;
}
public void setId( String id )
{
this.id = id;
}
public int getVer()
{
return this.ver;
}
public void setVer( int ver )
{
this.ver = ver;
}
#JsonTypeInfo( use = Id.NONE )
public T getObj()
{
return obj;
}
public void setObj( T obj )
{
this.obj = obj;
}
}
public class HappyGeneric
{
private String someStuff();
public String getSomeStuff()
{
return this.someStuff();
}
public void setSomeStuff( String someStuff )
{
this.someStuff = someStuff;
}
}
If I instantiate a class like this:
HappyClass<HappyGeneric> hc = new HappyClass<HappyGeneric>();
If I send it to Spring in a #ResponseBody it returns this:
{
"id" : "iamsomeid",
"ver" : 123,
"obj" : {
"someStuff" : "iamsomestuff"
}
}
However, when Spring and/or Jackson attempts to unmarshal the same JSON, it figures out that the main class is a HappyClass, however, the getObj() it unmarshals to a LinkedHashMap and not a HappyGeneric no matter what I seem to annotate it with.
Anybody have any ideas how I can force Jackson to unmarshal that generic to the original class?
Thanks!
EDIT: I'm aware I can call mapper.convertValue( blah.getObj(), HappyGeneric.class ) and get the object out that way-- I was hoping to get Spring to figure it out automatically (through annotations, for example).