Gson - deserialize json with nested map - json

I am trying to deserialize this type of json
{
"_embedded": {
"list": [
{
"000000": {
"date": "2015-07-10 14:29:15"
}
},
{
"111111": {
"date": "2015-07-11 14:29:15"
}
}
]
}
}
I manage to get a list inside my embedded object but the list entries are empty.
My Embedded class looks like this
public class Embedded {
#SerializedName("list")
private List<ListEntry> entries;
}
But I am not able to deserialize the list's entries. I think the problem is the fact that the map is nested inside an object that doesn't have a name.
public class ListEntry {
private Map<String, ListEntryInfo> map;
}

The initial problem is the way you declared your hierarchy. A ListEntry is a Map<String, ListEntryInfo> but does not have a Map<String, ListEntryInfo>. To make it work you have three options:
declare the ListEntry class as class ListEntry extends HashMap<String, ListEntryInfo> {}, which is a bad idea in my opinion
get rid of the ListEntry class and declare the entries list like this #SerializedName("list") List<Map<String, ListEntryInfo>> entries;
use the approach I initially described below, by implementing a custom deserializer
As said before what you could do is to write a custom deserializer, so that you have a more typed list of entries.
As a ListEntry instance has only one ListEntryInfo value mapped to a key, I would change the structure to this:
class ListEntry {
private String key;
private ListEntryInfo value;
public ListEntry(String key, ListEntryInfo value) {
this.key = key;
this.value = value;
}
public String toString() {
return key + " -> " + value;
}
}
class ListEntryInfo {
//assuming we store the date as a String for simplicity
#SerializedName("date")
private String date;
public ListEntryInfo(String date) {
this.date = date;
}
public String toString() {
return date;
}
}
Now you need to write a deserializer to create a new ListEntry instance when you deserialize the JSON:
class ListEntryDeserializer implements JsonDeserializer<ListEntry> {
#Override
public ListEntry deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
Iterator<Map.Entry<String, JsonElement>> ite = json.getAsJsonObject().entrySet().iterator();
//you may want to throw a custom exception or return an "empty" instance there
Map.Entry<String, JsonElement> entry = ite.next();
return new ListEntry(entry.getKey(), context.deserialize(entry.getValue(), ListEntryInfo.class));
}
}
This deserializer will read each ListEntry. As it's composed of a single key-value pair (in the first case the String "000000" is mapped to one ListEntryInfo and so on), we fetch the key and deserialize the corresponding ListEntryInfo with JsonDeserializationContext instance.
The final step, is to register it within the parser:
Gson gson = new GsonBuilder().registerTypeAdapter(ListEntry.class, new ListEntryDeserializer()).create();
Running it on your example, you should get:
[000000 -> 2015-07-10 14:29:15, 111111 -> 2015-07-11 14:29:15]

Related

Gson throws exception when expected string array returns object

I'm using an api where a particular field (below) normally contains a string array. However, if the array is empty, the api returns an empty object in what is normally the array of string. Here is the field that's causing problems.
Normal.
"a": [
"str"
]
Empty.
"a": [
{}
]
The second case causes Gson to crash with a JsonSyntaxException. How do I handle this?
I don't know if this is the best way, but it works.
The faulty field can be annotated with #JsonAdapter(MyTypeAdapter.class). The TypeAdapter can then use its read method and check using peek() weather or not the next value is of the expected type.
Let us assume you have a class representing the API response, like:
public class Response {
private String[] a;
private String b;
private String c;
}
One way to get the Response object parsed whether JSON for a is valid or not is to create a JsonDeserializer that checks if a can parsed and excludes parsing of a if it fails, so leaves a to null.
public class SkipBadSyntaxDeserializer implements JsonDeserializer<Response> {
// This strategy is used if parse of field a fails
private final ExclusionStrategy excludeA = new ExclusionStrategy() {
#Override
public boolean shouldSkipField(FieldAttributes f) {
return "a".equals(f.getName());
}
// no need to care of this used only here for the Response class
#Override
public boolean shouldSkipClass(Class<?> clazz) {
return false;
}
};
// one parser for good and another for bad format
private final Gson gson = new Gson(),
gsonBadFormat = new GsonBuilder()
.addDeserializationExclusionStrategy(excludeA).create();;
#Override
public Response deserialize(JsonElement json, Type typeOfT,
JsonDeserializationContext context)
throws JsonParseException {
try {
return gson.fromJson(json, Response.class);
} catch (JsonSyntaxException e) {
// parse a failed try again without it
return gsonBadFormat.fromJson(json, Response.class);
}
}
}
Try it with:
new GsonBuilder().registerTypeAdapter(Response.class,
new SkipBadSyntaxDeserializer())
.create()
.fromJson(JSON, Response.class);
If JSON would be like:
{
"a": [{}],
"b": "bval",
"c": "cval"
}
then properties for Response would be:
a=null
b="bval"
c="cval"
Update
Based on your own answer: if it is possible to alter DTO for response then using annotation #JsonAdapter will let you to handle this per field. Deserializer will then be simply:
public class SkipExceptionAdapter implements JsonDeserializer<String[]> {
#Override
public String[] deserialize(JsonElement json, Type typeOfT,
JsonDeserializationContext context)
throws JsonParseException {
try {
return context.deserialize(json, String[].class);
} catch (JsonSyntaxException e) {
return new String[] {}; // or null how you wish
}
}
}
and annotation in Response.a
#JsonAdapter(SkipExceptionAdapter.class)
private String[] a;
will handle it for that field only.

Gson: Is there more efficient ways to do custom (de)serialization of a verbose Json object

I am working with some json objects that I call verbose:
{
"user": {
"name": "username",
"email": "blah#blah.com",
"time_zone": "America/New_York"
}
}
But I'd prefer to just deal with them in terms of java POJOs like:
class UserDetails {
String name;
String email;
String timeZone;
...
}
Note that I have no control over the POJO as it is generated code.
My two requirements for (de)serialization is that
the timeZone field maps to time_zone in JSON
the outer user is ignored
So I have some customer (de)serializers:
class UserDeserializer implements JsonDeserializer<UserDetails> {
#Override
public UserDetails deserialize(JsonElement je, Type type, JsonDeserializationContext jdc)
throws JsonParseException {
JsonElement content = je.getAsJsonObject().get("user");
UserDetails userDetails = new GsonBuilder()
.setFieldNamingStrategy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES)
.create()
.fromJson(content, UserDetails.class);
return userDetails;
}
}
class UserSerializer implements JsonSerializer<UserDetails> {
#Override
public JsonElement serialize(UserDetails userDetails, Type typeOfSrc,
JsonSerializationContext context) {
JsonObject obj = new JsonObject();
JsonElement je = new GsonBuilder()
.setFieldNamingStrategy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES)
.create().toJsonTree(userDetails);
obj.add("user", je);
return obj;
}
}
I feel like creating new Gson objects in the (de)serializer logic is not ideal/efficient just to add and remove the outermost user key.
EDIT: Actually .setFieldNamingStrategy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES) does work fine on deserialization.
I don't really think it's a good idea in general, and you should probably better have a single Wrapper<T> for all "top-most" purposes (if you don't want your inner objects to be considered "verbose").
But you're right when you say
I feel like creating new Gson objects in the (de)serializer logic is not ideal/efficient just to add and remove the outermost user key.
So:
Creating a Gson is a relatively expensive operation.
This just creates unnecessary objects and hits the heap.
Gson may be configured in a special way and you might want to share the same Gson configuration everywhere.
JsonSerializer and JsonDeserializer operate on JSON trees (JsonElement and its subclasses), therefore it creates an intermediate in-memory tree representations before/after serialization/deserialization.
You might consider a faster solution, that's free of those items.
final class VerboseTypeAdapterFactory
implements TypeAdapterFactory {
private final Map<Class<?>, String> mappings;
private VerboseTypeAdapterFactory(final Map<Class<?>, String> mappings) {
this.mappings = mappings;
}
static TypeAdapterFactory get(final Map<Class<?>, String> mappings) {
// Create a defensive copy to make sure the map is not modified from outside
final Map<Class<?>, String> mappingsCopy = mappings
.entrySet()
.stream()
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
return new VerboseTypeAdapterFactory(mappingsCopy);
}
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
final Class<? super T> rawType = typeToken.getRawType();
// Not something we can handle?
if ( !mappings.containsKey(rawType) ) {
// Then let Gson do its job elsewhere
return null;
}
// Getting a property name we want to use for a particular class
final String propertyName = mappings.get(rawType);
// And getting the original type adapter for this class (effectively ReflectiveTypeAdapterFactory.Adapter)
final TypeAdapter<T> delegateTypeAdapter = gson.getDelegateAdapter(this, typeToken);
return VerboseTypeAdapter.get(propertyName, delegateTypeAdapter);
}
private static final class VerboseTypeAdapter<T>
extends TypeAdapter<T> {
private final String propertyName;
private final TypeAdapter<T> delegateTypeAdapter;
private VerboseTypeAdapter(final String propertyName, final TypeAdapter<T> delegateTypeAdapter) {
this.propertyName = propertyName;
this.delegateTypeAdapter = delegateTypeAdapter;
}
private static <T> TypeAdapter<T> get(final String propertyName, final TypeAdapter<T> delegateTypeAdapter) {
return new VerboseTypeAdapter<>(propertyName, delegateTypeAdapter)
// A convenient method to simplify null-handling
.nullSafe();
}
#Override
#SuppressWarnings("resource")
public void write(final JsonWriter out, final T object)
throws IOException {
// Open the object with `{`
out.beginObject();
// Prepend the object with its reserved name
out.name(propertyName);
// Write the object
delegateTypeAdapter.write(out, object);
// And close the object with `}`
out.endObject();
}
#Override
public T read(final JsonReader in)
throws IOException {
// Assume the very first token is `{`
in.beginObject();
// Peeking what's the actual property name
final String actualPropertyName = in.nextName();
// And if it's not we expect, throw a JSON parse exception
if ( !actualPropertyName.equals(propertyName) ) {
throw new JsonParseException("Expected " + propertyName + " but was " + actualPropertyName);
}
// Otherwise read the value led by the property name
final T object = delegateTypeAdapter.read(in);
// And make sure there are no more properties
if ( in.hasNext() ) {
throw new JsonParseException(propertyName + " is expected to be the only top-most property");
}
// Assume the very last token is `}` (this works for the check above, but we made it more semantical)
in.endObject();
return object;
}
}
}
So, for example, the following code
private static final Gson gson = new GsonBuilder()
.setFieldNamingStrategy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES)
.registerTypeAdapterFactory(VerboseTypeAdapterFactory.get(ImmutableMap.of(UserDetails.class, "user")))
.create();
...
final UserDetails userDetails = gson.fromJson(jsonReader, UserDetails.class);
System.out.println(userDetails.name);
System.out.println(userDetails.email);
System.out.println(userDetails.timeZone);
final String json = gson.toJson(userDetails);
System.out.println(json);
produces
username
blah#blah.com
America/New_York
{"user":{"name":"username","email":"blah#blah.com","time_zone":"America/New_York"}}
As the conclusion:
No more excessive Gson instantiation.
Original Gson instance configuration inherited (i.e. FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES set once).
No intermediate JsonElement instances.

How to deserialize json object with an internal object to a single java object by using google gson library?

I have the following json file:
{
"sqldb": [
{
"name": "mydb",
"label": "sqldb",
"plan": "sqldb_free",
"credentials": {
"port": 50000,
"db": "SQLDB",
"username": "xxxxxxx",
"host": "75.126.155.92",
"hostname": "75.126.155.92",
"jdbcurl": "jdbc:db2://75.126.155.92:50000/SQLDB",
"uri": "db2://xxxxxxx:xxxxxxx#75.126.155.92:50000/SQLDB",
"password": "xxxxxxx"
}
}
]
}
The object has following structure: list of sqldb objects, which has internal credentials object. How can I parse it by Google Gson library into single sqldb object? I mean is it possible to create by using gson's annotations, Java Object like following:
public class VcapObject{
private String name;
private String label;
private String plan;
private String port;
private String db;
private String username;
private String host;
private String hostname;
private String jdbcurl;
private String uri;
private String password;
}
which will be filled by:
VcapObject vcapObject = gson.fromJson(vcapString, VcapObject.class);
for example?
To get rid of the internal credentials object, you need a custom JsonDeserializer that copies the fields from the inner class to the outer class. There are a few different strategies you could take, but here is one that modifies the JSON tree and feeds it back to Gson.
public class VcapDeserializer implements JsonDeserializer<VcapObject> {
private static final String MERGE_FIELD = "credentials";
final private Gson gson;
public VcapDeserializer() {
gson = new Gson();
}
#Override
public VcapObject deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
if(json.isJsonObject()) {
JsonObject jsonObject = json.getAsJsonObject();
if(jsonObject.has(MERGE_FIELD)) {
// We have the object, get all the fields
JsonObject mergeObject = jsonObject.get(MERGE_FIELD).getAsJsonObject();
Set<Map.Entry<String, JsonElement>> entries = mergeObject.entrySet();
for(Map.Entry<String, JsonElement> entry : entries) {
// Copy property to top level object
jsonObject.add(entry.getKey(), entry.getValue());
}
// now that we have copied the fields, remove the nested object
jsonObject.remove(MERGE_FIELD);
}
// Deserialize the top-level object
return gson.fromJson(jsonObject, VcapObject.class);
} else {
// top level is supposed to be an object
throw new IllegalStateException();
}
}
}
Then create a custom Gson instance with GsonBuilder --
Gson gson = new GsonBuilder().registerTypeAdapter(VcapObject.class, new VcapDeserializer()).create();
Note -- from your question it is unclear if you want to get rid of the top-level wrapper object. I assumed not because you did not say the array was always going to return one element. So you will still need that --
public class SqlDbWrapper {
List<VcapObject> sqldb;
}
and then deserialize with --
SqlDbWrapper sqlDbWrapper = gson.fromJson(jsonString, SqlDbWrapper.class);
You can make a wrapper class containing a Collection of VcapObject, for example VcapList. Then you can pass VcapList as class to fromJson. Add a method to VcapList to get the first if not empty.

How to use Jackson Annotations to process JSON with random object names

I have the following JSON:
{
"animals": {
"113110": {
"id": 113110,
"name": "Dog",
.....
},
"121853": {
"id": 121853,
"name": "Cat",
.....
}
}
}
Ideally, the JSON should be as follows and implementing Jackson annotations will be trivial:
{
"animals": [
{
"id": 113110,
"name": "Dog",
.....
},
{
"id": 121853,
"name": "Cat",
.....
}
]
}
However, is there a way to use Jackson to abstract the object names so I can work with the original JSON, if anybody gets my meaning?
EDIT:
I do not know how to create my POJO. I could create an Animal class, with objects 113110 and 121853, but as these objects will always vary, how do I use Jackson annotations in my Animal class so that I can deserialize the JSON?
Thanks all, but I couldn't really understand the rest of the answers ( I don't really want to delve into Jackson, I just want to convert it to a POJO), so I found an alternative solution.
I left out a key bit of information: The JSON I posted is part of a much larger JSON object.
I ended up using Jackson's #AnySetter as I noticed that any "un-parsable" JSON data related to "animals" could be retrieved in additionalProperties defined as follows in its parent class:
public class AnimalParent {
#JsonIgnore
private Animal animal;
#JsonIgnore
private Map<String, Object> additionalProperties =
new HashMap<String, Object>();
public Animal getAnimal() {
return this.animal;
}
public void setAnimal(Animal animal) {
this.animal = animal;
}
#JsonAnyGetter
public Map<String, Object> getAdditionalProperties() {
return this.additionalProperties;
}
#JsonAnySetter
public void setAdditionalProperty(String name, Object value) {
this.additionalProperties.put(name, value);
}
}
Then in my main method where I parse the parent JSON, I have the following after the parsing is completed to parse the animals.
// start parsing parent JSON
...
// end parsing parent JSON
// parse animal
ObjectMapper mapper = new ObjectMapper();
if (animalParent.getAdditionalProperties() != null) {
for (Map.Entry<String, Object> item : animalParent
.getAdditionalProperties().entrySet()) {
Animal animal = mapper.convertValue(item.getValue(), Animal.class);
animalParent.setAnimal(animal);
}
It is always a bit messy when your JSON is "dynamic" as per the OPs example. The main approaches are
parse the JSON to some kind of dynamic Map-structure
parse the JSON to a tree-structure (i.e. JsonNode)
use a custom deserializer to parse the JSON and map it to a POJO
There are downsides to all of these approaches. The Map-approach offers no type safety and does not offer much functionality when it comes to traversing the object structure.
The JsonNode approach offers some nice type-methods and also some traversal methods. IMO this is a cleaner approach than the Map-approach.
The POJO-approach is type safe but a custom deserializer is required which is generally not pretty...
So, maybe the following "hybrid" approach can be of use.
// Setup the mapper
final ObjectMapper mapper = new ObjectMapper();
// Parse the json to a tree (JsonNode). This is IMO nicer than the
// Map since it exposes some nice methods for managing the
// underlying data
final JsonNode json = mapper.readTree(jsonString);
// Alt 1, use JsonNode directly
for (final JsonNode animal : json.path("animals")) {
final int id = animal.get("id").asInt();
final String name = animal.get("name").asText();
// Do stuff with name and id...
}
If the JsonNode approach feels a bit too raw then it is possible to convert the JsonNode object to a POJO without the use of a deserializer. If you assume the following POJO:
public class Animal {
private final int id;
private final String name;
#JsonCreator
public Animal(#JsonProperty("id") final int id, #JsonProperty("name") final String name) {
this.id = id;
this.name = name;
}
public int getId() {
return id;
}
public String getName() {
return name;
}
}
Then, this code can be used for converting to POJOs:
final ObjectMapper mapper = new ObjectMapper();
final JsonNode json = mapper.readTree(jsonString);
// Alt 2, convert to a Pojo
for (final JsonNode animal : json.path("animals")) {
final Animal a = mapper.treeToValue(animal, Animal.class);
// Handle the animal instance...
}
Finally, if the POJO still contains dynamic data you can use the following approach to handle that. In your POJO, declare the following:
private final Map<String, Object> dynamic = new HashMap<>();
#JsonAnySetter
private void set(String name, Object value) {
dynamic.put(name, value);
}
Note that it is not a must for the method to be public (i.e. it can be hidden from the outside world). This way you'll get hold of all the unknown/dynamic JSON elements.
Personally, any time I'm dealing with weird JSON that doesn't map easily to POJOs, I just do custom serialization.
I would probably make the POJOs look something like this:
public class Animal
{
String id;
String name;
}
public class JsonThing
{
List<Animal> animals;
}
Then I would implement a custom parser using the Jackson stream API. Here's a quick stub of a JsonDeserializer<JsonThing>:
public Stuff deserialize(JsonParser jp, DeserializationContext ctxt)
throws IOException, JsonProcessingException
{
.... // Start by creating a JsonThing instance and init the list.
while (jp.nextToken() != JsonToken.END_OBJECT)
{
jp.nextToken();
switch (jp.getCurrentName())
{
case "animals":
jp.nextToken(); // Skip to {
jp.nextToken(); // Skip id field
Animal a = jp.readValuesAs(Animal.class);
// Add to list
}
}
..... // Return JsonThing
}
If the keys are not known in advance then use Map instead of POJO.
Have a look at Example 1 and Example 2
You can try any one.
sample code: (using Jackson Library)
TypeReference<Map<String, Object>> typeRef = new TypeReference<Map<String, Object>>() {};
ObjectMapper mapper = new ObjectMapper();
try {
Map<String, Object> data = mapper.readValue(jsonString, typeRef);
} catch (Exception e) {
System.out.println("There might be some issue with the JSON string");
}
sample code: using GSON Library
Type type = new TypeToken<Map<String, Object>>() {}.getType();
Map<String, Object> data = new Gson().fromJson(jsonString, type);
Perhaps it's just a question of combining simple Maps with pojos? Like:
public class Wrapper {
public Map<Long, Animal> animals;
}
public class Animal {
public long id;
public String name;
}
and that's it; although ids there match, maybe there is no need to try to model that dependency.

Telling Retrofit to what variable it should map a certain json field?

The REST API I'm talking to is responding to some of the requests in a structure as such:
{
"_links": {
"next": "NEXT_DATA_BLOCK_URL",
"prev": "PREV_DATA_BLOCK_URL",
"self": "CURRENT_DATA_BLOCK_URL"
},
"RESPONSE_DATA_NAME": [
{
... DATA_FIELDS ...
}
]
}
Where 'RESPONSE_DATA_NAME' is the data "name" - changes according to desired request. for example, it might be 'teams' or 'messages'.
Therefore I created a generic class with the following members:
public class PagedResponse<T> {
public PagingLinks _links;
public List<T> _data;
}
Is there any way I can set up my RestAdapter so that it'll always map 'RESPONSE_DATA_NAME' to the '_data' member, no matter what the field name actually is?
Thanks ^_^
Using gson you can annotate your _data field with the #SerializedName. The parameter (value) of this annotation is the name to be used when serialising and deserialising objects. For example, the Java field _data is represented as RESPONSE_DATA_NAME in JSON.
public class PagedResponse<T> {
public PagingLinks _links;
#SerializedName(value="RESPONSE_DATA_NAME")
public List<T> _data;
}
Further see doc
If you want to control the json field then you have to write custom de-serializer as like below
public class CustomDeserializer implements JsonDeserializer<PagedResponse> {
#Override
public PagedResponse deserialize(final JsonElement json,
final Type typeOfT, final JsonDeserializationContext context)
throws JsonParseException {
Gson gson = new Gson();
PagedResponse pagedResponse = new PagedResponse<>();
List list = new ArrayList<>();
pagedResponse = gson.fromJson(json, PagedResponse.class);
Type listType = new TypeToken<List>() {}.getType();
Set<Entry<String, JsonElement>> enteries = json.getAsJsonObject().entrySet();
for (Entry<String, JsonElement> entry : enteries) {
JsonElement jsonElement = (JsonElement) entry.getValue();
if (jsonElement.isJsonArray()) {
list.add(gson.fromJson(jsonElement, listType));
}
}
pagedResponse.set_data(list);
return pagedResponse;
}
}
finally parse it
GsonBuilder gsonBuilder = new GsonBuilder();
gsonBuilder.registerTypeAdapter(PagedResponse.class, new CustomDeserializer());
Gson gson = gsonBuilder.create();
gson.fromJson(Your_JSON_STRING_HERE, PagedResponse.class);
So I finally found a solution to the problem...
I created a costume de-serializer, which adds the data field to the existing JsonObject, and copies the content of the RESPONSE_DATA_NAME (which is a JsonArray).
Then I serialize it normaly with GSON simple conversion (gson.fromJson()).
It's a bit stupid but it works =P
The de-serializer's class:
public class PagedResponseDeserializer implements JsonDeserializer<PagedResponse> {
#Override
public PagedResponse deserialize(JsonElement json, Type typeOfT,
JsonDeserializationContext context) throws JsonParseException {
Gson gson = new Gson();
JsonElement value = null;
JsonObject jsonObject = json.getAsJsonObject();
Iterable<Entry<String,JsonElement>> entries = jsonObject.entrySet();
for (Entry<String, JsonElement> entry : entries) {
value = entry.getValue();
if (value.isJsonArray()) break;
}
jsonObject.add("data", value);
return gson.fromJson(jsonObject, typeOfT);
}
}