How to parse a json response with multi type values coming for same field? - json

Question

How to parse answerData key from json response in kotlin as it is changing its type in each block? I tried keeping it Any but was not able to type cast. how to parse answerData?
{
"status": "OK",
"data": [
{
"id": 10,
"answerData": null
},
{
"id": 21,
"answerData": {
"selectionOptionId": 0,
"selectionOptionText": null
}
},
{
"id": 45,
"answerData": {
"IsAffiliatedWithSeller": false,
"AffiliationDescription": null
}
},
{
"id" : 131,
"answerData" : [
{ "2" : "Chapter 11" },
{ "3" : "Chapter 12" },
{ "1" : "Chapter 7" }
]
},
{
"id" : 140,
"answerData" : [
{
"liabilityTypeId" : 2,
"monthlyPayment" : 200,
"remainingMonth" : 2,
"liabilityName" : "Separate Maintenance",
"name" : "Two"
},
{
"liabilityTypeId" : 1,
"monthlyPayment" : 300,
"remainingMonth" : 1,
"liabilityName" : "Child Support",
"name" : "Three"
}
]
}
]
}

Answer 1

As commented and explained in other answers you really should ask changes to the JSON format. However it is not so unusual to have list of elements of which the data included varies. For such case there should at least be some field indication the type of data to be deserialized. (not saying it is not an anti-pattern sometimes it might be).
If you reach that agreement it is possible to use - for example - RuntimeTypeAdapterFactory
like explained in linked question (sorry it is Java).
If not you will run into troubles. It is still quite easy to isolate the problem. Not saying it is easy to solve. I present one possible (sorry again, Java but guess it is easily adaptable to Kotlin) solution. I have used lots of inner static classes to make the code more compact. The actual logic has not so many rows most of the code is to map your JSON into java classes.
Make the model abstract in a way that it does not hinder Gson to do its job whatever it heads in that problematic field:
#Getter #Setter
public class Response {
private String status;
#Getter #Setter
public static class DataItem {
private Long id;
// below 2 rows explained later, this is what changes
#JsonAdapter(AnswerDataDeserializer.class)
private AnswerData answerData;
}
private DataItem[] data;
}
As you see there is declared this AnswerData and #JsonAdapter for handling the actual more complex stuff:
public class AnswerDataDeserializer
implements JsonDeserializer<AnswerDataDeserializer.AnswerData> {
private final Gson gson = new Gson();
// The trick that makes the field more abstract. No necessarily
// needed answerData might possibly be just Object
public interface AnswerData {
// just to have something here not important
default String getType() {
return getClass().getName();
}
}
// here I have assumed Map<K,V> because of field name cannot be plain number.
#SuppressWarnings("serial")
public static class ChapterDataAnswer extends ArrayList<Map<Long, String>>
implements AnswerData {
}
#SuppressWarnings("serial")
public static class LiabilityDataAnswer
extends ArrayList<LiabilityDataAnswer.LiabilityData>
implements AnswerData {
#Getter #Setter
public static class LiabilityData {
private Long liabilityTypeId;
private Double monthlyPayment;
private Integer remainingMonth;
private String liabilityName;
private String name;
}
}
#Override
public AnswerData deserialize(JsonElement json, Type typeOfT,
JsonDeserializationContext context)
throws JsonParseException {
if(json.isJsonArray()) {
try {
return gson.fromJson(json, ChapterDataAnswer.class);
} catch (Exception e) {
return gson.fromJson(json, LiabilityDataAnswer.class);
}
}
if(json.isJsonObject()) {
// do something else
}
return null;
}
}
I have above presented only the two more complex array types. But as you can see you will have to check/peek all the deserialized AnswerData in some way to determine the actual type in method deserialize
Now you need still need to know about different types of AnswerData. Maybe there are such types that collide in a way that you cannot determine the type.
NOTE: you can also always also deserialize whole stuff or any object as a Map or Object (Gson will make it LinkedHashMap if I remember correct)
Whether way you do it you still need to check the instance of the object after deserialization what it is and use cast.

Answer 2

The design of the input JSON is terrible and really hard to use.
Let me say that:
it mixes elements and collections for the answerData attributes with dozens of cons against it;
answer elements lack the type discriminator field so the deserialize must analyze each JSON tree to produce a valid deserialized object with another dozen of cons against it (including "there is no way to determine the exact type precisely" and "it may require too much memory because of JSON trees");
Some tools like OpenAPI/Swagger use the discriminator field to deserialize to a dedicated type without doing any heuristics.
Any won't work for you of course, as Gson has no even a single idea what those payloads are supposed to be deserialized to.
Since you didn't provide your mappings, I'll provide mine demonstrating an example idea of how such terrible JSON documents can be deserialized.
This also includes:
using Java 11 and Lombok instead of Kotlin (as it does not really matter as you stated in the notice);
mapping an answer with a list of answers even if the incoming JSON node contains an object instead of an array to unify all of that;
creating a deducing deserializer that naively does some "magic" to get rid of the bad JSON design.
To resolve the first issue, elements vs arrays/lists, I've found a ready-to-use solution right here at S.O.:
#RequiredArgsConstructor(access = AccessLevel.PRIVATE)
public final class AlwaysListTypeAdapterFactory<E> implements TypeAdapterFactory {
#Nullable
#Override
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
if (!List.class.isAssignableFrom(typeToken.getRawType())) {
return null;
}
final Type elementType = resolveTypeArgument(typeToken.getType());
#SuppressWarnings("unchecked")
final TypeAdapter<E> elementTypeAdapter = (TypeAdapter<E>) gson.getAdapter(TypeToken.get(elementType));
#SuppressWarnings("unchecked")
final TypeAdapter<T> alwaysListTypeAdapter = (TypeAdapter<T>) new AlwaysListTypeAdapter<>(elementTypeAdapter).nullSafe();
return alwaysListTypeAdapter;
}
private static Type resolveTypeArgument(final Type type) {
if (!(type instanceof ParameterizedType)) {
return Object.class;
}
final ParameterizedType parameterizedType = (ParameterizedType) type;
return parameterizedType.getActualTypeArguments()[0];
}
private static final class AlwaysListTypeAdapter<E> extends TypeAdapter<List<E>> {
private final TypeAdapter<E> elementTypeAdapter;
private AlwaysListTypeAdapter(final TypeAdapter<E> elementTypeAdapter) {
this.elementTypeAdapter = elementTypeAdapter;
}
#Override
public void write(final JsonWriter out, final List<E> list) {
throw new UnsupportedOperationException();
}
#Override
public List<E> read(final JsonReader in) throws IOException {
final List<E> list = new ArrayList<>();
final JsonToken token = in.peek();
switch ( token ) {
case BEGIN_ARRAY:
in.beginArray();
while ( in.hasNext() ) {
list.add(elementTypeAdapter.read(in));
}
in.endArray();
break;
case BEGIN_OBJECT:
case STRING:
case NUMBER:
case BOOLEAN:
list.add(elementTypeAdapter.read(in));
break;
case NULL:
throw new AssertionError("Must never happen: check if the type adapter configured with .nullSafe()");
case NAME:
case END_ARRAY:
case END_OBJECT:
case END_DOCUMENT:
throw new MalformedJsonException("Unexpected token: " + token);
default:
throw new AssertionError("Must never happen: " + token);
}
return list;
}
}
}
Next, for the item no. 2, a deducing type adapter factory might be implemented like this:
#RequiredArgsConstructor(access = AccessLevel.PRIVATE)
public final class DeducingTypeAdapterFactory<V> implements TypeAdapterFactory {
public interface TypeAdapterProvider {
#Nonnull
<T> TypeAdapter<T> provide(#Nonnull TypeToken<T> typeToken);
}
private final Predicate<? super TypeToken<?>> isSupported;
private final BiFunction<? super JsonElement, ? super TypeAdapterProvider, ? extends V> deduce;
public static <V> TypeAdapterFactory create(final Predicate<? super TypeToken<?>> isSupported,
final BiFunction<? super JsonElement, ? super TypeAdapterProvider, ? extends V> deduce) {
return new DeducingTypeAdapterFactory<>(isSupported, deduce);
}
#Override
#Nullable
public <T> TypeAdapter<T> create(final Gson gson, final TypeToken<T> typeToken) {
if (!isSupported.test(typeToken)) {
return null;
}
final Map<TypeToken<?>, TypeAdapter<?>> cache = new ConcurrentHashMap<>();
final TypeAdapter<V> deducedTypeAdapter = new TypeAdapter<V>() {
#Override
public void write(final JsonWriter jsonWriter, final V value) {
throw new UnsupportedOperationException();
}
#Override
public V read(final JsonReader jsonReader) {
final JsonElement jsonElement = Streams.parse(jsonReader);
return deduce.apply(jsonElement, new TypeAdapterProvider() {
#Nonnull
#Override
public <TT> TypeAdapter<TT> provide(#Nonnull final TypeToken<TT> typeToken) {
final TypeAdapter<?> cachedTypeAdapter = cache.computeIfAbsent(typeToken, tt -> gson.getDelegateAdapter(DeducingTypeAdapterFactory.this, tt));
#SuppressWarnings("unchecked")
final TypeAdapter<TT> typeAdapter = (TypeAdapter<TT>) cachedTypeAdapter;
return typeAdapter;
}
});
}
}
.nullSafe();
#SuppressWarnings("unchecked")
final TypeAdapter<T> typeAdapter = (TypeAdapter<T>) deducedTypeAdapter;
return typeAdapter;
}
}
Basically, it does no deducing itself, and only delegates the filter and deducing jobs elsewhere using the Strategy design pattern.
Now let's assume your mappings are "general" enough (including using #JsonAdapter for Answer to coerce single elements to become lists):
#RequiredArgsConstructor(access = AccessLevel.PACKAGE, staticName = "of")
#Getter
#EqualsAndHashCode
#ToString
final class Response<T> {
#Nullable
#SerializedName("status")
private final String status;
#Nullable
#SerializedName("data")
private final T data;
}
#RequiredArgsConstructor(access = AccessLevel.PACKAGE, staticName = "of")
#Getter
#EqualsAndHashCode
#ToString
final class Answer {
#SerializedName("id")
private final int id;
#Nullable
#SerializedName("answerData")
#JsonAdapter(AlwaysListTypeAdapterFactory.class)
private final List<AnswerDatum> answerData;
}
#RequiredArgsConstructor(access = AccessLevel.PRIVATE)
abstract class AnswerDatum {
interface Visitor<R> {
R visit(#Nonnull Type1 answerDatum);
R visit(#Nonnull Type2 answerDatum);
R visit(#Nonnull Type3 answerDatum);
R visit(#Nonnull Type4 answerDatum);
}
abstract <R> R accept(#Nonnull Visitor<? extends R> visitor);
#RequiredArgsConstructor(access = AccessLevel.PACKAGE, staticName = "of")
#Getter
#EqualsAndHashCode(callSuper = false)
#ToString(callSuper = false)
static final class Type1 extends AnswerDatum {
#SerializedName("selectionOptionId")
private final int selectionOptionId;
#Nullable
#SerializedName("selectionOptionText")
private final String selectionOptionText;
#Override
<R> R accept(#Nonnull final Visitor<? extends R> visitor) {
return visitor.visit(this);
}
}
#RequiredArgsConstructor(access = AccessLevel.PACKAGE, staticName = "of")
#Getter
#EqualsAndHashCode(callSuper = false)
#ToString(callSuper = false)
static final class Type2 extends AnswerDatum {
#SerializedName("IsAffiliatedWithSeller")
private final boolean isAffiliatedWithSeller;
#Nullable
#SerializedName("AffiliationDescription")
private final String affiliationDescription;
#Override
<R> R accept(#Nonnull final Visitor<? extends R> visitor) {
return visitor.visit(this);
}
}
#RequiredArgsConstructor(access = AccessLevel.PACKAGE, staticName = "of")
#Getter
#EqualsAndHashCode(callSuper = false)
#ToString(callSuper = false)
static final class Type3 extends AnswerDatum {
#Nonnull
private final String key;
#Nullable
private final String value;
#Override
<R> R accept(#Nonnull final Visitor<? extends R> visitor) {
return visitor.visit(this);
}
}
#RequiredArgsConstructor(access = AccessLevel.PACKAGE, staticName = "of")
#Getter
#EqualsAndHashCode(callSuper = false)
#ToString(callSuper = false)
static final class Type4 extends AnswerDatum {
#SerializedName("liabilityTypeId")
private final int liabilityTypeId;
#SerializedName("monthlyPayment")
private final int monthlyPayment;
#SerializedName("remainingMonth")
private final int remainingMonth;
#Nullable
#SerializedName("liabilityName")
private final String liabilityName;
#Nullable
#SerializedName("name")
private final String name;
#Override
<R> R accept(#Nonnull final Visitor<? extends R> visitor) {
return visitor.visit(this);
}
}
}
Note how AnswerDatum uses the Visitor design pattern to avoid explicit type casting.
I'm not sure how it is leveraged in Java when using sealed classes.
public final class DeducingTypeAdapterFactoryTest {
private static final Pattern digitsPattern = Pattern.compile("^\\d+$");
private static final TypeToken<String> stringTypeToken = new TypeToken<>() {};
private static final TypeToken<AnswerDatum.Type1> answerDatumType1TypeToken = new TypeToken<>() {};
private static final TypeToken<AnswerDatum.Type2> answerDatumType2TypeToken = new TypeToken<>() {};
private static final TypeToken<AnswerDatum.Type4> answerDatumType4TypeToken = new TypeToken<>() {};
private static final Gson gson = new GsonBuilder()
.disableInnerClassSerialization()
.disableHtmlEscaping()
.registerTypeAdapterFactory(DeducingTypeAdapterFactory.create(
typeToken -> AnswerDatum.class.isAssignableFrom(typeToken.getRawType()),
(jsonElement, getTypeAdapter) -> {
if ( jsonElement.isJsonObject() ) {
final JsonObject jsonObject = jsonElement.getAsJsonObject();
// type-1? hopefully...
if ( jsonObject.has("selectionOptionId") ) {
return getTypeAdapter.provide(answerDatumType1TypeToken)
.fromJsonTree(jsonElement);
}
// type-2? hopefully...
if ( jsonObject.has("IsAffiliatedWithSeller") ) {
return getTypeAdapter.provide(answerDatumType2TypeToken)
.fromJsonTree(jsonElement);
}
// type-3? hopefully...
if ( jsonObject.size() == 1 ) {
final Map.Entry<String, JsonElement> onlyEntry = jsonObject.entrySet().iterator().next();
final String key = onlyEntry.getKey();
if ( digitsPattern.matcher(key).matches() ) {
final String value = getTypeAdapter.provide(stringTypeToken)
.fromJsonTree(onlyEntry.getValue());
return AnswerDatum.Type3.of(key, value);
}
}
// type-4? hopefully...
if ( jsonObject.has("liabilityTypeId") ) {
return getTypeAdapter.provide(answerDatumType4TypeToken)
.fromJsonTree(jsonElement);
}
}
throw new UnsupportedOperationException("can't parse: " + jsonElement);
}
))
.create();
private static final TypeToken<Response<List<Answer>>> listOfAnswerResponseType = new TypeToken<>() {};
#Test
public void testEqualsAndHashCode() throws IOException {
final Object expected = Response.of(
"OK",
List.of(
Answer.of(
10,
null
),
Answer.of(
21,
List.of(
AnswerDatum.Type1.of(0, null)
)
),
Answer.of(
45,
List.of(
AnswerDatum.Type2.of(false, null)
)
),
Answer.of(
131,
List.of(
AnswerDatum.Type3.of("2", "Chapter 11"),
AnswerDatum.Type3.of("3", "Chapter 12"),
AnswerDatum.Type3.of("1", "Chapter 7")
)
),
Answer.of(
140,
List.of(
AnswerDatum.Type4.of(2, 200, 2, "Separate Maintenance", "Two"),
AnswerDatum.Type4.of(1, 300, 1, "Child Support", "Three")
)
)
)
);
try (final JsonReader jsonReader = openJsonInput()) {
final Object actual = gson.fromJson(jsonReader, listOfAnswerResponseType.getType());
Assertions.assertEquals(expected, actual);
}
}
#Test
public void testVisitor() throws IOException {
final Object expected = List.of(
"21:0",
"45:false",
"131:2:Chapter 11",
"131:3:Chapter 12",
"131:1:Chapter 7",
"140:Two",
"140:Three"
);
try (final JsonReader jsonReader = openJsonInput()) {
final Response<List<Answer>> response = gson.fromJson(jsonReader, listOfAnswerResponseType.getType());
final List<Answer> data = response.getData();
assert data != null;
final Object actual = data.stream()
.flatMap(answer -> Optional.ofNullable(answer.getAnswerData())
.map(answerData -> answerData.stream()
.map(answerDatum -> answerDatum.accept(new AnswerDatum.Visitor<String>() {
#Override
public String visit(#Nonnull final AnswerDatum.Type1 answerDatum) {
return answer.getId() + ":" + answerDatum.getSelectionOptionId();
}
#Override
public String visit(#Nonnull final AnswerDatum.Type2 answerDatum) {
return answer.getId() + ":" + answerDatum.isAffiliatedWithSeller();
}
#Override
public String visit(#Nonnull final AnswerDatum.Type3 answerDatum) {
return answer.getId() + ":" + answerDatum.getKey() + ':' + answerDatum.getValue();
}
#Override
public String visit(#Nonnull final AnswerDatum.Type4 answerDatum) {
return answer.getId() + ":" + answerDatum.getName();
}
})
)
)
.orElse(Stream.empty())
)
.collect(Collectors.toUnmodifiableList());
Assertions.assertEquals(expected, actual);
}
}
private static JsonReader openJsonInput() throws IOException {
return // ... your code code here ...
}
}
That's it.
I find it pretty difficult and unnecessarily complicated.
Please ask your server-side mates to fix their design for good (note how the current situation makes deserializing harder than it might be when designed well).

Answer 3

The Json response is wrong. There is no need to handle this response in client side, the Json response should be changed from the server side. Otherwise this is going to be a huge burden for you in future. A Json object should have a properly defined keys and its values.